OpenAI Insider’s Dark Warning on AI & The Hypocrisy of CEO Sam Altman

On this episode, Karen How, the first journalist ever allowed inside OpenAI's offices. She sat face to face with Sam Olman's lieutenants, asking the vital questions that they suddenly struggled to answer. She believes that she saw what seemed a secretive empire in the making, powered by billions, powered by one man's ideology, and then a race to control your future. This is the room where it happened. Firsthand accounts from behind closed doors in business. inside the rooms, feeling the pressure and helping you work out what to do if it's ever your turn at the table. Brought to you by IFS. This is the room where it happened with Karen How.

Well, Karen, let's start by you taking us into the room where you had a conversation with Sam Alman's lieutenants. What did they reveal to you that perhaps made you slightly worried and maybe even changed the entire narrative of what you were going to write about OpenAI?

Yeah. So this conversation happened in 2019 when when I was a reporter at MIT Technology Review covering fundamental AI research and OpenAI at the time was certainly not the company that I was today and didn't have the public reputation that it did today. But it was on my radar because I was trying to find organizations that were really pushing the frontier of AI and and OpenAI fashion themselves as that. And so I embedded within the company for 3 days and the very first interview that I had is the one that you mentioned where I then sat down with Greg Brockman, the chief technology officer at the time, and Ilia Sudskver, the chief scientist at the time.

What I was really curious about was OpenAI fashioned itself as first and foremost a nonprofit working on these fundamental AI research breakthroughs for the benefit of all of humanity. But secondarily, they had this term that they were really popularizing in the lexicon, artificial general intelligence. This idea that they were specifically pushing to advance AI software to a point where it could become as capable, if not more capable, than humans. And the reason why they had to do that was they first of all thought it was possible and second of all believed that they needed to, like the good guys needed to get there first instead of the bad guys. So I was kind of just probing Greg and Ilia in that conversation. What is artificial general intelligence? What do you envision a future like with this technology? How are you actually going to get there? What are the harms or risks of the bad guys getting AGI first? Essentially just asking really, really basic questions about their premise as an organization. They were already three, a little over 3 years into developing OpenAI as it as it was. So, I thought that they would have relatively straightforward answers. And it turns out that they really struggled to answer even those basic questions. And that's when I started becoming a lot more skeptical about whether or not what OpenAI said it was was really what was happening beneath the surface.

>> What was the thing that was said where you went, "Wo, hold on. That's not what I thought you were doing here."

There was nothing specifically that they said where I just had a shock factor moment. It was more like, you know, I I when I one of the first questions I asked was, why are you spending billions of dollars on the problem of artificial general intelligence instead of something else? You know, you're saying that you could build these systems to then solve, solve climate change, but why not just deploy billions of dollars on existing climate technologies that we know are already going to make a dent on this particular problem? And they couldn't even really articulate that. Like I remember Greg paused for about 15 seconds and then said, "Ilia, maybe you take this one." And and so it was little moments like that. And then after that meeting where we had kind of it was it was 45 minutes roughly and we kind of went around a little bit in circles where basically the answer to every question ultimately ended up with, well, OpenAI's mission is to build artificial and general, uh, is to ensure artificial general intelligence benefits all of humanity, and that is what we're doing. And and I thought the logic was very cyclical. Like no matter what question I asked, it kind of always went back to the mission statement. So then after that meeting, I started interviewing researchers and other executives within the organization over the course of the 3 days. And another thing that I started noticing was people said that the fact that no one could really define what AGI was was a feature, not a bug. This was very much a scientific endeavor. They were exploring the frontiers of science to figure out the shape of what this technology would look like. Which sounds reasonable until you consider the fact that they're deploying billions of dollars along this, the production of this technology that they are then promising is going to potentially bring utopia. And that is how they're continuing to accumulate those billions of dollars and continuing to get public license to forge ahead and and create this technology. And then the other thing that I noticed was that the organization was weirdly secretive for what a nonprofit, a a company that professed to be still a nonprofit that was meant to do research that they would open source to the public. They would not let me interview people without a comms person present. They wouldn't let me go just walk around to different parts of the office. And it was all because they were worried that I would see something that was not sanctioned to be publicly o opened up to the public. And and it was just it was just odd because the premise was that everything would be opened up to the p public. So, there were all of these just little moments that ultimately amounted to me wondering whether to actually take what they said about themselves at face value. And then after my time within the company, I continued to source interviews on my own with people who worked there. and they started to confirm that there was a disconnect between what the brand and the veneer of OpenAI was and what they actually did behind closed doors.

>> If you haven't already subscribed, please do right now. It means you don't miss an episode of The Room Where It Happened and it helps us attract the brightest and best new brains to the show. We'll carry on the conversation in just a moment. But first, if I asked you to picture AI in action, I reckon you'd think of a large language model or something that you think optimizes your own personal productivity. But have you stopped to think about the impact that AI actually has on the big industries? Industries like aviation, manufacturing, construction, and energy. The kind of businesses that we interact with every single day that rely on AI without us even being aware of it. Let's take service management. Did you know 2 billion people a day use elevators and walkways? Maintained and serviced with IFS.AI. Industrial AI is IFS.AI. I've been working with IFS for years. I know them really well. I've seen the impact up close and personal and I'm so proud that they're partnering with Room where it happened on this show. And if you're like me and your business deals with assets and you want to deliver the very best asset service to your people and your customers, please check them out at ifs.ai. Proud partners of Room where it happened.

And the reason why I think this is a really important conversation for us to have, right, is that there will be some people listening to this going, "Well, they're a business. You they don't have to tell you everything that they're doing when they're running their own business. Like, you can't go and get every single secret from McDonald's or Coca-Cola or Nike." But this is not just any other business. This is the business that is effectively controlling the future of our planet and therefore the future of humanity. It's a business that we're so excited to dive into, so excited to use, so excited to work with. It's going to transform business, transform relationships, transform the world. So, I feel like you that we do need to know like what is the end game here? Like, what are you doing with all of our data? What are you doing with the hours that we spend on your platforms trying to run our lives using your systems? So, do you feel that at any time, Karen, you have the answer to this? Have you have you worked out whether they just didn't really know or whether they are and were then actually doing things they just don't want the likes of you and I to know?

>> It's a great question and I completely agree with everything that you mentioned that that this business is is not any other business and the lack of transparency does have profound consequences and ripple effects. I would also add that it's funny because when I was embedding within OpenAI, it was actually not a business. It was a nonprofit. So, and it continues to kind of this this is like a key tension all the way up until present day where there are still people within the organization that are still not quite convinced that OpenAI is a business. Um, but

>> Do you think it's a business?

>> Do I think it's a business? 100%. But I also think that to understand it purely as a business to your point is insufficient for really understanding the the full shape and texture of of its operations. So in my book, Empire of AI, the reason why I call the book Empire is because these companies I really think need to be thought of as new forms of empire. They're not just operating in the business sphere. They are also operating in the political sphere. They're accruing an enormous amount of social, political, economic capital. They're accruing enormous amount of land, energy, water, raw resources to build this technology. One of the driving motivators behind what OpenAI does is a kind of this quasi-religious ideology that artificial general intelligence is right on the horizon and it's somehow going to bring profound benefit to humanity. And this is also very imperial in nature. You know, the empires of old, they also had they were driven by a religious ideology of we need to bring progress and modernity to all of humanity. And that's why we need to engage in grabbing all this land, grabbing all these raw resources, exploiting labor, building these technologies that ultimately fortify the empire and could potentially end up exacerbating global inequalities. Um, and that is essentially what is also motivating OpenAI and it is because that because the the driver is not just financial, it is also ideological. I think different people that you talk to within the organization will have totally different interpretations of what they're doing and why because it is every person kind of has their own personal value system that they're using as a prism through which to see OpenAI's mission and how to achieve it.

And did you ever have an opportunity to sit down directly with Sam Alman and say, "What is the endgame here? What is this all about?"

>> No. Sam Alman never agreed to sit down with me, but you know, other journalists have asked him that question. And he's effectively said different stories depending on what is relevant of the moment. So when he's in front of Congress or the UK government and he's talking about what why OpenAI shouldn't be regulated and and needs to continue pushing forward and advancing the cutting edge, he'll evoke these ideas of AGI will ultimately resolve poverty. It will cure cancer. It will bring it will completely resolve all of our climate problems and paint these really far-fetched but beautiful visions of a possible future world of of a a profound abundance. Other times when he's talking with journalists, he will kind of just turn into an awesome. I'm just trying to build like cool products and trying to help people and trying to create a super assistant that does amazing things in your life. And so the the story kind of shifts depending on what his audience is and it ultimately doesn't, you know, it doesn't actually get to the core of the revealed preference of what OpenAI's actions show.

What do you think is the endgame from all the research you've done, the conversations you've had, the the rooms that you have had access to that people like me haven't? What do you think are they are they aiming for here?

>> It's very complicated. Depends on who you are talking about within the organization because ultimately, you know, an organization doesn't have a single endgame. It's sort of a collection of different people with different endgames. In terms of talking about Sam Alman's endgame, you know, I I cannot ascertain explicitly what his endgame is, but it is not lost on me that every single tech billionaire now has an AI outfit. That is an important fact to just understand like and what I would gather from the fact that they all have an AI company and they all are trying to build a chatbot is that they see this technology as some kind of ultimate quest and if they can control developing their version of AI, they can make it in their own image and that is going to help them to continue to accumulate significant power and potentially more wealth and more control over the future of how humanity is going to go. Other people within the organizations, you know, I mentioned this word quasi religious. There are people who just they think that they are creating a a digital god or protecting us from a digital demon. And that this is their calling, that they were brought to this earth to usher in a new form of intelligence that will be the next stage of the human species or or the next continuation of the human species. And so the organization OpenAI as a whole ends up being a strange conglomeration of all of these different beliefs, ideologies, endgames, and the manifestation is that it ends up having this incredibly imperial-like conquest where it's trying to dominate the whole world.

You see, one of the things that I'm thinking about here, Karen, is I don't know what the other alternative is because we have uh OpenAI which feels like it's tied quite closely to one man's belief and ideology and the direction that he wants it to go. And we have all these many other people working within the organization that might have different ideas, but someone has to be driving the direction of OpenAI. Someone has to be deciding what our future looks like. Someone has to be thinking, right, what's the most valuable thing this can do for humankind? Someone has to be looking at the biggest risks that are posed to us as as a species and wondering if AI can solve them. So if it isn't these people then then who is it? And and is there a system do you think whereby we as the human beings that are going to be impacted by this deserve to have some kind of say over the people that are making these decisions for us?

Yeah, I mean this is my fundamental critique of the tech industry in general and OpenAI in particular is that the way that I I agree with you that there there needs to be people that think deeply about the challenges that we face and how to get to a better future. You know, we all want to improve our quality of life. We want better healthcare, better education. We don't want war anymore. We want peace. These are all things that there need to be people thinking about, but they should also be elected representatives that represent all of us. And there should be a democratic form of governance around these extremely consequential issues and how they should be resolved and how we tackle them. And in general, that's just not at all how Silicon Valley operates. OpenAI doesn't operate that way. Meta doesn't operate that way. Google, Microsoft, Apple, they're they operate in a much more techno authoritarian way where they dictate to billions of users or in OpenAI's case, hundreds of millions of users exactly what their tech like these these profoundly influential algorithms are going to do and integrate into their lives and and shape the course of their healthcare, their economic opportunities, their educational opportunities. I mean ultimately like we we need to have a say in these things that are molding who we become as a society. And I think that maybe at the very beginning of this journey, we didn't feel that we needed a say because we were sold this dream that these are people building structures to protect us as human beings, right? AGI was going to solve all of our problems for us. So let us get on with it and we will do it. And actually, I wonder whether the more you've looked and the more that you've scratched and the more you've seen below the surface, you think that actually that's some really smart branding cloaking a kind of classic Silicon Valley business.

>> Yes. Short answer. One thing that's really interesting about Altman as a person is he's extremely strategic. And you know, both his detractors and his boosters agree on this, that he's he's very long-term strategic. And I never got a smoking gun or found a document or whatever that said that his intention from the very beginning of co-founding OpenAI as a nonprofit was to just ultimately shed the nonprofit. But there is something to be said about how one if you are an incredibly strategic person, how you would go about building an organization like OpenAI. So in the beginning, OpenAI was co-founded at the end of 2015. At that time, Google was the dominant AI player on the block. It already had a monopoly on some of the best AI researchers in the world. And so OpenAI, in order to catch up to Google, the main bottleneck was talent. They needed to figure out a way to appeal to some of these top AI researchers to leave Google or new new graduates that were coming out of PhD programs to not go to Google and go to OpenAI instead. And so from the early days in those first few years, OpenAI's main strategy, they couldn't compete on compensation. And so they competed on a sense of mission and the nonprofit was a really convenient signal of how committed OpenAI was to that mission. And Ilia Sudscover, the chief scientist, who was a very critical hire in the early days of OpenAI that legitimized the organization. He explicitly came to OpenAI because of that mission. He took a huge pay cut to come to OpenAI. And once OpenAI then had its team, the bottleneck shifted. Then the bottleneck became about capital because now it had all the it had a core group of really smart people and it just needed to build. And they specifically chose to advance AI development in one of the fastest ways possible, which is to simply pour more data and use larger supercomputers to train a single AI model. And supercomputers are really expensive. So that's when they thought, nonprofit doesn't cut it anymore. We're going to put a for-profit arm within this nonprofit and use it as a vehicle to raise billions of dollars. And if you look at that, you're like, well, that kind of seems like it was the strategy all along, you know, like you choose you first design the organization to tackle one bottleneck and then once you've solved that bottleneck, you morph it to tackle the next bottleneck. And it's really convenient that this original nonprofit mission-driven ethos continues to ensconce your organization and kind of warm fuzzy feelings so that when you make mistakes or whatever, you can play or appeal to those original roots and massage the bad PR and turn it into good PR.

>> And another layer to this of course is the relationship between OpenAI and Microsoft. And I wonder what you think about this, that you know, they are so closely aligned. Um, to what extent do you think that reliance on Microsoft, the funding, the infrastructure, the political leverage is an issue?

So interestingly, Microsoft and OpenAI are very not aligned now as much as uh as much as Alman pretends otherwise. But certainly when they were in the beginning, they were extremely aligned as organizations because OpenAI could not build its own supercomputers and needed someone to build them. And Microsoft had been struggling at that point to catch up to Google in the AI race. And so it was looking to supercharge its capabilities. And after debating whether or not to try and just supercharge it by cultivating strengthening its own internal AI research efforts, it decided that it was just easier to acquire essentially, quote unquote, acquire another lab and get that helping hand externally. And that was absolutely a very powerful alliance in that you have an organization that is an extremely successful, highly profitable tech giant backing another organization that has this quasi-religious mission and fervor for pursuing advanced AI. Now, we're seeing the relationship really fracture because ultimately these are two very different organizations. Microsoft is at its core a business and OpenAI, as I mentioned, it doesn't really fully operate as a business. It also operates as a very ideologically driven actor. And so as the business case for AI has started to weaken and evidence has started to chip away at the original high-fluting claims that AI was suddenly going to grow the economy by 10%, solve all these problems and so forth. Microsoft has begun to pull back while OpenAI has continued to lean forward. And so now there's this friction where they still are contractually legally linked to one another, but there's been a ton of really great reporting that has shown that OpenAI is really desperately trying to extricate itself from this relationship because now it feels like it's beholden to Microsoft's slower pace.

>> So interesting, Karen. I mean, you have you have dedicated your life, I imagine, right, for the last few months and years on this. I started covering AI seven years ago and yeah, it's it's become an obsession.

>> So the question for the likes of me and many people that will listen or watch this episode is with everything that you've seen, what what are the what is the thing or the things that worry you the most?

My biggest concern is that these companies, OpenAI, but all of the other Silicon Valley giants and tech and AI giants, they have grown so powerful that they really are beginning to completely warp and distort democracy. They are able to make, you know, they can acquire land, create digital currencies, um, take people's energy and water, hike up their utility bills, distort people's educational and economic opportunities just with the wave of a hand based on what they decide to deploy in their products one day to the next based either on business or ideological grounds. And now we're seeing in the US this strategic alliance between Silicon Valley and the US government, which was arguably originally the only entity that h that was more powerful than Silicon Valley and could actually regulate these companies. And now under the Trump administration, we see absolutely no interest from the US government in in playing that countervailing force. And so what I see as the historical parallel is really the British East India Company and the British Crown where you had a corporate empire and a nation-state empire that were strategically aligned and benefiting from one another in a way that really ultimately led to a lot of exploitation and harm around the world. Um, and so that is my greatest fear is that if we continue to be on this trajectory where these companies get more and more and more and more powerful and more and more and more wealthy where they can just buy up whatever they want and and knock over whatever whatever obstacles in their way that we are going to lose the ability to collectively self-determine our futures and that's when democracy dies.

And why do you think the US government or other governments maybe are not willing to play the counterbalance with these organizations?

>> I think the US government very likely sees these companies the same way that the British Crown saw the British East Indian Company as as an empire-building asset to their own imperial ambitions. You know, the US government has made under the Trump administration very clear that they want to rebuild the American empire and expand and strengthen it. And so they are they see these American tech companies as tools for doing so as quickly as possible. So they're striking up these deals where they're trying to deploy OpenAI's technologies all around the world and they're trying to help OpenAI build more data centers to facilitate that. And you know, there was an op-ed in the Financial Times from a former Dutch parliamentarian, Marath Shake, who mentioned, you know, isn't isn't it convenient that the US government is trying to install all this American hardware and software all around the world that they could then in their imagination turn on and off at any time. So I think that is part of the reason why the US government is is unwilling to do anything to check these companies because it it believes that it can maintain control over these organizations and continue to use them as hands to perpetuate like their own political will.

>> And why does this seem to worry you so much more than the way the world has looked for the last 20 years anyway? Because if I think about it, I don't know what the other alternative is because we have uh OpenAI which feels like it's tied quite closely to one man's belief and ideology and the direction that he wants it to go. And we have all these many other people working within the organization that might have different ideas, but someone has to be driving the direction of OpenAI. Someone has to be deciding what our future looks like. Someone has to be thinking, right, what's the most valuable thing this can do for humankind? Someone has to be looking at the biggest risks that are posed to us as as a species and wondering if AI can solve them. So if it isn't these people then then who is it? And and is there a system do you think whereby we as the human beings that are going to be impacted by this deserve to have some kind of say over the people that are making these decisions for us?

It's the scale. So even even compared to social media, which already we can argue was problematic in its scale. AI is way worse. So if we look at Meta as an example, which both existed in the social media era and now is trying to refashion itself as an AI company, you know, under the social media era, Meta accumulated around 4 billion user accounts worth of data. When it came time for them to develop generative AI and they wanted to outcompete ChatGPT, the New York Times reported that there were internal conversations of executives lamenting the fact that they did not have enough user data to do that, which is bananas. Like they had 4 billion user accounts of data at that point. And so they were talking about, in addition to that data, what about if we bought out Simon and Schuster for all of their books so that we could train a generative AI system? And what if we continue scraping the internet? And what if we continue acquiring all this other data? And in the end, they didn't buy Simon and Schuster. They just tormented all the books from the dark web and used it to train their AI models instead. So that is just one example through the lens of data for understanding how the scale has really shifted here. But then the other thing is Meta is also, you know, they used to build data centers as a social media company and now they also build data centers as an AI company. Mark Zuckerberg has been boasting the fact that their AI supercomputers will be roughly the size of Manhattan. Like one of their supercomputers will be roughly the size of Manhattan and they will use the energy draw of Manhattan. And that is something that is unprecedented and that is single-handedly currently reversing an extraordinary amount of climate progress that we made in the last decade because the amount of energy that we now need to add to the global grid is equivalent to adding half to 1.2 two times the amount of energy used in the UK annually onto the global grid in the next 5 years. That's how much we would have to add to continue to support this current pace of growth. And most of that will come from fossil fuels, from coal plants that were meant to be retired, from gas plants that were meant to be retired. So this the the scale has just completely exploded to an unfathomable degree. And and sometimes I feel like the the size and scale and numbers that these tech CEOs throw out at people is purposely so large and extraordinary that the average person in the world has never encountered something of that magnitude and therefore it just becomes incomprehensible. And that's how that's part of the strategy of how they can slip all of this under the radar. You know, like what does it even mean to have a supercomputer the size of Manhattan? I mean, that's just like you can't even begin to really think about that.

On the Room Where It Happened, we speak to our high-performance CEOs, founders, industry leaders, and there's one thing that they all have in common. They know the value of energy, of clarity, of consistency. But for years, I struggled with that myself, especially actually when it came to sleep. But then I found this. This is Vitaly Heights. Just two capsules a day packed with 20 essential nutrients, omega-3s, vitamin D3. This is backed by over 1,400 scientific studies. And honestly, after a couple of weeks, I noticed a genuine change. Deeper sleep, sharper focus, more energy when I was at work. And my sleep monitor backed it all up. Because the truth is, in business, you can't afford to run on empty. So, if you're looking to show up at your best, start with your sleep. Head to heights.com/room or use the code room 20 and you'll get 20% off your first subscription order or just click the link in the description.

It's so interesting and good to think about it because all I see at the moment is people getting excited that they can bring pictures of their loved ones back to life. They don't actually think, well, what's the other side of the coin for these kind of strange little games that we're playing? What's what's the real game that's actually being played under the surface? And when you were embedded with OpenAI and you had the chance to ask them about the environmental impact, you know, they say that AGI is going to save the world. Well, this a supercomputer the size of Manhattan is the kind of thing that ends up killing the world, right? So, what was their answer to those questions from you?

>> Yeah. So at that time, there had just been a paper that had come out that I mean, we're talking like this in 2019. Large language models then do not even qualify as large language models today. We're talking about around four orders of magnitude of increase in the amount of data and and the size of the super supercomputers that are now training the models today. But even then, OpenAI had just developed GPT2, which was a generation and a half before ChatGPT. And there had already been a paper that was calling out the fact that these types of language models were incredibly carbon intensive. And so I asked, you know, you're saying that AI is going, you could potentially get to an AGI that solves climate change, but also along the way, there are not potential environmental consequences. There are documented real environmental consequences. And so you're saying that you'll be able to get to AGI, this theoretical possibility, before the real consequences of the process of getting there just completely take over everything. And Greg Brockman said, "I wouldn't go too far down that rabbit hole."

>> That answer in itself tells you quite a lot, doesn't it?

>> Yeah. And um and and, you know, Ilia Zetsker later said to Cade Mets, who is a New York Times reporter, longtime chronicler of of the AI industry as well, "This technology will become so useful that the earth will be covered in supercomputers and power stations." And he considered that to be like a good future that we'll just be surrounded by data centers and power stations no matter where we go.

And in many ways, Karen, you know, the more you look, the more you find. So, we can talk about the, you know, the carbon that's created. We can talk about the amount of water that needs to be drawn to keep these supercomputers cool. We can talk about supercharging mines and e-waste, which is the byproduct of what you've just mentioned. We can talk about AI-enabled weapons and suddenly we could live in a world where AI is making decisions that at the moment human beings are making around warfare and other things. And we can of course talk about, as you have, concentrating AI power among a very, very small group of individuals and organizations. But if I'm listening to this, I'm thinking, well, hold on, what about all the great stuff? What about the advances in treatments for awful diseases or trying to create food in countries where people are starving to death or all of these issues that have plagued humankind for hundreds of years? Is there not a a part of you that looks at it and goes, "Well, you know what? There's going to be a cost to AI, but maybe it's a cost that we should bear for the good that comes."

There are many different types of AI and I I often use the analogy that AI is like the word transportation where you can have a car, a bicycle, a rocket, all different forms of transportation, but they have very different cost-benefit tradeoffs. And what these companies are trying to do is build the rocket version of AI, which arguably we do need some rockets for some things, but we don't need everyone to have a rocket. You know, if you were commuting from Dublin to London by rocket, like people would be like, "What are you doing? This is completely disproportionate to the task." And there are all of these other types of AI technologies that are more like the bicycle version of transportation where it's much more efficient. It's much more task-specific. It does one thing really well, like the advancing medical and drug discovery. So, DeepMind, before it got onto the AGI train, they were working on task-specific models such as one called AlphaFold, where AlphaFold is just a system that does one thing well. It predicts the protein folding structure from a sequence of amino acids. It's not trained on the internet. It's not trained on everyone's chat interactions. It's just trained on amino acid sequences and protein folding. Because of that, it's able to do this task with very, very high precision and that is an important task for ultimately drug discovery, for understanding disease, for improving healthcare, and it ultimately won the Nobel Prize for chemistry last year. And so when people say, "Well, isn't AI going to bring us potentially all these benefits, so the costs are worth it?" It's actually a false trade-off because we're talking about totally different AI systems here. There's plenty of AI systems that already have demonstrated benefit in things like healthcare, education, climate, sustainability, so on and so forth, that actually have extremely small costs in terms of their development, both both in terms of data and the environmental impact. They do not need to be trained on supercomputers the size of Manhattan. They can be trained on powerful laptops in some cases. Whereas the ChatGPTs of the world where which we're being told is going to bring us all these benefits have really not demonstrated much in those in results in those areas, but they certainly have already created an extraordinary, you know, amount of consumption, resource consumption, energy consumption, water consumption in their creation process.

>> Look Karen, your book is blowing up across the states. People are talking about the things that are in it. But I suppose the most important question is, what now needs to be done? If your book has one impact, what would you like that impact to be?

I really want people to understand that technology development is not inevitable and that it is also not just the purview of people that work in tech companies. Actually, every single person, regardless of where you sit in society, can have a say in technology development because when you think about how these companies develop these AI models, they actually need because it's so resource-intensive, they need access to a whole host of different ingredients that are owned collectively owned by different groups in society. You know, the the data that they train on. There are that's the data of artists and writers and people posting on social media, people posting their kids' photos on platforms like Flickr. That and now we're seeing significant pushback from some of those groups saying, no, we're going to sue you because we can't accept that you're just going to take our IP that we have we have copyright protections on this work. And that is them reclaiming ownership over a key ingredient that this company needs access to. We're seeing hundreds of protests breaking out around the world with people pushing back against data center development because they're saying, we don't want to host a data center in our community. It's hiking up our utility bills. It's creating all these other environmental impacts. It's reducing our air quality and it's stalling it's successfully stalling some of these projects or at the very least forcing companies to sit face to face with these communities because usually they don't. They just strike these deals behind closed doors, plop that massive data center into their environment, and then the community has no say about it. We're also seeing students and teachers have huge discussions about whether or not we actually want AI in education, whether AI is actually enabling better education or it's just degrading people's critical thinking skills and what are better ways of integrating AI into the school environment. You know, to give OpenAI a little bit of credit, they have been responding to that criticism that ChatGPT might be eroding students' critical thinking by adding a study mode to their chatbot that was in direct response to the huge amount of criticism and blowback and discussion that people started having around this particular issue. So like all of these, I I often look at like AI development. There's a supply chain just like any other product. It's a digital technology so people don't really think of it as having a physical supply chain. But just like the fashion industry, there is actually a physical supply chain there. There are all of these different ingredients. There are all of these different places where AI is either used or people decide to not use it. And if we consider these as different sites of democratic contestation, then we'll if and we have different movements, different groups that are pushing to actually assert what they want out of the technology, how they want the technology developed, all in concert with one another. Companies have to respond and I hope that that is the one thing that people take away from my book is that they have an active role to play in shaping our future.

And if we believe that it's the role of government to step in and solve these problems for us, do you believe that will happen?

>> I don't think people should wait for government. The beautiful thing about democracy is that even when there is an absence of leadership at the top, there's leadership from the bottom.

And I think with all these things as we wrap up, I think there were people listening to this going, "Well, I'm not too concerned because I'm not sure it really affects me." And look, this is a business show, right? So let's say someone in the UK is starting a business. It might not even be in a tech business. How quickly could that person find themselves in their new business in competition with or even facing challenges from those big AI firms?

>> It's a great question. It depends on the industry that they're in and it also depends on like the structure of their business, whether they're going for quality or quantity. You know, one of the things that is kind of an interrelated question that is on many people's minds is how quickly am I going to lose my job to AI? And it's not you. It's not AI suddenly replacing your job. It's not AI suddenly replacing your business. It's executives that decide to fire layoff workers. And ultimately, if you are an executive of a business, like it is your decision to decide like how you are going to compete. You know, I've found that people have now that they've gotten really, really tired of AI slop and poor quality services that are being AI run, that they're gravitating towards higher quality services. So orient your business towards the things that AI can't do and beat out the competition in that way. But but I I do think that even as certainly AI is going to throw a lot of interesting challenges at you and create a lot more pressure in the business environment for you to succeed, that there's still quite a lot of maneuvering that you can do to outperform.

And to finish with a question about Sam Alman. I mean, they're very good at talking about things that I think will excite people. But I saw a headline saying, "Artificial intelligence will generate enough wealth to pay each adult $13.5,000 a year." And I can imagine a lot of people suddenly thinking, "Hold on, that's amazing. I'm going to get rich." But here's a quote from someone that I'll finish with. I'd love your reaction to this. He says, "Productivity gains from AI could make the society of the future much less divisive and enable everyone to participate in its gains." Do you think that's what we'll see?

Well, what I know is what we're seeing now, which is that the AI boom has created more billionaires at a faster rate than ever before and has left new college graduates with an inability to find economic opportunities. So what I see currently is that is creating more inequality, more divisiveness, and that should be the strongest signal for what would come next if we don't take our future in our own hands and change the trajectory of AI development.

Karen, there is so much talk about the exciting things that AI is going to bring us. I think it's a really important conversation for you to show us the other side of the coin. And it's not just um guesswork. You've been in there. You've spoken to the people. You've seen the whites of their eyes. You've spent years researching this. You've put in the hours. Um, and I think people should listen to you. So, thank you very much for your time.

>> Thank you so much, Jake.

So, I think the biggest thing that I thought about having that conversation with Karen is how I feel like I have just seen one side of AI. All I've thought about is how does it benefit me? How can it benefit my businesses? How can I work faster? How can I work smarter? How can I get it into the hands of my teams? I've genuinely not stopped until this conversation and thought to myself, what is the cost of this? What is the other side of that coin? How risky and dangerous is it that these companies, these AI businesses, which are without question going to be a huge part of our future, are controlled by very few people? How worried should we be that big governments are not stepping in to control it, to regulate it, to protect us as individuals? But at the same time, I felt inspired by the fact that she was saying that we as individuals who seemingly are often powerless actually do have the power to make a real difference here. And I don't want you to leave this thinking, well, that was a scaremongering conversation about AI. No one is saying that AI isn't going to make an incredible difference to your life, to my life, to the lives of everybody around us. But it's really important we also have these conversations so we understand the cost. We understand the people who are behind these movements and we understand that it's not all going to be good in the future. There are going to be things that you and I really need to think about and think about carefully. Thanks so much for joining me. I hope you feel you've got loads to take away from being in the room where it happened. If you can just take a couple of seconds to subscribe, it means you get each episode directly to your inbox. And it also means we can attract more incredible guests like George Heaton, Rory Sutherland, or Richard Branson. Thanks for joining us. See you next time.

[Music]