Transcription
I just finished reading Empire of AI, Dreams and Nightmares in some Outman's Open AI by Karen How, and it's completely changed how I think about what's happening behind AI headlines. This isn't another book hyping up AI capabilities. It's also not a book that tries to convince you humanity is going to be destroyed by AI or something like that. It is actually an investigative, down-to-earth, deep dive into the real costs of building current AI systems, written by a journalist who happened to have extensive access to the inner workings of big tech.
I find this book quite uncomfortable to read at times, and honestly, it's quite brutal and disturbing, but it's filled with insights I haven't seen anywhere else and that OpenAI probably doesn't want you to know. If you love using ChatGPT and AI tools, or just curious about what's really happening in this industry, I think you'll find this book really eye-opening, uh, as I did. So, in this video, I want to share with you some of the most important things that I learned from this book.
So, first of all, AI research has traditionally been an open field. For decades, researchers shared their data, methods, and results freely. Developments of important AI models in the past, like ResNet, were built on collaboration across research labs in the US and China, and this practice helps researchers and companies share ideas and leverage each other's work. OpenAI was founded in 2015 with a clear mission: develop safe artificial general intelligence that benefits all of humanity. The "open" in their name represented a commitment to transparency and shared research. But over time, most people would agree, it can better be called "closed AI."
When OpenAI realized they needed massive computational resources to train ChatGPT, everything changed. They created a for-profit arm, secured billions in funding from Microsoft, and started keeping their research secret. Their justification was that their AI might be too dangerous to share openly. Yet, they had no problem releasing some of these same dangerous systems to millions of users later, sometimes without thorough safety evaluation, which is another issue we'll talk about later. And this shift fundamentally changed the entire AI landscape. It triggered an arms race. It woke up Google, China. It infuriated Elon Musk, who decided to build his own AI. Everyone started racing aggressively to build the most powerful AI and AGI first, without even agreeing on what AGI looks like or whether society is prepared for it. So, the collaborative, open-source spirit in AI research is now replaced by corporate secrecy and relentless competition.
But this shift also had another consequence. It started draining talent away from independent research. In 2004, about 21% of AI PhD graduates went to work for corporations. By 2020, that number had jumped to 70%. Universities simply can't compete with the salaries and resources that tech giants offer. We are talking about compensation packages worth hundreds of thousands of dollars, sometimes millions of dollars for top AI researchers, plus access to computing infrastructure that academic institutions can't match. And the result is that the best AI researchers, the people who should be asking hard questions about AI safety and ethics, are increasingly working for those companies whose primary goal is profit and power, not some altruistic cause.
So, the author argues that this brain drain is severely limiting the diversity of AI research. We are funneling all the talent into one particular approach: making bigger models using more data and compute, while other approaches might potentially be safer or more beneficial directions. And also, to exploit those scaling laws, meaning more data, more computation, larger models, also led to a whole new set of challenges. As OpenAI developed ChatGPT, they ran into a fundamental problem: they were running out of high-quality training data from the internet. So, they started scraping everything they could find. They turned to low-quality sources like Common Crawl, which is literally the dump site of the internet. No one had the time to go through it manually and filter out harmful stuff. They also included content from Reddit threads, forums, and other low-quality sources filled with biased and harmful content. They needed the volume, even if the quality was terrible.
So, now, instead of solving the input problem by finding better data sources and ensuring good quality input that goes into those AI models, they decided to pre-train models, including GPT-3 and GPT-4, with all the data they have, good and bad, and then solve the output problem through content moderation. Basically, trying to tame the model not to spit out harmful stuff to the end users.
And then I came to this part of the book, which is generally hard to read. A lot of people who work in the output moderation exercise come from disadvantaged groups in global South countries. They were temporary contractors, paid as little as a few dollars a day to review horrific content, hate speech, violence, sexual abuse, suicidal thoughts, disturbing imagery, to train AI moderation systems. Karenha interviewed several of those workers, and their stories are heartbreaking. These people had no idea they were contributing to what would become what we now know as ChatGPT. Their existence is completely invisible to the public and received no compensation relative to the value they create. There are some accounts told in this book that they also didn't receive adequate mental health support for doing the work that would traumatize most people.
So, looking at this kind of business practice, Karenha draws a parallel to colonial exploitation, extracting value from vulnerable populations in the Global South to benefit consumers in wealthier countries. It's a comparison that's uncomfortable but hard to dismiss.
Then the book goes on to discuss safety concerns. OpenAI adopted a "move fast and break things" mentality, and moving fast and breaking things also means safety research has to take a backseat. This reminds me of a comment Max Tegmark made in one of his interviews: "Do you know the AI? It's currently the only industry in the US that makes powerful stuff that has less regulation than sandwiches." At some point, OpenAI rushed to give API access to the ChatGPT model to a wide range of businesses and without any safety measures or guardrails. And this led to the use of the model for harmful stuff, such as child sexual abuse and sexual content. And later, somewhere last year, the GPT-4 model was also not thoroughly tested for dangerous capabilities. Given the ambitious launch schedule, there was just not enough time for the safety team to perform proper evaluations, and this worried many researchers at OpenAI itself, and many people in the safety team left the company as a result. Dario Amodei is one example who left OpenAI to build Anthropic with the hope of doing things differently. Chief Scientist and co-founder Ilya Sutskever also left the company later to start a new firm dedicated to safer AI.
The internal conflicts and lack of trust within the company also led to the board firing Sam Altman for not being consistently candid in his communications, citing manipulative power-seeking behavior in attempts to prioritize commercial interests over safety commitments. But then Sam Altman was reinstated five days later after investor pressure. So, it's quite an uneasy feeling to understand how much tension OpenAI has within the company itself. And the important point that the author raises in this book is: can we trust a small group of tech executives to make decisions that potentially could affect all of humanity, especially when their own employees keep raising safety concerns and also when they can't seem to agree among themselves on the direction of the technology?
Another area brought up in this book is how these AI companies handle intellectual property and creative work. AI companies have trained the models on millions of both open-source and copyrighted works from books, articles, artwork, code, anything they could scrape from the internet, because the more data fed into the model, the more capable the model could become. Many individual artists, writers, and coders have raised concerns that their work has been used to train those AI models, creating billion-dollar businesses for big tech without any credit, consent, or compensation for their work. And ironically, these exact AI systems or models are now being used to automate away their jobs. In fact, OpenAI defined AGI as systems that outperform humans at most economically valuable work. So, the ultimate goal is to replace humans in most jobs, including the very people who create that training data. Honestly, I do like the idea of not having to work, but I don't like the idea of stealing people's work and actually automating away the parts of their job that give them meaning.
And many AI companies, including OpenAI, have been doing this while keeping their training data sources secret. None of these companies will tell you exactly what's included in the data that they used, nor can they explain how the original data might surface to the final output. And this makes it impossible to scrutinize. The best argument they have is it was too much of a hassle to connect with every single artist and author to ask for copyright permission, and it was too complicated to come up with a mechanism to compensate them properly. This also actually rings an alarm about how current policies are actually utterly unprepared to deal with copyright and intellectual property where it comes to developing artificial intelligence.
But there's another cost that's even more hidden from public view: the environmental impact of all this computational power. One chapter in this book called "Plundered Earth" is dedicated to this topic. Training large AI models requires massive data centers that consume enormous amounts of electricity and also fresh water for cooling. The book mentions regions like Arizona and some countries in South America. They are already facing water scarcity due to climate change. But tech companies are using millions of gallons of fresh water daily to cool their servers. So, local communities are seeing their drinkable water supplies disappear. And with this, up until now, AI seems to worsen the impact of climate change, which it claims to solve in the first place.
So, this book goes very much in depth into all of these topics and gives really touching, convincing narratives about what's happening. But the most important takeaway from this book, I think, is that none of this is inevitable. Tech companies often frame AI development as an unstoppable force that we just have to accept. But that's not true. These are often choices being made by a small number of companies, small groups of elites in pursuit of money, power, or whatever goals they set for themselves. It is not collective choices and preferences made by larger groups of people that are actually representative of the society as a whole.
This actually reminds me of another book I reviewed a while back called "Power and Progress," which challenges the exact idea that technology is destiny and always leads to shared prosperity. And this book made a convincing case that throughout human history, time and again, we've seen that technology fails to benefit everyone without democratic rights and values and proper policies in place. So, shared prosperity is not automatic. But hopefully, more organizations are recognizing the need for more equitable and sustainable AI development. But ultimately, we need better policies and regulation. We need to create incentives for better business practices and even try to redistribute or distribute benefits more fairly. And you probably agree that something as transformative as AI needs at least more regulation than sandwiches.
So, reading this book, Empire of AI, has made me much more skeptical of public statements from AI companies about responsible development and benefiting humanity. I think I've learned to pay attention to what they don't say as much as what they do say in their polished PR statements and interviews. I do believe companies like Google, OpenAI, and Anthropic are creating insanely cool and useful products, and they're trying their best. And maybe AGI will help us cure terminal diseases, solve climate change, and improve lives. But this book also convinced me that the current path dominated by a handful of profit-driven organizations with minimal oversight, led by small groups of fellow human beings, seems quite dangerous.
So, all in all, this book is an uncomfortable read, but I'd highly recommend it for anyone trying to understand what's really happening behind the scenes in the AI industry, especially around OpenAI. And I'd love to hear your thoughts in the comments below. If you're interested in seeing other book reviews on AI, check out these videos over here. Thank you for watching. Bye-bye.