Transcription
There is a system of power that has been constructed where these companies and the people running these companies get to make decisions that affect billions of people's lives around the world, and those billions of people do not get any say in how it goes. The industry is designed in a way that is extremely inhumane.
Guys, Karen How, one of the top AI researchers, dropped a pretty eye-opening interview on AI and its dangers. During the interview, Karen warned that we should stop believing the tech companies that the AI they're making would stay aligned with humans. She said they're intentionally allowing AI to rewrite its own code to achieve AGI, and this could become an existential threat to humanity. It could lead us to a major catastrophe in the coming years.
Guys, I believe the main issue right now is the complete loss of control of the government over these tech companies. Big tech is constantly training AI to replace humans from their jobs. And I believe the biggest threat we're going to face in the next couple of years is mass unemployment because once mass unemployment begins, it would lead to civil war and all sorts of chaos. If a huge fraction of the population has nothing to lose and nothing to gain, it'll be a collapse of society.
Now, I'm going to play some clips from this interview. I'll give my opinion as we go. In the first part, Karen explains why AI companies are blindly running the race of AI. Let's watch.
So, why are they doing it? I mean, you've interviewed all these people. I think it's what, 300 people in total, 80 or 90 of them from OpenAI, the maker of CHACHBC. Why do you think they're doing it?
I think it's because they're driven by an imperial agenda. And that is why I call these companies empires of AI.
What do you mean by an imperial agenda? What does that term mean?
Empire is the only metaphor that I've ever found to fully encapsulate all of the dimensions of what these companies do and the scale that they operate and what motivates them to do what they do. And there are many parallels that you see between what I call the empires of AI and the empires of old. They lay claim to resources that are not their own in the pursuit of training these models. That's the data of individuals, the intellectual property of artists, writers, and creators. Their land grabbing in order to build these supercomputer facilities for training the next generation models. Second, they exploit an extraordinary amount of labor. They contract hundreds of thousands of workers all around the world, including in the US, to ultimately make these technologies. We can talk about that more. And they also design their tools to be labor automating so that when the technologies are deployed, it also affects labor rights because it erodes away labor rights. And this is a political choice that they have. Third, they monopolize knowledge production. So they project this idea that they're the only ones that really understand how the technology works. And so if the public doesn't like it, it's because they don't actually know enough about this technology. They do this to the public. They do this to policymakers. And they've also captured the majority of the scientists that are working on understanding the limitations and capabilities of AI.
You think they're gaslighting the public in a way?
They are. Yeah. So if most of the climate scientists in the world were bankrolled by fossil fuel companies, do you think we would get an accurate picture of the climate crisis?
No.
And in the same way, they employ and bankroll the AI industry employs and bankrolls most of the AI researchers in the world. So they set the agenda on AI research in soft ways simply by funneling money to their priorities so that only certain types of AI research are produced, but they also will censor researchers when they do not like what the researcher has found. And so I talk about the case of Dr. Timnit Gebru in my book, who was the ethical AI team co-lead at Google when she was literally hired to critique the types of AI systems that Google was building. She then co-wrote a critical research paper that was showing how large language models specifically were leading to certain types of harmful outcomes. And in an attempt to try and stop this research from being published, Google ended up firing Gebru and then fired her other co-lead, Margaret Mitchell. And so they control and quash the research that is inconvenient to the empire's agenda.
Did you have an example where this is happening to journalists as well that are asking questions of their team members? I think I was watching a video of yours where there was a young man that was saying he had someone show up at his door, knocked on his door and asked for information, emails, text messages, and this person was from one of the big AI companies.
This was OpenAI. They started subpoenaing some of its critics. Yeah. Um, as a, as part of a what appears to be a campaign of intimidation, but also what appeared to be a campaign of fishing for more information to figure out to map out the network of critics further. But this was a man who runs a small watchdog nonprofit, and they had been doing a lot of work during that time to try and ask questions about OpenAI's attempt to convert from a nonprofit to a for-profit. Ultimately, OpenAI was successful in that conversion. But during the period where it was sort of existential for OpenAI to complete this conversion, there were a lot of civil society groups and watchdog groups like MIDAS who were trying to prevent the process from happening in the dead of night. They were trying to get more transparency. They were trying to have more public debate about this because it's unprecedented. And it was then that, um, there was a knock on his door and he was served papers.
What did the papers say?
The papers asked him to reproduce every single piece of communication that he had had that might have involved Musk. So this was like this strange paranoia that OpenAI had that Musk was somehow funding these people to block the conversion. None of them were actually funded by Musk. So in this particular case, their request, he simply was just answered, you know, I, I don't have any documents because this doesn't exist.
Okay. Next, Karen explains how the minds of CEOs who are building AI actually work. Guys, one thing I want to add here, and this is just a personal thought. I believe AI would eventually end capitalism. Capitalism may work at first, but eventually, there will be no circulation of money due to robots taking jobs, which would cause a complete collapse of capitalism as we know it. And obviously, a few new financial systems might emerge out of all this. But the question is, who benefits from it? The way a handful of people are controlling AI, the future looks pretty dystopian.
Do you think the people building these AI companies believe that the outcome is going to be all good now? Do you think they think that it's going to be, it's going to serve everyone? It's going to be the age of abundance. Everything's going to go up? Well, what do you think they believe? What do you think Sam believes?
So, so this is so funny. It's such a core part of the mythology that they create around the AI industry includes the belief that it could go very badly. It goes hand in hand. Like, they need that part of the myth in order to then say, and that's why we need to be in control of the technology, because that's the only way that it's going to go really, really well. And Altman has said publicly, you know, the worst case, lights out for everyone, but best case, we cure cancer, we solve climate change, and there's abundance. And Dario Amodei, same kind of rhetoric was like, worst case, catastrophic or existential harm for humanity, best case, mass human flourishing. So, this is like two sides of the same coin. Like, they have to use both of these narratives in order to continue justifying an extremely anti-democratic approach to AI development, where there should not be broad participation in developing this technology. They must be the ones controlling it at every step of the way.
Sam Altman did a tweet saying, "There are some books coming out about OpenAI and me. We only participated in two of them. One by Kesh Hegy..."
Keegy.
"...Keegy focused on me, and one by Ashley Vance on OpenAI." Um, he went on to say, "No book will get everything right, especially when some people are so intent on twisting things, but these two authors are trying to." You quote-tweeted that tweet from Sam Altman and you said, "The unnamed book, Empire of AI, is mine." Do you believe that tweet from Sam Altman was in reference to your book?
100%. Because there's only three books coming out about him.
And he had caught wind that your book was coming out?
And he knew my book was coming out because I had contacted OpenAI from the very beginning of my process and said, "I'm working on a book now. Will you participate in it?" And actually, initially, they said yes. Even though, so my history with OpenAI, I profiled the company for MIT Technology Review. I embedded within the office for three days in 2019. My profile comes out in 2020. The leadership are very unhappy. And in my book, I actually quote an email that I received that Sam Altman sent to the company about my profile saying, "Yeah, this is not great." And from then on, the company's stance to me was, "We are not going to participate in anything that you do. We are not going to respond to anything, any of the questions that you receive." And this was, you know, this was things that they explicitly articulated. It wasn't like me inferring. Um, so I, I had a, a colleague at MIT Technology Review that also covered AI. And at one point, OpenAI sent him this press release being like, "We would love for you to cover this story." And he was like, "I'm really busy. Will you send it to Karen?" And they were like, "Oh no, we have a history. You understand?"
Okay, so this next part is pretty interesting. The host asked Karen why so many people leave OpenAI and what kind of advanced tech these people see there which compels them to separate themselves from the company in general. Guys, this is a very important point because so far, we've seen so many great AI scientists leaving OpenAI without giving any solid reasoning. My theory is that when they find out the end goal of the company, they don't want to be a part of that mission. I believe these people know that OpenAI is going to achieve AGI in the near future, and the kind of chaos AGI is going to bring would be destructive for humans. So, I mean, probably these scientists are leaving OpenAI because they don't want to be a part of such an evil mission.
How does a CEO of a major company get fired by the board? Because board members, there's a quote in your book on page 357 where you say about Ilya, saying, "I don't think Sam is the guy who should have the finger on the button for AGI." Now, I, I asked myself this question. You know, I work with lots of people here. We have 150 people that work in this business. And those people know me best.
Yeah.
They see me on camera, they see me off camera. So, if they said, "We don't think Steven is the right person to host the direc..."
Yeah.
"...it would take a lot for them to say that."
Yeah.
"They must have seen some off-camera for them to go, we don't think he's the right person to be on camera."
Yeah.
Or for whatever reason. And in the case of AI, which is much more consequential than a podcast that is, you know, filmed in my old kitchen, um, it almost sends a chill down one's body to think that the co-founder of a business has gone to the board and said, "This isn't the guy to lead this consequential."
It wasn't just Ilya. Mira Murati then also said, "I don't think Altman is the right guy."
And then they both left later.
So then Altman comes back, and lo and behold, Ilya never comes back. So his concerns about the fact that Altman founding out would be bad for him manifested. He ended up not coming back, and Mira Murati then left shortly thereafter.
Quite a lot of these people leave, don't they? OpenAI.
They do. So if you consider one of the origin stories of OpenAI is this dinner that happened at the Rosewood Hotel, which is a very swanky hotel, um, right, right in the heart of Silicon Valley, that, uh, was one of Elon Musk's favorites whenever he was coming up from LA to the Bay Area. And there was this dinner that was there where Altman was intending to recruit the OG team that would start OpenAI. So he's kind of telling everyone, you might have a chance to meet Musk because Musk is going to come to this dinner. Dinner. And he cold emails Ilya and gets Ilya to then come because, and Ilya specifically wants to come because he wants to meet Musk. And he also emails all these other people, including Greg Brockman, Dario Amodei. These are all people that ended up working at Open.
And they all, almost all of them, not not every one of them, but almost all of them end up working at OpenAI.
And leaving.
Almost all of them end up leaving specifically after they clash with Altman. And Ilya, he left and launched a company called Safe Superintelligence.
Yeah.
Which is, I mean, that's an indirect if I've ever heard one. Do you know what I mean? Do you know what I mean? If someone like co-founded this podcast with me and then they left and started a podcast called Safe Podcasting, I, I'd take that as a slight. I'd have people knocking on their door and asking for their texts. One of the things that is happening here is,
It is not a coincidence that every single tech billionaire has their own AI company.
Mhm.
They want to create AI in their own image, and that's why they keep not getting along. And in fact, it's not just don't get along, they end up hating each other after working together.
Mhm.
And then splinter off into their own organizations. So after Musk leaves, he starts XAI. After Dario leaves, he starts Anthropic. After Ilya leaves, he starts Safe Superintelligence. After Mira leaves, she starts Thinking Machines Lab. They want to have control over their own vision of this technology. And the best way that they have derived from their experiences of trying to put their vision into the arena is by creating a competitor and then competing with OpenAI and with all the other companies out there.
Okay. Next, the host asks Karen whether the CEOs of AI companies actually understand or care that they're literally summoning the demon in the form of AGI. Guys, I find this really funny that the entire world knows that AGI is going to take our jobs. It is going to cause civil wars. It's going to cause chaos throughout the world. And despite knowing all this, we're still okay with these companies pursuing AGI. It's really weird that nobody's talking about this on the government level either. It's like everyone has already accepted that AI is going to one day replace them and there's no way to stop it. Do you think some of these AI CEOs realize that they are quite literally summoning the demon, as Elon said 10 years ago, but they don't really care because being the person that summoned the demon makes you consequential and powerful and historical, even if the outcome is potentially horrific. Even if there's like a 20% outcome of it being horrific. I remember I think it was Dario, he's the one that said there's somewhere between a 10% and 25% chance of things going catastrophically wrong on the scale of human civilization. 25% is a one in four chance. If you put bullets in a four-chamber revolver and said, "Steven, the upside is you could become a multi-gazillionaire and be remembered forever. The downside is that there would be a bullet in your head." There is no chance that I would take that bet with a 25% potential chance of things going catastrophically wrong.
So, I have a very long answer to this because do they know if they're summoning the demon? It really depends on what we define as summoning the demon. And in this particular case, to go back to what we were saying before, there's a mythology that the AI industry uses where summoning the demon is an integral part of convincing everyone that therefore they can be the only ones that are developing this technology.
Uh, I got it. So on one end, you got to say, if we don't, China will, and that's terrible.
Yeah.
But if we let anyone else do it other than me, then we're as well.
Exactly.
So that means that I have to do it, and you have to give me money and support.
Exactly. So when they're saying these things, we should understand it as not as like a genuine prediction based on what they're seeing, because first of all, we don't predict the future. We make it. We should understand this as an act of speech to persuade other people into believing that they should seed more power, more resources to these individuals. And so, do they know that they're summoning the demon? I mean, they are purposely trying to create this this feeling within the public that they are, because it is a crucial part of their power. But do they, if we were to define, just do they realize that the things that they are doing are having already really harmful impacts all around the world on vulnerable people, vulnerable communities, vulnerable countries? That's where I'm like, maybe yes, maybe no. And they don't really care because in the frame of mind, like I sometimes use the analogy that the AI world is like Dune. Dune, for anyone that doesn't know, Dune,
science fiction epic written by Frank Herbert, and it's set in this intergalactic era where there are all these houses and they're fighting each other for spice. So, it's a callback to colonialism and empire, and they all are trying to control the spice. But one of the features of this story is that there are these myths that are seeded on the different planets about a, a religious myth basically about the coming of the Messiah that are used as ways to control the people. And Paul Atreides, when he arrives at the planet, uh, with, with the intention of, um, trying to then fight against the empire and, um, avenge his father's death, he steps into a myth that has been seeded on this planet that says that one day there will be a Messiah that comes and saves the planet. So he steps into the role of the Messiah and leans into this idea in order to better control the people and rally them behind him as a leader to help with this quest. He knows that it's a myth in the beginning, but because he lives and breathes and embodies it, it kind of starts to blur in his mind whether this is really a myth or whether he's really the Messiah. And this is what I think happens in the AI world. On one hand, there are all these executives that actively engage in mythmaking because, you know, I have all these internal documents that I write about in the book where they are very keenly aware of how to bring the public along with them by showing them dazzling demonstrations of the technology, by using crafting a mission that will sound really good, uh, and, and, and make people give more leniency to their companies. So they know they're doing the mythmaking, and also I think many of them lose themselves in the myth because they have to live and breathe and embody it day in and day out. And so when, you know, Dario says he thinks that 10 to 25% of the future could be catastrophic or, or whatever the probability is, 10 to 25%, he is actively engaging in the mythmaking, but also he's losing himself in the myth. Like, I think if you were to ask him, "Do you genuinely believe that?" He would be like, "Yes, I genuinely believe that." Because there's been a blurring of when he's saying something just to say something versus when he actually believes what is he's required to believe in order to then continue doing the things that he's doing.
And this is the whole psychology of cognitive dissonance, right? Or you, the brain struggles to hold two conflicting worldviews at the same time. So it's, it's incentivized or it endeavors to dismiss one. So if you, you know, if you wanted to be a healthy person but also a smoker.
Um, and I pointed out smoking is bad for you. The first words out of your mouth are going to be, "Yes, but..."
"...smoking helps me with stress."
Yeah. But I only do it when I think, I don't know, I kind of see that at the moment because these companies have to raise extortionate, like huge amounts of money to fund their AI research, and they're building out all of these data centers.
So when they're out in the public, they're always fundraising. All of these major companies are fundraising all the time at the moment.
So you can't be fundraising and saying, "I'm going to destroy your children's future potentially. There's a 25% chance that your children aren't going to have a great life." Which might be the truth. I mean, that is actually what they say. Dario, this is what famously Dario Amodei does. He's like,
He does that, but the others, Sam's not doing that as much anymore.
Yes. And it's because, you know, it goes back to like, each of them kind of distinguish themselves a little bit as, as the brand that they need to project.
Do you think any of them are more, have a stronger moral compass than others? 'Cause I think Dario often gets the credit for having more of a, you know, more of a backbone and being more conscious of implications.
He does get a lot of credit for that.
He's from Claude and Anthropic for anyone that doesn't know. I don't think it truly matters that question, the answer to that question, because to me, even if you were to swap all the CEOs for someone that people would say is better at running these companies, it doesn't fix the problem that I identify in the book, which is that there is a system of power that has been constructed where these companies and the people running these companies get to make decisions that affect billions of people's lives around the world. And those billions of people do not get any say in how it goes.
All right, guys. That's it for today's video.