Transcription
So, ChachiT kind of set off the starting pistol for this this whole race and subsequently it appears that every other major technology company now is investing godly amounts ungodly amounts of money in competing in this AI race and they're pursuing this thing called AGI which we hear this word used a lot.
Yes. What is what is AGI and how is that different from what I use at the moment on chatb or Gemini?
Yeah. So that's the thing that people really need to get is that these companies are not racing to provide a chatbot to users. That's not what their goal is. If you look at the mission statement on OpenAI's website or all the websites, their mission is to be able to replace all forms of human economic labor in the economy. Meaning an AI that can do all the cognitive labor meaning labor of the mind. So that that can be marketing, that can be text, that can be illustration, that can be video production, that can be code production. Everything that a person can do with their brain, these companies are racing to build that. That is artificial general intelligence. General meaning all kinds of cognitive tasks. Demis Hassabis the co-founder of um Google DeepMind used to say first solve intelligence and then use that to solve everything else. Like it's important to say why why is AI distinct from all other kinds of technologies? It's because if I make an advance in one field like rocketry, if I just let's say I uncover some secret in rocketry, that doesn't advance like biio medicine knowledge or it doesn't advance energy production or doesn't advance coding. But if I can advance generalized intelligence, think of all science and technology development over the course of all human history. So science and technology is all done by humans thinking and working out problems. Working out problems in any domain. So if I automate intelligence, I'm suddenly going to get an explosion of all scientific and technological development everywhere. Does that make sense?
Of course. Yeah. It's foundational to everything.
Exactly. Which is why there's a belief that if I get there first and can automate generalized intelligence, I can own the world economy because suddenly everything that a human can do that they would be paid to do in a job, the AI can do that better. And so if I'm a company, do I want to pay the human who has health care, might whistleblow, complains, you know, has to sleep, has sick days, has family issues, or do I want to pay the AI that will work 24/7 at superhuman speed, doesn't complain, doesn't whistleblow, doesn't have to be paid for healthcare. There's the incentive for everyone to move to paying for AIs rather than paying humans. And so AGI, artificial general intelligence, is more transformative than any other kind of of technology that we've ever had and it's distinct.
With the sheer amount of money being invested into it and the money being invested into the infrastructure, the physical data centers, the chips, the compute, do you think we're going to get there? Do you think we're going to get to AGI?
I do think that we're going to get there. It's not clear uh how long it will take. And I'm not saying that because I believe necessarily the current paradigm that we're building on will take us there, but you know, I'm based in San Francisco. I talked to people at the AI labs. Half these people are friends of mine. You know, people at the very top level. And you know, most people in the industry believe that they'll get there between the next two and 10 years at the latest. And I think some people might say, "Oh, well, it may not happen for a while. Phew. I can sit back and we don't have to worry about And it's like we're heading for so much transformative change faster than our society is currently prepared to deal with it. The reason I was excited to talk to you today is because I think that people are currently confused about AI. You know, people say it's going to solve everything, cure cancer, uh solve climate change, and there's people say it's going to kill everything. It's going to be doom. Everyone's going to go extinct. If anyone builds it, everyone dies. And those those conversations don't converge. And so everyone's just kind of confused where how can it be, you know, infinite promise and how can it be infinite peril? And what I wanted to do today is to really clarify for people what the incentives point us towards which is a future that I think people when they see it clearly would not want.
So what are the incentives pointing us towards in terms of the future?
So first is if you believe that this is like it's metaphorically it's like the ring from Lord of the Rings. It's the ring that that creates infinite power because if I have AGI, I can apply that to military advantage. I can have the best military planner that can beat all battle plans for anyone. And we already have AIs that can obviously beat Gary Kasparov at chess, beat Go, the Go Asian um board game, or now beat Starcraft. So you have AI that are beating humans at strategy games. Well, think about Starcraft compared to an actual military campaign, you know, in Taiwan or something like that. If I have an AI that can out compete in strategy games, that lets me out compete everything. Or take business strategy. If I have an AI that can do business strategy and figure out supply chains and figure out how to optimize them and figure out how to undermine my competitors and I have a, you know, a step function level increase in that compared to everybody else, then that gives me infinite power to undermine and out compete all businesses. If I have a super programmer, then I can out compete programming. 70 to 90% of the code written at today's AI labs is written by AI.
Think about the stock market as well.
Think about the stock market. If I have an AI that can trade in the stock market better than all the other AIs, because they're currently there's mostly AIs that are actually trading on the stock market, but if I have a jump in that, then I can consolidate all the wealth. If I have an AI that can do cyber hacking, that's way better at cyber hacking in a step function above what everyone else can do, then I have an asymmetric advantage over everybody else. So AI is like a power pump. It pumps economic advantage. It pumps scientific advantage and it pumps military advantage. Which is why the countries and the companies are caught in what they believe is a race to get there first. And anything that is a negative consequence of that, job loss, rising energy prices, more emissions, stealing intellectual property, you know, security risks, all of that stuff feels small relative to if I don't get there first, then some other person who has less good values as me, they'll get AGI and then I will be forever a slave to their future. And I know this might sound crazy to a lot of people, but this is how people in at the very top of the AGI AI world believe is currently happening. And that's what
conversations.
Yeah.
You you've had I mean know Jeff Hinton and and Roman Ylonsky on and other people Mogadat and they're saying the same thing. And I think people need to take seriously that whether you believe it or not, the people who are currently deploying the trillions of dollars, this is what they believe. And they believe that it's win or take all. And it's not just first solve intelligence and use that to solve everything else. It's first dominate intelligence and use that to dominate everything else.
Have you had concerning private conversations about this subject matter with people that are in the industry?
Absolutely. I think that's what most people don't understand is that um there's a different conversation happening publicly than the one that's happening privately. I think you're aware of this as well.
I am aware of this.
What do they say to you?
So, it's not always the people telling me directly. It's usually one step removed. So, it's usually someone that I trust and I've known for many, many years who at a kitchen table says, "I met this particular CEO. We were in this room talking about the future of AI. this particular CEO they're referencing is leading one of the biggest AI companies in the world and then they'll explain to me what they think of the future's going to look like and then when I go and watch them on YouTube or podcasts what they're saying is they they have this real public bias towards the abundance part that you know we're going to cure cancer
cure cancer universal high income for everyone
yeah all this all this stuff
doesn't work anymore
but then privately what I hear is is exactly what you said which is really terrifying to me there was actually since since the last time we had a conversation about AR and podcast, I was speaking to a friend of mine, very successful billionaire, knows a lot of these people, and he is concerned because his argument is that if there's even like a 5% chance of the adverse outcomes that we hear about, we should not be doing this. And he was saying to me that some of his friends who are running some of these companies believe the chance is much higher than that, but they feel like they're caught in a race where if they don't control this technology and they don't get there first and get to what they refer to as um takeoff, like fast takeoff.
Yeah. Uh recursive self-improvement or fast takeoff, which basically means what the companies are really in a race for you're pointing to is they're in a race to automate AI research. Um because so right now you have open AI, it's got a few thousand employees. Human beings are coding and doing the AI research. They're reading the latest research papers. They're writing the next, you know, they're hypothesizing what's the improvement we're going to make to AI. What's a new way to do this code? What's a new technique? And then they use their human mind and they go invent something. They they run the experiment and they see if that improves the performance. And that's how you go from, you know, GPT4 to GPT5 or something. Imagine a world where Sam Alman can instead of having human AI researchers can have AI AI researchers. So now I just snap my fingers and I go from one AI that reads all the papers, writes all the code, creates the new experiments to I can copy paste a 100 million AI researchers that are now doing that in an automated way. And it the belief is not just that, you know, the companies look like they're competing to release better chat bots for people, but the what they're really competing for is to get to this milestone of being to automate an intelligence explosion or automate recursive self-improvement, which is basically automating AI research. And that, by the way, is why all the companies are racing specifically to get good at programming because the faster you can automate a human programmer, the more you can automate AI research. And just a couple weeks ago, Cloud 4.5 was released and it can do 30 hours of uninterrupted complex programming tasks at the at the high end. That's crazy. So right now one of the limits on the progress of AI is that human humans are doing the work but actually all of these companies are pushing to the moment when AI will be doing the work which means they can have an infinite arguably smarter zerocost workforce that's right scaling the AI. So when they talk about fast takeoff they mean the moment where they where the AI takes control of the research and it and progress rapidly increases
and it self-learns and recursively improves and invents. Um, so one thing to get is that AI accelerates AI, right? Like if I invent nuclear weapons, nuclear weapons don't invent better nuclear weapons.
Yeah.
But if I invent AI, AI is intelligence. Intelligence automates better programming, better chip design. So I can use AI to say, here's a design for the NVIDIA chips. Go make it 50% more efficient. And it can find out how to do that. I can say AI, here's a supply chain that I need for all the things for my AI company. And it can optimize that supply chain and make that supply chain more efficient.
Mhm. AI, here's the code for making AI. Make that more efficient. Um, AI, here's training data. I need to make more training data. Go run a million simulations of how to do this and it'll train itself to get better.
AI accelerates AI.
What do you think these people are motivated by the CEOs of these companies?
That's a good question.
Genuinely, what do you think their genuine motivations are when you think about all these names?
I think it's a subtle thing. I think there's um it's almost mythological because there's almost a way in which they're building a new intelligent entity that has never before existed on planet Earth. It's like building a god. I mean, the incentive is build a god, own the world economy, and make trillions of dollars, right? If you could actually build something that can automate all intelligent tasks, all goal achieving that will let you out compete everything. So that is a kind of godlike power that I think relative imagine energy prices go up or hundreds of millions of people lose their jobs. That those things suck. But relative to if I don't build it first and build this god, I'm going to lose to some maybe worse person who I think in my opinion, not my opinion, Tristan, but their opinion thinks is a worse person. It's it's a kind of competitive logic that self-reinforces itself, but it forces everyone to be incentivized to take the most shortcuts, to care the least about safety or security, to not care about how many jobs get disrupted, to not care about the well-being of regular people, but to basically just race to this infinite prize. So, there's a quote that um a friend of mine interviewed a lot of the top people at the AI companies, like the very top, and he just came back from that and and basically reported back to me and some friends, and he said the following. In the end, a lot of the tech people I talk to when I'm when I really grill them on it about like why you're doing this, they retreat into number one, determinism, number two, the inevitable replacement of biological life with digital life, and number three, that being a good thing. Anyways, at its core, it's an emotional desire to meet and speak to the most intelligent entity that they've ever met. And they have some ego religious intuition that they'll somehow be a part of it. It's thrilling to start an exciting fire. They feel they'll die either way, so they prefer to light it and see what happens.
That is the perfect description of the private conversations.
Doesn't that match what what you have description,
doesn't it? And that's the thing. So, people may hear that and they're like, "Well, that sounds ridiculous." But if you actually
I just got goosebumps cuz it's the perfect description. Especially the part they'll think they'll die either way.
Exactly. Well, and um worse than that, some of them think that in the case where they if they were to get it right and if they succeeded, they could actually live forever because if AI perfectly speaks the language of biology, it will be able to reverse aging aging, cure every disease. And and so there's this kind of I could become a god. And I'll I'll tell you um you know, you and I both have know people who've had private conversations. Well, one of them that I have heard from one of the co-founders of one of the most, you know, powerful of these companies when when faced with the idea that what if there's an 80% or 20% chance that everybody dies and gets wiped out by this, but an 80% chance that we get utopia. He said, well, I would clearly accelerate and go for the utopia. Given a 20% chance,
it's crazy. People should feel you do not get to make that choice on behalf of me and my family. We didn't consent to have six people make that decision on behalf of eight billion people. We have to stop pretending that this is okay or normal. It's not normal. And the only way that this is happening and they're getting away with it is because most people just don't really know what's going on.
Yeah. But I'm curious what what do you think when they
I mean everything you just said. It's that last part about the 8020% thing is almost verbatim what I heard from a very good very successful friend of mine who is responsible for building some of the biggest companies in the world when he was referencing a conversation he had with the founder of maybe the biggest air company in the world. And it was truly shocking to me because because it was said in such a blasé way.
Yes. It wasn't Yeah. That that's what I had heard in this particular situation. wasn't like a matter of fact.
It was a matter of fact, it's just easy. Yeah, of course I would do the I would take the I roll the dice.
And even Elon Musk said he actually said the same number in an interview with Joe Rogan. Um and if you listen closely when he said, "I decided I'd rather be there when it all happens. If it all goes off the rails, I decided in that worst case scenario, I decided that I' I'd prefer to be there when it happens." which is just is justifying racing to our collective suicide. Now, I also want people to know like you don't have to buy into the sci-fi level risks to be very concerned about AI. So, hopefully later we'll talk about um the many other risks that are already hitting us right now that you don't have to believe any of this stuff.
Yeah. The the Elon thing I think is particularly interesting because for the last 10 years he was this slightly hard to believe voice on the subject of AI. He was talking about it being a huge risk and an extinction level.
He was the first AI people. Yeah. He was saying this is more dangerous than nukes. He was saying I try to get people to stop doing it. This is summoning the demon. Those are his words, not mine.
Yeah.
Um we shouldn't do this. Supposedly, he used his first and only meeting with President Obama, I think in 2016, to advocate for global regulation and global controls on on AI um because he was very worried about it. And then really what happened is um Chachi BT came out and as you said that was the starting gun and now everybody was in an allout race to get there first. He tweeted words to the effect I'll put it on the screen. He tweeted that he had remained in I think he used a word similar to disbelief for some time like suspended disbelief but then he said in the same tweet that the race is now on.
The race is on and I have to race
and I have to go. I have no choice but to go. And he tried, he's basically saying, "I tried to fight it for a long time. I tried to deny it. I tried to hope that we wouldn't get here, but we're here now, so I have to go."
Yeah.
And at least he's being honest. He does seem to have a pretty honest track record on this because because he was the guy 10 years ago warning everybody. And I remember him talking about it and thinking, "Oh god, this is like 100 years away. Why are we talking about that?"
I felt the same, by the way. Some people might think that I'm some kind of AI enthusiast and I'm trying to ratch I I didn't believe that AI was a thing to be worried about at all until suddenly the last 2 three years where you can actually see where we're headed. But um oh man there's just there's so much to say about all this and so if you think about it from their perspective it's like best case scenario I build it first and it's aligned and controllable meaning that it will take the actions that I want. It won't destroy humanity and it's controllable which means I get to be god and emperor of the world. Second scenario, it's not controllable but it's aligned. So I built a god and I lost control of it but it's now basically it's running humanity. It's running the show. It's choosing what happens. It's out competing everyone on everything. That's not that bad an outcome. Third scenario, it's not aligned. It's not controllable. And it does wipe everybody out. And that should be demotivating to that person to an Elon or someone. But in that scenario, they were the one that birthed the digital god that replaced all of humanity. Like this is really important to get because in nuclear weapons, the risk of nuclear war is an omni lose-lose outcome. Everyone wants to avoid that. And I know that you know that I know that we both want to avoid that. So that that motivates us to coordinate and to have a nuclear non-prololiferation treaty. But with AI, the worst case scenario of everybody gets wiped out is a little bit different for the people making that decision. Because if I'm the CEO of DeepSeek and I make that AI that does wipe out humanity, that's the worst case scenario and it wasn't avoidable because it was all inevitable. Then even though we all got wiped out, I was the one who built the digital god that replaced humanity. And there's kind of ego in that. And uh the god that I built speaks Chinese instead of English.
That's the religious ego point.
That's the ego.
Such a great point because that's exactly what it is. It's like this religious ego where I will be transcendent in some way.
And you notice that it it all starts by the belief that this is inevitable.
Yeah.
Which is like is this inevitable? It's important to note because if you believe it's if everybody who's building it believes it's inevitable and the investors funding it believe it's inevitable, it cocreates the inevitability.
Yeah.
Right.
Yeah.
And the only way out is to step outside the logic of inevitability because if if we are all heading to our collective suicide, which I don't know about you, I don't think that I don't want that. You don't want that. Everybody who loves life looks at their children in the morning and says,"I want I want the things that I love and that are sacred in the world to continue." That's what night that's what everybody in the world wants. And the only thing that is having us not anchor on that is the belief that this is inevitable and the worst case scenario is somehow in this ego religious way not so bad if I was the one who accidentally wiped out humanity because I'm not a bad person because it was inevitable anyway. M
and I think the goal of of for me this conversation is to get people to see that that's a bad outcome that no one wants and we have to put our hand on the steering wheel and turn towards a different future because we do not have to have a race to uncontrollable inscrutable powerful AIs that are by the way already doing all the rogue sci-fi stuff that we thought only existed in movies like blackmailing people uh being self-aware when they're being tested scheming and lying and deceiving to copy their own code to keep themselves preserved. Like the stuff that we thought only existed in sci-fi movies is now actually happening. And that should be enough evidence to say we don't want to do this path that we're currently on. It's not that some version of AI progressing into the world is directionally inevitable, but we get to choose which of those futures that we want to have. If you love the D CEO brand and you watch this channel, please do me a huge favor. become part of the 15% of the viewers on this channel that have hit the subscribe button.