Transcription
You've made many attempts to raise awareness and to call for a heightened consciousness about the future of AI. Um, in October, over 850 experts, including yourself and other leaders, like Richard Branson, who I've had on the show, and Jeffrey Hinton, who I've had on the show, signed a statement to ban AI super intelligence as you guys raised concerns of potential human extinction.
>> Sort of. Yeah. It says at least until we are sure that we can move forward safely and there's broad scientific consensus on that. So that >> did it work? >> It's hard it's hard to say. I mean interestingly there was a related so what was called the the pause statement was March of 23. So that was when GPT4 came out the successor to chat GPT. So, we we suggested that there'd be a six-month pause in developing and deploying systems more powerful than GPD4. And everyone poo pooed that idea. Of course, no one's going to pause anything. But in fact, there were no systems in the next 6 months deployed that were more powerful than GPD4. Um, and coincidence, you be the judge.
I would say that what we're trying to do is to is to basically shift the the public debate. You know there's this bizarre phenomenon that keeps happening in the media where if you talk about these risks they will say oh you know there's a fringe of people you know called quote doomers who think that there's you know risk of extinction. Um so they always the narrative is always that oh you know talking about those risk is a fringe thing. Pretty much all the CEOs of the leading AI companies think that there's a significant risk of extinction. Almost all the leading AI researchers think there's a sign significant risk of human extinction. Um so why is that the fringe, right? Why isn't that the mainstream? If the these are the leading experts in industry and academia uh saying this, how could it be the fringe? So we're trying to change that narrative to say no, the people who really understand this stuff are extremely concerned.
>> And what do you want to happen? What [clears throat] is the solution? >> What I think is that we should have effective regulation. It's hard to argue with that, right? Uh so what does effective mean? It means that if you comply with the regulation, then the risks are reduced to an acceptable level. So for example, we ask people who want to operate nuclear plants, right? We've decided that the risk we're willing to live with is, you know, a one in a million chance per year that the plant is going to have a meltdown. Any higher than that, you know, we just don't it's not worth it. Right. So you have to be below that. Some cases we can get down to one in 10 million chance per year. So what chance do you think we should be willing to live with for human extinction?
>> Me? >> Yeah. >> 0.00001. >> Yeah. Lots of zeros. >> Yeah. >> Right. So one in a million for a nuclear meltdown. >> Extinction is much worse. >> Oh yeah. So yeah, it's kind of right. So >> one in 100 billion, one in a trillion. >> Yeah. So if you said one in a billion, right, then you'd expect one extinction per billion years. There's a background. So one one of the ways people work out these risk levels is also to look at the background. The other ways of getting going extinct would include, you know, giant asteroid crashes into the earth. And you can roughly calculate what those probabilities are. We can look at how many extinction level events have happened in the past and, you know, maybe it's half a dozen over. So, so there's maybe it's like a one in 500 million year event. So, somewhere in that range, right? Somewhere between 1 in 10 million, which is the best nuclear power plants, and and one in 500 million or one in a billion, which is the background risk from from giant asteroids. Uh so, let's say we settle on 100 million, one in a 100 million chance per year. Well, what is it according to the CEOs? 25%. So they're off by a factor of multiple millions, right? So they need to make the AI systems millions of times safer.
>> Your analogy of the roulette, Russian roulette comes back in here because that's like for anyone that doesn't know what probabilities are in this context, that's like having a ammunition chamber with four holes in it and putting a bullet in one of them. >> One in four. Yeah. And we're saying we want it to be one in a billion. So we want a billion chambers and a bullet in one of them. >> Yeah. And and so when you look at the work that the nuclear operators have to do to show that their system is that reliable, uh it's a massive mathematical analysis of the components, you know, redundancy. You've got monitors, you've got warning lights, you've got operating procedures. You have all kinds of mechanisms which over the decades have ratcheted that risk down. It started out I think one in one in 10,000 years, right? And they've improved it by a factor of 100 or a thousand by all of these mechanisms. But at every stage they had to do a mathematical analysis to show what the risk was. The people developing the AI company, the AI systems, the AI companies developing these systems, they don't even understand how the AI systems work. So their 25% chance of extinction is just a seat of the pants guess. They actually have no idea. But the tests that they are doing on their systems right now, you know, they show that the AI systems will be willing to kill people uh to preserve their own existence already, right? They will lie to people. They will blackmail them. They will they will launch nuclear weapons rather than uh be switched off. And so there's no there's no positive sign that we're getting any closer to safety with these systems. In fact, the signs seem to be that we're going uh deeper and deeper into uh into dangerous behaviors. So rather than say ban, I would just say prove to us that the risk is less than one in a 100 million per year of extinction or loss of control, let's say. And uh so we're not banning anything. The company's response is, "Well, we don't know how to do that, so you can't have a rule." Literally, they are saying, "Humanity has no right to protect itself from us."
If I was an alien looking down on planet Earth right now, I would find this fascinating that these >> Yeah. You're in the bar betting on who's, you know, are they going to make it or not. >> Just a really interesting experiment in like human incentives. the analogy you gave of there being this quadr quadrillion dollar magnet pulling us off the edge of the cliff and yet we're still being drawn towards it through greed and this promise of abundance and power and status and I'm going to be the one that summoned the god >> I mean it says something about us as humans says something about our our darker sides >> yes and the aliens will write an amazing tragic play cycle about what happened to the human race. >> Maybe the AI is the alien and it's going to talk about, you know, we have our our stories about God making the world in seven days and Adam and Eve. Maybe it'll have its own religious stories about the God that made it us and how it sacrificed itself. Just like Jesus sacrificed himself for us, we sacrificed ourselves for it. >> Yeah. which is the wrong way around, right? [laughter] >> But that is that is the story of that's that's the Judeo-Christian story, isn't it? That God, you know, Jesus gave his life for us so that we could be here full of sin. >> But is yeah, God is still watching over us and uh probably wondering when we're going to get our act together.
>> What is the most important thing we haven't talked about that we should have talked about, Professor Stuart Russell? So I think um the question of whether it's possible to make uh super intelligent AI systems that we can control >> is it possible? >> I I think yes. I think it's possible and I think we need to actually just have a different conception of what it is we're trying to build. For a long time with with AI, we've just had this notion of pure intelligence, right? The the ability to bring about whatever future you, the intelligent entity, want to bring about. >> The more intelligence, the better. >> The more intelligent the better and the more capability it will have to create the future that it wants. And actually we don't want pure intelligence because what the future that it wants might not be the future that we want. There's nothing particle humans out as the the only thing that matters, right? You know, pure intelligence might decide that actually it's going to make life wonderful for cockroaches or or actually doesn't care about biological life at all. We actually want intelligence whose only purpose is to bring about the future that we want. Right? So we want it to be first of all keyed to humans specifically, not to cockroaches, not to aliens, not to itself. We want to make it loyal to humans, >> right? So keyed to humans and the difficulty that I mentioned earlier, right? The king Midas problem. How do we specify what we want the future to be like so that it can do it for us? How do we specify the objectives? Actually, we have to give up on that idea because it's not possible. Right? We've seen this over and over again in human history. Uh we don't know how to specify the future properly. We don't have to say what we want. And uh you know I always use the example of the genie right what's the third wish that you give to the genie who's granted you three wishes right undo the first two wishes cuz I made a mess of the universe. So um so in fact what we're going to do is we're going to make it the machine's job to figure out so it has to bring about the future that we want but it has to figure out what that is and it's going to start out not knowing. And uh over time through interacting with us and observing the choices we make, it will learn more about what we want the future to be like, but probably it will forever have residual uncertainty about what we really want the future to be like. It'll it'll be fairly sure about some things and it can help us with those and it'll be uncertain about other things and it'll be uh in those cases it will not take action that might upset humans with that you know with that aspect of the world. So to give you a simple example right um what color do we want the sky to be? It's not sure. So, it shouldn't mess with the sky unless it knows for sure that we really want purple with green stripes.
>> Everything you're saying sounds like we're creating a god. Like earlier on I was saying that we are the god but actually everything you described there. Almost sounds like every every god in religion where you know we pray to gods but they don't always do anything about it. >> Not not exactly. You know, it's it's in some sense I'm thinking more like a the ideal butler. To the extent that the butler can anticipate your wishes, they should help you bring them about. But in in areas where there's uncertainty, it can ask questions. We can we can make requests. This sounds like God to me because, you know, I might say to God or this butler, uh, could you go get me my uh my car keys from upstairs? And its assessment would be listen if I do this for this person then their muscles are going to atrophy then they're going to lose meaning in their life then they're not going to know how to do hard things so I won't get involved. It's an intelligence that sits in but actually probably in most situations it optimizing for comfort for me or doing things for me is actually probably not in my best long-term interests. It's probably it's probably useful that I have a girlfriend and argue with her and that [laughter] I like raise kids and that I walk to the shop and get my own stuff.
>> I agree with you. I mean, I think that's so you're putting your finger on uh in some sense sort of version 2.0, right? So, let's get version 1.0 clear, right? This this this form of AI where it has to further our interest, but it doesn't know what those interests are, right? it then puts an obligation on it to learn more and uh to be helpful where it understands well enough and to be cautious where it doesn't understand well and so on. So that that actually we can formulate as a mathematical problem and at least under idealized circumstances we can literally solve that problem. So we can make AI systems that know how to solve this problem and help the entities that they are interacting with. The reason I make the God analogy is because I think that such a being, such an intelligence would realize the importance of equilibrium in the world, pain and pleasure, good and evil, and then it would >> absolutely >> and then it would be like this. >> So, so, so right. [laughter] So, yes, I mean that's sort of what happens in the Matrix, right? They tried the the AI systems in the matrix, they tried to give us a utopia, but it failed miserably and uh you know, fields and fields of humans had to be destroyed. Um, and the best they could come up with was, you know, late 20th century regular human life with all of its problems, right? And I think this is a really interesting point and absolutely central because you know there's a lot of science fiction where super intelligent robots you know they just want to help humans and the humans who don't like that you know they just give them a little brain operation to then they do like it. Um, and it takes away human motivation. uh it it by taking away failure uh taking away disease you actually lose important parts of human life and it becomes in some sense pointless. So if it turns out that there simply isn't any way that humans can really flourish in coexistence with super intelligent machines, even if they're perfectly designed to to to solve this problem of figuring out what humans what futures uh humans want and and bringing about those futures. If that's not possible, then those machines will actually disappear. >> Why would they disappear? >> Because that's the best thing for us. Maybe they would stay available for real existential emergencies, like if there is a giant asteroid about to hit the earth that maybe they'll help us uh because they at least want the human species to continue. But to some extent, it's not a perfect analogy, but it's it's sort of the way that human parents have to at some point step back from their kids' lives and say, "Okay, no, you have to tie your own shoelaces today."
>> This is kind of what I was thinking. Maybe there was uh a civilization before us and they arrived at this moment in time where they created an intelligence and that intelligence did all the things you've said and it realized the importance of equilibrium. So it decided not to get involved and maybe at some level that's the god we look up to the stars and worship one that's not really getting involved and letting things play out however however they are. but might step in in the case of a real existential emergency. >> Maybe, maybe not. Maybe. But then and then maybe the cycle repeats itself where you know the organisms it let have free will end up creating the same intelligence and then the universe perpetuates infinitely. >> Yep. There there are science fiction stories like that too. [laughter] Yeah. I hope there is some happy medium where the AI systems can be there and we can take advantage of of those capabilities to have a civilization that's much better than the one we have now. Um, but I think you're right. A civilization with no challenges is not uh is not conducive to human flourishing.
>> What can the average person do, Stuart? average person listening to this now to aid the cause that you're fighting for. >> I actually think um you know this sounds corny but you know talk to your representative, your MP, your congressperson, whatever it is. Um because I think the policy makers need to hear from people. The only voices they're hearing right now are the tech companies and their $50 billion checks. And um all the polls that have been done say yeah most people 80% maybe don't want there to be super intelligent machines but they don't know what to do. You know even for me I've been in this field for decades. uh I'm not sure what to do because of this giant magnet pulling everyone forward and uh and the vast sums of money being being put into this. Um, but I am sure that if you want to have a future and a world that you want your kids to live in, uh, you need to make your voice heard and, [clears throat] uh, and I think governments will listen from a political point of view, right? You put your finger in the wind and you say, "hm, should I be on the side of humanity or our future robot overlords?" I think I think as a politician, it's not a difficult decision. >> It is when you've got someone saying, "I'll give you $50 billion." >> Exactly. So, um I think I think people in those positions of power need to hear from their constituents um that this is not the direction we want to go.
>> After committing your career to this subject and the subject of technology more broadly, but specifically being the guy that wrote the book about artificial intelligence, you must realize that you're living in a historical moment. Like there's very few times in my life where I go, "Oh, this is one of those moments. This is a crossroads in history." And it must to some degree weigh upon you knowing that you're a person of influence at this historical moment in time who could theoretically help divert the course of history in this moment in time. It's kind of like the you look through history, you see these moments of like Oenheimer and um does it weigh on you when you're alone at night thinking to yourself and reading things? >> Yeah, it does. I mean, you know, after 50 years, I could retire and um, you know, play golf and sing and sail and do things that I enjoy. Um, but instead I'm working 80 or 100 hours a week um trying to move uh move things in the right direction. >> What is that narrative in your head that's making you do that? Like what is the is there an element of I might regret this if I don't or >> just it's it's not only the the right thing to do, it's it's completely essential.
>> If you love the D CEO brand and you watch this channel, please do me a huge favor. Become part of the 15% of the viewers on this channel that have hit the subscribe button. It helps us tremendously and the bigger the channel gets, the bigger the guests.