Transcription
Two possibilities. It is either possible for super intelligent AI to hack through all the layers of boxing and escape, and by doing so, show us a prototype, show us how it's done. Or it's even better. We are successful at boxing super intelligent AI. It cannot escape, and now we solve the AI control problem. That would be really good for us.
So we talked about what we're interested in. I wanted to see what has already been tried by people. Many things don't work. So realizing that you are in a simulation doesn't shut it down. We know that much. We published papers. We see it's still here. Praying doesn't seem to work. Uh, doing horrible things, torturing babies and puppies. No one comes around to rescue them. So whatever ethical standards we have, this is not going to cause any interference.
Some people think that, okay, it's a simulation, there is a processor. If you overburden the processor, we can create some impact. So people tried running the Bitcoin network. It's very computationally intensive. Didn't make any difference. It's possible that we just have a lot of compute power, but maybe, maybe it's not causing a problem. There are some others, uh, you can declare that I no longer consent to being in a simulation, and nothing changes. People also worried about large hardware colliders creating a black hole and destroying the universe. But that's slightly different, a different aspect of it.
So it seems like we are not smart enough to hack the simulation. We're still here. Nothing seems to work. Perhaps we can get some help. It's probably something with weird quantum physics, and we need someone super intelligent to help us accomplish this. So what if we created super intelligent hacker AI to assist us?
We seem to be on a way. A couple years ago, we started this process of creating more and more intelligent AI agents, with OpenAI releasing their ChatGPT program, which was for the first time realized to be a somewhat general form of intelligence. It wasn't a narrow AI designed for just one thing, playing checkers or driving a car. It was general. It could learn in new domains. It did really well on exams, tests. It outperformed most college students. And it was able to learn new tasks in new domains. In fact, it was part of what we know as a scaling hypothesis, where we don't need to have new inventions to make better AI. It's enough to add more data, more compute, bigger parameter models, and they keep getting better and better. So far, we haven't hit any limiting, uh, results in that space. And so every, every day, a new model gets released. And, uh, I've been traveling here for like two days, and by the time I got here, a new model is 10 times better than the previous one. So it's happening pretty quickly.
How soon will we get to human-level performance and then super intelligent performance? I don't know for sure, but people who run those large labs are saying it's about two years, and they said it about a year ago. If you look at prediction markets, the best tool we have for predicting the future, likewise, we are told it's happening in a couple of years. So human-level, which allows us to automate science and engineering, and soon after, we get to super intelligent performance.
People are worried about it. Then I started researching this topic over a decade ago. It was pure science fiction. Now it's pure science. Those models are released. People have access to them. Every top university has an AI safety lab. And more and more people are concerned that if they get that smart, there is some concern about existential risks associated with that technology. Thousands of scholars signed a letter comparing advanced intelligence to the dangers of nuclear weapons. There is a lot of effort from governments, from academia, from private organizations to figure out, well, if we're creating this super intelligent hacker assistant, how do we control and make sure that it does what we want, not something really bad?
Part of my research is trying to figure out all the different ways we can control AI. If we can control AI, I started by looking at everything people suggested we do in the space. But 300 references, we classified them. I don't have time to tell you about all of them, but I'll point out a few interesting things. The first time somebody said machines are really getting out of hand, we need to do something about it, is 1863. That gives you an idea about the scope of the problem. There are different social methods, computer-based methods, economic, political. None of them actually work. The problem is we're not even agreeing on what the problem is.
About the first 50 years of research in AI, people talked about ethics, human ethics, machine ethics, getting human ethics into machines. Once we started getting more general, more capable AIs, uh, the titles switched. We talked about friendly AI, control problem. I like AI safety and security engineering as a name for it. But really, it is the problem, and it is the most important problem we'll ever face. How do we control super intelligent machines? So we get the benefits, but not the problems which come with those machines.
And interestingly, in computer science, you usually start by figuring out, well, how hard is this problem? Is it even solvable? Do I have enough computational resources to address this problem? And for this problem, we don't really know the answer. People disagree. There could be different interpretations of what it means to be in control, direct control, delegated control. And this is what I try to look at in my research.
I realize to control advanced AI, we'll need a toolbox of capabilities. I should be able to explain how they work, understand the explanation, predict what it's going to do, be able to verify its decisions, model code, and a few others. So, let's see how well we are doing in obtaining those tools. So, each one of those is a published paper. You can look it up. Interesting results. But the general tendency is that there are upper limits in our ability to do things in that space.
And for example, we cannot fully comprehend how a large language model works. It has billions of neurons, trillions of weights connecting them. And a full explanation is the full model. So if I give you the full model, it tells you nothing. It's too large. You can't even survey it. If I give you a simplified explanation, you're getting something like lossy compression. It's helpful, but it doesn't tell you the full answer. If you were denied a loan, it will tell you the top two reasons you were denied. But the real decision was billions of weights integrated together.
Another limiting capability is predicting specific actions of more intelligent agents. If I'm playing chess with an advanced AI, the only thing I know and can predict is that it's going to beat me. I have no idea how. If I could predict specific moves, I'd be playing at the same level. It would no longer be super intelligent. So, we don't know what it's actually going to do.
There are also limits to verifying code and mathematical proofs. The simple limit is from infinite regressive verifiers. Somebody has to verify the program which verifies the code, or the mathematician who did it. You have levels of peer review, and a lot of times the mathematical community doesn't agree on whether an actual proof is legit or not. It's too long. It's too complex. There are not enough people qualified enough to address it. So there are limits to our ability to check if the model of AI matches implementation, and even more so, we have no idea how to do it for systems which continue to learn, self-improve, interact with its environment, be exposed to malevolent users. At best, we can verify fixed, deterministic, mission-critical software, short functions for space flight or something like that.
There are also limits to our ability to simply monitor and test what the system is doing, what it's capable of, both during the training phase and deployment phase. Training could take months, six months, a year. And while the system is training, if you're not testing it at the same time, you have no idea how capable it is. If the last model was just below human level, and you train for six months, by the time you start testing, it could be super intelligent already. Same with testing. We're not really sure how to test software which is general and very capable. We know how to test edge cases for narrow systems, like I can verify that a tic-tac-toe program is working as expected, but nothing more complex.
So since technical results are not very encouraging, people switch to governance. If we can pass laws saying it's illegal to destroy humanity, and you'll pay high fines if you do. Yeah, it doesn't work. So another problem there. If you like such results, we published in a very good journal, published a paper with about 50 impossibility results. Every flavor you can desire from physics, mathematics, all the subfields which contribute to AI and computer science, there are relevant results telling us you cannot do this at that level of performance. And we need very high levels of guarantees to make sure that a super intelligent hacker is safe.
So overall, this set of results and limited tools got us to publish a paper on limits to control in general, where we classify different levels of control and show that each one of them has impossibility results. If it's direct control, usually it's known as a genie problem. You have three wishes, and usually your second wish is to undo your first wish because you screwed up really bad. You want to escape at this point. The other option is delegated control. Another extreme. There are some in the middle. But another extreme is delegated control, where you have a super intelligent system. It knows all about you. It knows you better than you know yourself. It predicts what you like. And you basically go, you tell me what to do. You're no longer in control. You might be happy with it, but you definitely are not controlling the system.
So those are the different states. And we published this. We got a good paper out of it. We had an article in Time magazine. Nobody seems to care about this too much. The results are actually not novel. If you look at the history of computer science, history of AI, starting with Alan Turing, the founding father, he basically said, well, obviously you can't control super intelligent godlike machines. That's a given. Like, why are you even trying? And more modern scholars seem to agree. That's that's where we're at.
So, early on, we had an idea, well, we need to study those machines in a safe environment. Like when you have a computer virus, you put it in a computer which is separated from the internet. It's air-gapped, and you study inputs, outputs, what it's trying to do, how it's operating. You have a limited language of communicating so it cannot perform social engineering attacks against you. This is known as a boxing problem. AI boxing problem. You put AI in a box. Well, if you realize the two problems I talked about are the same problem. AI boxing problem is just a subset of simulation escape problem. To AI, whatever you are, two layers deep in a virtual world or five layers deep doesn't matter. Two possibilities. It is either possible for super intelligent AI to hack through all the layers of boxing and escape, and by doing so, show us a prototype, show us how it's done. Or it's even better. We are successful at boxing super intelligent AI. It cannot escape, and now we solve the AI control problem. That would be really good for us.
So part of what I studied was different accidents which we see and happen with AI. For the first 10, 20, 30 years, you had a few here and there. They became more impactful, more common as more people deployed those systems. And some of them read kind of funny, if you can read the small font. But I stopped collecting them after a while because there are just too many of those accidents. Every day you interact with those systems, you get some horrible mistake. And so I was thinking, what would be a mistake in this scenario where we ask super intelligence, well, help us hack out of our simulation? And one possibility is that we are not in a simulation.
To continue watching this video, click the link in the top left or in the description below. With a free trial, you can enjoy the full talk and thousands more. Thank you for being part of the conversation.