Transcription
Thank you very much for the introduction. It's wonderful to be here. My first time in Pohang, uh, in this beautiful university campus. So I'm going to talk about artificial intelligence.
Uh, so to do that, we need to agree on what it is, and obviously AI is about making intelligent machines. And what that has meant since the beginning of the field, uh, is that a machine is intelligent to the extent that its actions can be expected to achieve its objectives. This is the same notion that we apply to humans. Uh, it borrows from philosophy and economics thousands of years of thought about, uh, what it means to, uh, to think correctly, to do the right thing.
So here are some simple examples. So AlphaGo is a program that played Go, and its objective is to win the game, and it does a very good job. Uh, when you use navigation software on your phone or in your car, uh, it has the objective of finding the shortest route to the destination. Uh, something that people are starting to do now is to create fully automated corporations, and their job is to maximize expected shareholder return.
So this concept, uh, is very general and very powerful, but AI actually has, uh, an even more ambitious goal. Since the beginning, our goal in the field has been to create general-purpose AI, or sometimes called AGI, artificial general intelligence. AI systems that are capable of quickly learning high-quality behavior—meaning as good as or better than humans—uh, in any task. So they exceed human capabilities along every relevant dimension, and this is what we are trying to do.
So let me quickly go over how we have been trying to do that. Uh, in the first 20 years or so, um, we really had no idea how to do it. So we tried all kinds of things. Uh, and we call this the "look ma no hands," which is like a small child riding a bicycle and showing off to their mother that they can ride the bicycle with no hands, but they don't really know what they're doing. And um, 1956 in Dartmouth was the official birthplace of AI, a conference at a university.
And um, some of the methods that were initiated in those days included, uh, what we call symbolic AI. So AI that reasons with symbols, think of them as words that refer to concepts, but also some of the earliest machine learning systems such as perceptrons, which were the precursors of today's neural networks.
And then from 1970 until about 2010, uh, AI developed along a fairly traditional, uh, engineering discipline route. Uh, the tools of that engineering discipline were logic, probability, statistics, optimization. These are well-developed areas of mathematics, uh, that we could use to build systems that reason, that learn, that plan, uh, and can be very successful.
Uh, in the late 1970s, early 1980s, there was a big boom, uh, in what we called expert systems. So these are systems that you could fill with expert knowledge, and they could do reasoning in order to solve problems, uh, and answer questions. Uh, and this technology was sold to many, many, many companies as the solution to all of their problems requiring knowledge and expertise. Unfortunately, the technology was too rigid, too narrow, too brittle. Uh, and it was perceived by about 1988 to have failed to deliver on its promises. Uh, and that led to the AI winter, uh, which is by analogy to the nuclear winter, where the the world is frozen for 10 years after a nuclear war. Uh, the AI winter—for 10 years there was no interest in AI, no investment. Uh, students didn't come to my classes. It was a very sad period.
After that, uh, so beginning in the the '90s, um, the technology of AI accelerated rapidly. Uh, lots of new ideas came. The mathematical depth of the field increased considerably, but still there was little commercial interest, uh, in the field. There were not big AI companies. There was not a large amount of investment.
Uh, starting around 2010, deep learning emerged. So the perceptrons of the 1950s got bigger and bigger and bigger, and now we call it deep learning. And uh, this technology made real progress on problems that had not been successfully addressed by the previous generations of AI technology, including speech recognition, computer vision, machine translation. Um, and now what are called foundation models, which are very, very, very large, uh, deep learning networks trained on enormous amounts of text data, are perceived as a potential basis for the general-purpose AI that I mentioned before.
So let me just go through those those ideas in a little bit more detail. So an AI system, we can think of as a box where sensory input comes in—could be from a keyboard, could be from a camera—um, and then is processed to produce behavior. That behavior could be turning the steering wheel on a car. It could be displaying answers on a screen. Uh, it could be speaking. Uh, it could be any number of things. And the question is what goes in between? How do you fill that box? And uh, we have tried lots of answers, uh, over the period of history.
So here's an answer that was tried in the 1950s. We fill it with Fortran programs. How many people have ever programmed in Fortran? Very few, and mostly the ones with no hair. Okay. Uh, so so Fortran was was the first real programming language, uh, in 1954, and um, what you did here, you filled it with programs, and then you made lots and lots of small mutations to those programs. You also took pairs of programs and mixed them together to make new programs. And the idea was this mimicked evolution. And if we did, did it for long enough, uh, we would evolve Fortran programs that were really intelligent. Uh, it didn't work. But at the same time, we have to remember that in those days, we were using a million, million, million times less computation. So 10 to the power of 18 less computation than we use now to build our biggest machine learning systems. So we actually don't know whether this method would work now. Uh, no one is trying it now, but perhaps it would work.
So for most of the history, as I said, uh, we built knowledge-based systems. So the the formally represented knowledge, uh, initially represented using mathematical logic, subsequently with probability, uh, and now with a technology called probabilistic programs, which I'll illustrate in a second, um, we were able to build very successful programs for reasoning, for planning, and so the AI system would take its input, use that input to, uh, augment what it knew about the state of the world, uh, use what it knows about how the world operates, uh, what we call the transition model of the world, to figure out what to do, uh, what's going to happen next, and generate behavior.
So I'll talk a little bit about probabilistic programming because it's not something that, uh, you see a lot about in the news, but it's a very, very powerful technology, uh, that began in the late 1990s, actually with some of my students, and what it does is it combines probability theory, which is the mathematics of uncertainty, uh, probability theory also underlies deep learning, u, but it combines probability theory with general, general-purpose programming languages or with first-order logic. And this gives us the ability to create extremely powerful representations of the world.
And just to give you an example, in a deep learning system or other type of circuit language, so deep learning is just very, very large circuits, uh, to write down the rules of Go would require about 1 million pages. Uh, in a language such as, uh, first-order logic or Python, uh, you can write down the rules of Go in about one page. This turns out to be really, really important, uh, and it's a source of power that deep learning is not able to access, uh, because of its restricted representational capability.
So I'll just give you a very simple example, uh, which is actually quite relevant for Korea, um, which is the Comprehensive Nuclear Test Ban Treaty. So the Comprehensive Nuclear Test Ban Treaty says nobody is allowed to test nuclear weapons anywhere on the earth, and uh, the organization which is in Vienna, uh, has a very large network of monitoring stations in every part of the world—hundreds of monitoring stations, mostly seismic—so they're measuring, uh, infinitesimal vibrations of the earth; they can measure movements of one nanometer in size, so just a few atoms; uh, they can detect a movement of that size. And so these, uh, seismic stations record the movement of the earth, and they send all that data to Vienna, and then every day, uh, the organization in Vienna produces a bulletin saying these are all of the seismic events—small or large earthquakes, explosions, landslides, volcanoes, and nuclear explosions—that happened today. Uh, and this problem is extremely important. About half of the budget for geophysics in the world is devoted either directly or indirectly to nuclear monitoring.
And so if we want to formulate nuclear monitoring using probabilistic programs, then we collect the evidence, which is the raw information coming from the seismic stations, and we ask a question, "What happened today?" And the probabilistic programming system simply answers that question. It answers it using a probability model. And that model represents the elementary geophysics of how and where events can occur in the earth—mostly on the surface or near, near to the surface—how signals from events are transmitted through the earth, uh, by various paths. Some of them go into the earth, round the core of the earth twice, and come back out the other side, uh, how those signals are detected, and then the level of noise in the background. So all of that model can be written down very concisely in a probabilistic programming language.
So here is the monitoring system for the Nuclear Test Ban Treaty. Uh, it took about 20 minutes to write this. Uh, and this monitoring system works about three times better than the previous monitoring system, which took the seismology community about a hundred years to develop. And here it is accurately detecting a nuclear explosion, uh, in your northern neighbor. Uh, so not far, not that far from here, uh, and it was able to do this instantaneously and also much more accurately than than the world's geophysicists were able to do, uh, using their own expertise.
But deep learning is is now the dominant method of doing artificial intelligence systems, and the idea of deep learning is you fill this box with a big circuit. The circuit has, uh, up to trillions of elements, and those elements are connected by connections whose strengths can be adjusted by the machine learning algorithm, and you simply adjust those connections so that the output of the system, uh, is a better, uh, representation of your objective. So it's it's, uh, using that objective to decide how to adjust those connection strengths in order to improve the achievement of the objective. And so stochastic gradient descent is the technical term for that.
Put another way, you take one of these big circuits, you do, um, about a trillion, trillion, trillion small random perturbations to the connection strengths of the network, uh, and you hope that it works after that.
So deep learning technology has done some amazing things. Uh, perhaps the most impressive result so far, in my view, uh, is solving the protein folding problem. So protein folding is: given a sequence of amino acids, so we know what the protein is made of, how does that sequence of amino acids fold up into the complicated structures that make life possible? Uh, and this is viewed for the last 50 years as one of the most important problems, uh, in biochemistry and biology. And uh, using machine learning, using deep learning systems, we've been able to basically solve it. So we can now correctly predict the structures of hundreds of millions of different proteins.
Another thing we can do with deep learning is scientific simulation. And here we're showing simulation of, uh, the climate over several weeks. And uh, this is a problem that could be solved before, but required months and months and months of computing on the biggest computers that we have in the world. And using deep learning, we can speed up simulation by about five orders of magnitude. So a 100,000 times faster with the same accuracy. And so now because we can then run more simulations, uh, varying the parameters of the simulation, the details of the model, uh, we can get much better predictions; we can also get much better estimates of the uncertainty in the predictions, and so this is revolutionizing weather forecasting and climate modeling and many other areas that require scientific simulation.
We can do the same kind of thing except with physical objects. So we can create and then optimize the structures that we need to build stuff. Um, this is a piece of a wheelchair. This is the piece that carries the electric battery for a motorized wheelchair. The one on the left is the one designed by a human being. The one on the right is the one designed by a generative AI system, a machine learning system. And it turns out to be 40% lighter and considerably stronger. Um, and what we find is that these structures, uh, actually look more organic. They look more like the kinds of structures that nature produces in plants, in cells, and so on. Uh, and they're more efficient than the ones that human beings have been able to design.
So I think we'll also see, uh, rapid progress in robotics, and that's happening where robots that are most out there in industry are mostly very inflexible. They're very useful, but they rigidly do the same thing over and over again. Uh, whereas if you want a robot to work in a in a warehouse at Amazon, uh, it has to be able to deal with hundreds of thousands of different products. So it can't do the same thing over and over again; uh, it has to be able to see, uh, and manipulate all kinds of different products. So now Amazon is starting to roll out robots that can do the job of the human warehouse workers. They have over a million human warehouse workers just in Amazon. And so, uh, if these robots are successful, uh, it will have a significant impact on employment opportunities.
Uh, self-driving cars, I think are a very familiar, uh, concept, and in San Francisco, they've become very familiar, actually, as, um, something for tourists to do. So tourists used to come to San Francisco to ride on the famous cable cars that climb up and down the very steep hills in the city. Uh, now they come to ride on the self-driving taxis, uh, that are all over the city delivering people. Mostly those taxis have worked pretty well. Um, but they work because San Francisco has been completely mapped before these taxis ever arrived. Uh, we created 3D, very, very detailed maps of the city. Uh, and so the cars are not really perceiving the world in the same way that you or I do. They have a 3D model inside, and they're just registering themselves: Where am I relative to my 3D model? Uh, the problem is that driving is not just, uh, following a white line. Driving—you have to deal with all kinds of things. For example, you have to make sure that you don't drive into wet cement, uh, which this car did. Uh, that was that was not a good idea. Uh, it had absolutely no idea because it had never seen wet cement before. There was no wet cement in the training data. So it just had no idea what to do and drove straight into it. More seriously, uh, the cars also don't know what it's like when you're dragging a pedestrian under your wheel. Uh, and so one of the Cruise taxis dragged a pedestrian for about 50 meters along the road and caused very serious injuries. Um, and, uh, the state shut down Cruise's robotaxi or self-driving taxi operation. And so we're still not seeing self-driving cars, uh, available to buy, uh, or driving all over, uh, the US or other countries because they lack the common sense to cope with these unusual circumstances.
So healthcare is another important application, and, uh, one of the most, uh, commonly, uh, described applications for AI in healthcare in the media is skin cancer detection. So you take your cell phone, you take a picture, it tells you, "Oh, that that looks fine, but that one there, that might, might be a cancer, so you should go get it looked at." So this is an this is an article from 2011. So 13 years ago, lots of excitement, amazing breakthrough in healthcare. Uh, three years later, the, uh, federal government said, "None of these skincare apps are actually working, and so we're going to take them all off the market." At 2018, "Wow, we have skin cancer apps that do a great job of diagnosing skin cancer." Uh, and that was 2018. Two years later, we find out they don't work. So healthcare is an area where, again, as with self-driving cars, there's been a lot of promising but not much delivering so far, and it's really because, uh, medicine is a very, very complex area where doctors have a lot of knowledge of how the human body works and how disease processes work, uh, and that's very different from simply shoveling tons of data into a deep network and hoping for the best.
Uh, in my view, education is the area where we could have the biggest impact in this decade because, uh, the large language models, variants on ChatGPT, are able to answer, uh, and understand, uh, a lot of the material that we teach, at least in, uh, the first few grades of school. If we could do this, um, I believe that we could actually improve the effectiveness of our education system by a factor of two or three. So we would have many children who are ready for university by the age of 11. And so that could be transformative both for the opportunities available to children, uh, and for our economies and our societies. So there are a lot of, still difficult problems, and education is not at the moment perceived to be a high-profit market. So companies have a very hard time figuring out how to make enough money, uh, by applying AI to education, and so most of the big AI companies are not focusing on this as an area.
So what I've told you so far is that deep learning is, uh, a very successful technology, but that it has some limitations, and the limitations come from the fact, uh, that it involves circuits, and those circuits are linear time computing systems. So what does that mean? It means that when, when input comes into the circuit, it just goes through the circuit and comes out the other side. And so the amount of computation that happens is exactly proportional to the size of the circuit. Now there are a whole class of computational problems that just take longer than that. You cannot solve them in a linear amount of time. In fact, for many of them, what we call the NP-hard tasks, uh, they require an exponential amount of time. And so what does that mean? It means that if those circuits are going to solve those kinds of problems, the circuit size itself has to blow up exponentially. So you get very, very large circuits to solve these problems. And in order to train a very, very large circuit, you need a very, very, very large amount of data. And that's exactly what we see. So the deep learning systems that learn to recognize objects like a giraffe—when you teach a child what a giraffe looks like, do you buy a book with three million pictures of giraffes and spend 12 weeks paging through all of those pictures? No. There's one picture of a giraffe. It's usually just a cartoon. You know, a big yellow animal with brown splotches and a long neck. And after that, the child can recognize giraffes in any situation, uh, for the rest of their lives from one example. And so this is the, um, the weakness of our current approach to AI is the vast data requirements in order to achieve even approximate representations of these difficult concepts.
So I'll give you an example of that. In 2017, um, AlphaGo defeated, uh, Ke Jie, who was the Chinese, uh, number one world-rated Go player. And this was the Sputnik moment for China. This was when China decided that supremacy in AI was essential for China's global dominance. And since this time, when it when the programs beat the world champion, they have gone on to be massively superhuman. So I'm going to show you a game between, uh, KataGo—or here it's JBX KataGo00005, which is one particular version of KataGo—and its rating is 5,200. The human world champion rating is 3,800. So it's massively superhuman. And I'm going to show you a game with, uh, one of our, uh, graduate students, Kellen Pelin. He's a good Go player. He's an amateur player. His rating is 2,300. So he should have no chance whatsoever against a massively superhuman program, but also Kellen is going to give a nine-stone handicap to KataGo. So that means KataGo starts with nine stones on the board. Um, and so it should, should be even easier for the computer to win this game, right? Nine-stone handicap is what you give to a small child when they're learning how to play so that they can stay in the game. So let's have a look what happens. So remember, uh, the human player is white, and the computer is black, and you should keep an eye on what happens in the bottom right quadrant. So, Kellen makes a small little group of white stones, and then, uh, KataGo very quickly begins to surround that group to prevent it from growing. And then Kellen begins to surround the black group. And the computer doesn't pay any attention for reasons we don't understand. It simply ignores the fact that Kellen is about to capture all these stones. And, uh, once the computer loses those stones, it very quickly loses the game. And so Kellen was able to beat KataGo 15 times in a row. Also, all the other leading Go programs very easily. And what we see here is that in fact the Go programs have not learned the basic concept of a group of stones. So they recognize some groups correctly, but these circular groups, they don't seem to recognize at all. They simply don't understand what's happening. And so they're very easy to beat. So we thought that computers were superhuman, but in fact, they're not.
So a question then arises, right? Many people are predicting that AI systems will exceed human capabilities in every area. What happens if we succeed? So I think if we succeed, we could use those AI systems to deliver our civilization, the nice civilization that we enjoy, um, in, uh, the developed countries. We could deliver that to everyone on Earth because AI systems can do all that stuff at much greater scale and a much lower cost. So just looking at delivering a middle-class standard of life to everyone on earth would be about a 10-fold increase in global GDP. And so the net present value, the cash value of general-purpose AI as a technology, is at least 15 quadrillion dollars. So that sort of explains why we are investing so much money, uh, in creating this technology. And we could do many other things besides—maybe better education as I mentioned, better healthcare eventually, and so on. We do have to worry about what human beings are going to do when the AI systems are looking after everything for us. Uh, and in Wall-E, the prediction is the humans become infantilized.
But the thing I'm going to talk about in the small amount of time remaining is how are we going to cope with systems that are more intelligent than us? This is Alan Turing, the founder of computer science, and he said at some stage we should have to expect the machines to take control. And this is a natural reaction when you think about building systems that are more intelligent and therefore more powerful than human beings. So how do we retain power over entities more powerful than us forever? Seems like a difficult question. But we can ask that question a slightly different way. We can say we get to choose how to design these systems. So what are we designing them to do? What is the mathematical problem that they are solving? And can we design that problem such that when the AI systems solve it, we are guaranteed to be happy with the result?
So think about it this way. We don't really want to solve the problem of intelligence. What we want is to solve the problem of helpfulness, right? Of AI systems that are very, very intelligent, but specifically intelligent in a way that is helpful to us. So how do we define that? Actually, it turns out to be not that difficult. You set the objective to be that they have to act in the best interests of humans. But initially, those machines are explicitly uncertain about what those human interests are. So they have two jobs: one is to help us, but two is to figure out what we want in the first place. And while they are uncertain about what we want, they will behave in ways that are actually allow us to remain in control. So we can turn this formulation into a mathematical problem called an assistance game. And we can show that assistance game solvers, so AI systems that conform to these principles, are guaranteed to remain under human control. We can prove mathematically that they are willing to be switched.
Off. For example, they have an incentive to allow us to switch them off as long as they are trying to solve this problem. And we can show that it's actually in our best interests to build AI systems that solve this mathematical problem. So you might say, well, then surely everyone in the world is building these assistance game solvers because this seems like a good idea.
Actually, no. There are really only two options for the human race: We can build safe AI, or we can have no AI at all. Right now, we're pursuing the third option, which is building completely unsafe AI and trying to make it superhuman. Uh, so here, this is a New York Times story, uh, showing that the employees of OpenAI, which makes ChatGBT, are blowing the whistle. They're saying OpenAI does not care about safety. They just want to build superhuman AI, and they don't care about the consequences. I think I'm going to have to skip over this in the interest of time.
Um, so here's what I think is going to happen. There are maybe half a dozen futures that could unfold. This is my current best guess about the most likely path that the world is going to take. So the first: further scaling of these large language models, such as Chat GPT, is not going to lead to AGI. I think the big AI companies already understand this, and they are working on alternative and complimentary approaches. They claim to be making a lot of progress, and I think it's likely that within a decade we will see those transformative advances where AI systems start to exceed human capabilities in very important ways—not necessarily in all ways, but in ways that are enough to create massive transformation in our world and to pose significant risks to us.
I also believe that governments are not going to legislate and enforce regulations on the safety of AI systems. So they are going to allow the companies to do what the companies themselves claim will lead to at least a 25% chance of human extinction. So, in the best case, we will have a Chernobylized disaster, and then the governments will wake up and do something. That's the best case. The worst case, obviously, is that the disaster is something that's irreversible, uh, and we lose control.
So, to sum up, I believe AI has vast potential to benefit humanity. Because of that, it has unstoppable momentum. But the path that we are currently following leads to a loss of human control. What I'm arguing is that there's really only one path that we can follow, and that's provably beneficial AI. AI that we can guarantee mathematically is safe and beneficial to the human race. So, we should stop talking about making AI safe, which is an after-the-act sticking plaster trying to fix the problems of unsafe AI, and instead make AI that is safe by design from the beginning. Thank you very much.