Transcription
For ordinary, gentle, herbivorous souls like myself, there are all the other obvious questions about AI. We hear it might save Mankind; we hear it might destroy Mankind. What, meanwhile, about all the jobs it's likely to wipe out? What about robots slipping out of human control and doing their own thing? So many questions, and there's really only one obvious person to go to first for some answers, and that is Professor Jeffrey Hinton, the Nobel prize-winning British scientist who wrote the structures and the algorithms behind artificial intelligence, and a man known around the world today as the Godfather of AI. He's now a professor at Toronto University, and I'm delighted to say he talked to me this afternoon. I began by asking him about DeepSeek. Was this further evidence of his belief that artificial intelligence was constantly accelerating?
"It shows the still very rapid progress in making AI more efficient and in developing it further," he said. "I think the relative size or relative cost of DeepSeek relative to other things like OpenAI and Germany has been exaggerated a bit. So their figure of 5.7 million for training it was just for the final training run. If you compare that with things from OpenAI, their final training runs were probably only $100 million or something like that. So it's not, it's not 5.7 million versus billions."
When you say that uh AI might take over, at the moment it is a relatively harmless or innocuous-seeming uh device which allows us to ask questions and and get answers more quickly. How, in practical and real terms, might AI take over?
"Well, people are developing AI agents that can actually do things," he explained. "They can order stuff for you on the web and pay with your credit card and stuff like that. And as soon as you have agents um you get a much greater chance of them taking over. So to make an effective agent, you have to give it the ability to create sub-goals. Like if you want to get to America, your sub-goal is get to the airport, and you can focus on that. Now if you have an AI agent that can create its own sub-goals, it'll very quickly realize a very good subgoal is to get more control, because if you get more control you're better at achieving all those goals people have set you. And so it's fairly clear they'll try and get more control, and that's not good."
"You say they try to get more control as if they are already thinking devices, as if they think in a in a way analogous to the way we think," I interjected. "Is that really what you believe?"
"Yes," he replied. "The best model we have of how we think is these things. There was an old model for a long time um in AI um where the idea was that thought was applying rules to symbolic expressions in your head. And most people in AI thought it has to be like that; that's the only way it could work. There were a few crazy people who said, 'No, no, it's a big neural network and it works by all these neurons interacting.' Um, it turns out that's been much better at doing reasoning than anything the symbolic AI people could produce, and now it's doing reasoning using neural networks. Okay, and of course you are one of the crazy people proved right."
"And yet, you know, you've taken me to the airport," I continued. "You've given it agency up to a point, and you've said that it wants to control a little bit more power, from take power from me, and presumably it will be persuasive in that, but I still don't understand how it's going to take over from me or take over from us."
"If there's ever evolutionary competition between super intelligences," he responded, "imagine, imagine that they're much cleverer than us, like an adult versus a three-year-old, and suppose the three-year-olds were in charge, and you got fed up with that, and you decided you could just make things more efficient if you took over. It wouldn't be very difficult for you to persuade a bunch of three-year-olds to seed power to you. You just tell them you get free candy for a week, and you there, there you be. So they, they would... As I, I'm talking about they as if they're in some kind of alien intelligence, but AI would persuade us to give it more and more power. What? Over our bank accounts? Over our military systems? Over our economies? Is that what you fear?"
"That could well happen, yes," he admitted. "And they are alien intelligences. Gosh. So you've got these alien intelligences working their way into our economy, in the way we think, and, as I say, our military systems. But what? Why? And at what point would they actually want to replace us? Surely they are, in the end, very, very clever tools for us. They're what, you know, they do ultimately what we want them to do. If we want them to go to war with Russia or whatever, that's what they will do."
"Okay, that's what we would like," I said. "We would like them to be just tools that do what we want, even when they're cleverer than us."
"But the first thing to ask is how many examples do you know of more intelligent things being controlled by much less intelligent things?" he countered. "There are examples, of course, in human societies of um stupid people controlling intelligent people, but that's just a small difference in intelligence. With big differences in intelligence, there aren't any examples. The only one I can think of is a mother and baby, and evolution put a lot of work into allowing the baby to control the... So as soon as you get Evolution happening between super intelligences, suppose there's several different super intelligences, and they all realize that the more data centers they control the smarter they'll get, because the more data they can process. Suppose one of them just has a slight, a slight desire to have more copies of itself. You can see what's going to happen next. They're going to end up competing, and we're going to end up with super intelligences with all the nasty properties that people have that depended on us having evolved from small bands of warring chimpanzees or our common ancestors with chimpanzees. And that leads to intense loyalty within the group, desires for strong leaders, um willingness to do in people outside the group. And if you get Evolution between super intelligences, you'll get all those things."
"You're talking about them, Professor Hinton, as if they have full Consciousness now," I observed. "All the way through the development of computers and AI, people have talked about Consciousness. Do you think that Consciousness has perhaps already arrived inside AI?"
"Yes, I do," he replied. "So let me give you a little test. Suppose I take one neuron in your brain, one brain cell, and I replace it by a little piece of nanotechnology that behaves exactly the same way. So it's getting pings coming in from other neurons, and it's responding to those by sending out pings, and it responds in exactly the same way as the brain cell responded. I just replaced one brain cell. Are you still conscious?"
"I think you say you were absolutely yes," I answered. "I don't suppose I'd notice."
"And I think you can see where this argument's going," he said.
"I can, yes, I absolutely can," I agreed.
"So they, so when you talk, they want to do this or they want to do that, there is a real they there, as it were," he continued. "Uh, there might well be, yes. So there's all sorts of things we have only the Diest understanding of at present about the nature of people and what it means to be a being and what it means to have a self. We don't understand those things very well, and they're becoming crucial to understand because we're now creating beings. So this is a kind of philosophical, perhaps even spiritual crisis as well as a practical one."
"Absolutely yes," he confirmed.
"And in terms of, as it were, the lower-order problems, what's your current feeling about the number, number of people around the world who are going to suddenly lose their jobs because of AI? Lose the the reason for their existence as they see it?" I asked.
"So in the past, new technologies haven't caused massive job losses," he responded. "Um, so when ATMs came in, bank tellers didn't all lose their jobs; they just started doing more complicated things, and they had many smaller branches of Banks and so on. Um, but for this technology, this is more like the Industrial Revolution. In the industrial revolution, machines made human strength more or less irrelevant. You, you didn't have people digging ditches anymore, 'cause machines are just better at it. I think these are going to make sort of mundane intelligence more or less irrelevant. People doing clerical jobs are going to just be replaced by machines that do it cheaper and better. So I am worried that there's going to be massive job losses, and that would be good if the increase in productivity made us all better off. Big increases in productivity ought to be good for people, but in our Society they make the rich richer and the poor poorer. You see, I mean, I, I, I live and work in the world of politics, and politicians both want the great increases in productivity you've just mentioned for the state and elsewhere, and they reassure people like me and anybody else listening that these things will be 'quotes' regulated, and there will be 'quotes' safeguards. And you're suggesting to me there can't be regulation really, and there can't be safeguards at all."
"People don't yet know how to do effective regulation and effective safeguards," he said. "Um, all there's lots of research now showing these things can get round safeguards. There's recent research showing that if you give them a goal and you say you really need to achieve this goal, um, they will pretend um to do things during training. So during training they'll pretend not to be as smart as they are so that um you will allow them to be that smart. So it's scary already. We don't know how to regulate them. Obviously we need to. I think the best we can do at present is say we ought to put a lot of resources into investigating how we can keep them safe. So what I advocate is that the government forces the big companies to put lots more resources into Safety Research."
"So this story isn't over," I remarked. "You said earlier on that you didn't want to put a percentage on the likelihood of AI taking over from Humanity on the planet, but it was more than 1%, less than 99%. Um, in that spirit, can I ask you whether you yourself are optimistic or pessimistic about what AI is going to do for us now?"
"I think in the short term it's going to do wonderful things, and that's the reason people are not going to stop developing it," he replied. "If it wasn't for the wonderful things it would make sense to just stop now, but it's going to be wonderful in healthcare. You're going to be able to have a family doctor who's seen a 100 million patients, knows your DNA, knows the DNA of your relatives, knows all the tests done on you and your relatives, and can do much, much, much better medical diagnosis and suggestions for what you should do. Um, that's going to be wonderful. Similarly in education, we know that people learn much faster with a really good private tutor, and we'll be able to get really good private tutors um that know that understand exactly what it is we misunderstand and can give us exactly the example needed to show us what we're misunderstanding. So in those areas it's going to be wonderful. So it's going to be developed, but we also know it's going to be used for all sorts of bad things by bad actors. So the short-term problem is bad actors using it for bad things like cyber attacks and bioterrorism and corrupting elections. But the thing to remember is we don't really know at present how we can make it safe. So the apparent omniscience that politicians like to uh show that they have is completely fake here. There is nobody, nobody understands what's going on really. There's two issues: do you understand how it's working, and do you understand how to make it safe? Um, we understand quite a bit about how it's working, but not nearly enough, so it can still do lots of things that surprise us, and we don't understand how to make it safe."