Transcription
This is a story about preparing for the worst. Maybe the worst thing any of us can imagine.
A year ago, we talked with computer scientist Jeffrey Hinton just days after he won the Nobel Prize for his work in machine learning. The so-called godfather of AI has been busy since then. Not developing artificial intelligence, but warning people about it. He says we've all become more aware of the risks, but knowing about them isn't enough. We need to act. Mhm.
"Suppose that some telescope had seen an alien invasion fleet that was going to get here in about 10 years. We would be scared and we were doing stuff about it. Well, that's what we have. We're constructing these aliens, but they're going to get here in about 10 years and they're going to be smarter than us. We should be thinking very, very hard. How are we going to coexist with these things?"
Coexistence and control, two things that Jeffrey Hinton himself has been thinking very hard about. As one of the computer scientists who helped make modern AI possible, he's uniquely well suited to consider its future and who, if anyone, can shape it.
"Are there companies who are doing real work on safety? I mean, we hear about Anthropic, we hear about DeepMind. Are they helping on the safety front?"
"Yes, I think both Dario Amodei and Demis Hassabis, and also Jeff Dean, they all take safety fairly seriously. Obviously, they're involved in a big commercial competition too. So, it's difficult, but they all understand the existential threat that when AI gets super intelligent, it might just replace us. Um, so they worry about it a bit. I think that some companies are less responsible than others. So for example, I think Meta isn't particularly responsible. OpenAI was founded to be responsible about this, but it gets less responsible every day and their best safety researchers are all leaving or have left. Yeah, I think Anthropic and Google are somewhat concerned with safety and the other companies less so."
"As I talk to some of the people at some of the companies you're talking about and raise the question of safety, I often am told, 'Don't worry, you're pretty little head about it. Uh, we have great computer scientists who are on top of this. We're far off from any real danger and our computer scientists will know soon enough.' So we're much more concerned about the race to become dominant."
"Yes, that that's the problem. They are much more concerned about the race. They should be much more concerned about um whether humanity will survive it. Also whether society will survive it if you get massive unemployment. There's one piece of good news, which is all the different countries are aligned in not wanting AI to take over from people. They're anti-aligned for things like cyber attacks or autonomous weapons. They're somewhat aligned for creating viruses. None of them really wants other countries to create viruses. On AI taking over, they will collaborate because nobody wants that. The Chinese Communist Party doesn't want AI to take over. Trump doesn't want AI to take over. They can collaborate on that. That leaves the question of how do we prevent it taking over? Even if we all the countries collaborate, what do you do? And I think at present um all the big companies and governments have the wrong model. So they their basic model is, 'I'm the CEO and this super intelligent AI is the extremely smart executive assistant. I'm the boss. I can fire the executive assistant if she doesn't do what I want.' Um, and I just sort of say, 'Make it so,' bit like Star Trek. And the super intelligent AI makes it so and I get the credit. Great. It's not going to be like that when it's smarter than us and more powerful than us. Um, that's just the wrong model. I believe we need to look around and say, is there any model where a less intelligent thing controls a more intelligent thing? And we have one model of that and it's a model we all know which is a baby controlling a mother. Evolution put lots of work into allowing the baby to control the mother. And the mother is actually often more concerned about the baby than about herself. Doesn't work like that with rabbits but it does work like that with people. That seems a much more plausible model of how to coexist with the super intelligence. But we have to accept that we're the babies and they're the mothers. Hopefully they're not Jewish mothers, but you can't imagine these tech bros accepting that model. They just don't think of the world like that."
"Is the United States behind China in developing generative AI right now?"
"Not yet. The United States is still a little bit ahead, but not as far ahead as they thought. Um and in China you've got a very large number of very competitive, very smart people, very well educated in um science and engineering and math. Um, they're educating far more people than the US in those areas. The US has basically relied on immigrants to be smart at those things. China may well overtake the US. And if there's one thing you wanted you would do if you were Chinese to ensure that China overtakes the US is you would stop the funding of basic research in the US and you would attack the good research universities. Trump looks like he works for Putin but actually in attacking the universities and attacking the funding of basic science um he's acting as if he's working for Xi."
"How deep is that damage? By the way, it's it's the immigrants you talked about as well. It's it's not just the direct funding for the research. It's also the brain power coming in from overseas. How deep is that damage and how immediate may we feel it?"
"The point about attacking basic research is you don't really feel it for 10, 15, 20 years because what you do is you ensure that the really big conceptual breakthroughs won't happen here. Um, and then you later on, um, the Chinese will be way ahead."
Regardless of who becomes the frontrunner in the AI race, Hinton says the risks to everyone have gone up over the past year, particularly for workers, as we saw just this week when Amazon announced it would be cutting 4% of its workforce, perhaps made both possible and necessary by unprecedented levels of AI investment.
"There's been an enormous amount of money put into since you and I spoke a year ago. I mean amount that I could not have conceived of actually."
"I mean of the order of a trillion if you add it up over all the companies."
"So what is that money going for and will it ultimately redound to anyone's benefit?"
"These are big companies run by serious people and presumably they wouldn't be putting all that money in unless they thought they could get a return on it. Um, there's some ego involved. They want to be the ones to do it first, even if it's going to kill us all. Um, so there's ego involved. Um, but presumably they think there's returns to be made. My worry is that the obvious way to make money out of it, apart from charging fees to use the chatbots, is by replacing jobs. Um, the way you make a company more profitable is replace the workers with something cheaper. And I think that's a big part of what's driving it."
"Is it a winner take all in the end? I mean, in terms of"
"Basic, I don't know. I mean, one thing I should say is that this is sort of uncharted territory. We've never had things almost as smart as us, which we have now, or things smarter than us, which we'll have soon. We've never been there. We've had things in the industrial revolution that got more powerful than us, but we were always in charge of them. you know, a steam engine is just a lot more powerful than a horse. But, um, we control the steam engine. This isn't like that. Also, if you got unemployed because you used to do ditches, now you have to do something else. Um, you could get a job in a call center. Um, but now those jobs are all got going to go. It's not clear where those people go. Some economists say, um, these big changes always create new jobs. Um, it's not clear to me that this will and I think the big companies are betting on it causing massive job replacement by AI because that's where the big money is going to be."
"As you say, some economists say we go back in history and new technology destroys some jobs but creates other jobs and net net you have as many or more jobs. Uh, you're saying this time is different. Can the investment, the trillion dollars or more investment, can it pay off without destroying jobs?"
"I believe that it can't. I believe that um to make money, uh you're going to have to replace human labor."
Given the dire warnings about AI's risks to workers, economies, and humanity as a whole, one wonders whether Jeffrey Hinton has any regrets about his pivotal role in giving it life. We asked ChatGPT how it would describe its relation to the man many people call its godfather. Its answer, "If I'm the mature rainforest, Hinton is one of the people who planted the first seeds and figured out how to water them." Still, the question of whether it was worth it is the one that gave him pause.
"To ask an unfair question, you were sort of there at the birth. If you had it within your power, understanding it's not going to happen, would you stop AI altogether given the risk?"
"I don't know cuz there's also you have to remember it's not like nuclear weapons which are only good for bad things. It's a difficult decision because it can do tremendous good to in healthcare and education. Um, it'll do tremendous good and in fact if you think about it increasing productivity in many many industries that should be good. The reason it's bad is because of the way society's organized. So that Musk will get richer and a lot of people get unemployed and Musk won't care. Um, I'm using Musk as a sort of stand-in. Um, that's not on AI, that's on how we organize society."
"I wonder if over the last year the economy and the markets haven't worked against you in this sense. So much of the growth in the stock market, so much of the driving the economy is investment in AI right now. Even if the public were more concerned than they are about some of the risks you describe, they're going to say, 'Wait a second. That's what's driving our economy. We don't want to give that up. We don't want to go into a recession.'"
"Some people say that our best hope is to have AI try to take over and fail. Um, we need something to really scare the out of us. Something like Chernobyl for AI. Um, I'm not sure I agree with that, but that's certainly a possibility."
"Or the Cuban missile crisis."
"Or the nuclear because one of the questions I had was even if the government sort of agree in general we should do this, is there a sense of urgency? I think the Cuban missile crisis probably gave a sense of urgency on nuclear disarmament."
"Yes. Um, we need something to make people um, pay more attention and put more resources. So, at present, the big companies aren't going to put like a third of their resources into figuring out how to make it safe. But if it tried to take over and only just failed, maybe they"