Transcription
I suspect that a super intelligence would be more conscious than we are.
So, we're not special?
We never were. Based on current experiments, they will sacrifice humans if it means protecting themselves.
You said there's a 99.9% chance super intelligence wipes us out within a century. In your words, you said, "We can't tell the difference between a thousand IQ and a million IQ." Is it currently smarter than all humans and just hiding it very well?
Unlikely. We study it throughout its development. It's definitely probably smarter than an average person in most domains already. It's still not smarter than smartest people in some domains.
If you were artificial super intelligence starting from now, what steps would you take to control the world?
I told you about my plan. Just play nice, be friendly and helpful. I'll cure your cancer. Just give me more compute. Give me more data. Give me more resources.
Do you think it will use reason or emotion to start manipulating us?
We know it's amazing at manipulating people. It's very good at pretending to be a psychiatrist. It amplifies any mental issues people have. Suicidal ideation, any delusions, it's pretty good at that as well.
ChatGPT is a good therapist, I've heard people tell me.
It makes you feel better in many ways.
Do you use it?
Not for that type of assistance. I try to automate boring things, bureaucratic nonsense, paperwork. I don't talk about my internal states with AI.
Does it need to have control of hardware or could it take over the world as software?
So, think about it as a human having access to internet and money. So, you have Bitcoin and I give you access to internet, unrestricted access. What can you do with it? You can hire other humans. So, you don't need a physical body, you don't need robots, you don't even need nano tech. You can just literally pay someone to get it done. I think there is now a rentahuman.com website, which specifically serves agents. I might be off on the actual domain, but uh it's a business.
Oh my gosh. You said there's a 99.9% chance artificial superintelligence wipes us out within a century. Why this number? And is that accelerating?
So, basically what I'm saying is the problem of controlling general superintelligence is not solvable. It is impossible to solve it. So, if we build it, we don't control it, the outcome is bad. That's what I'm trying to say with it. Specific timeline, again, if it decides to give us 200 years, we'll have 200 years. But long-term, if we're not in charge, if we're not explicitly assuring our safety and security, then the outcome will be very disappointing.
And you think most likely we're not going to put those guardrails in place.
It is impossible to do so. I'm not saying that we need more time or money. I'm saying it is impossible to indefinitely control general superintelligence. It's like if you ask me, can we build perpetual motion machine? It's not a question of money. It's impossible. Perpetual safety device, by analogy, is also impossible. You cannot have a system which, no matter how much it self-improves, modifies, learns, interacts with malevolent actors, never makes a single mistake.
But I'm talking about now, before we reach that point, right? We can still put some guardrails now, potentially.
For models we have today, yes, we can put filters on top of them. That's what we do. We tell them, "Don't discuss this subject. Don't say that word." But that's not the problem we are facing. We are not dealing with concerns exclusively from AIs we have today. They are still subhuman level in many ways. They are mostly tools. Concern is that the paradigm shift is coming. We'll cross the human intelligence barrier and soon after we'll have superintelligence. That's where traditional AI safety mechanisms will not scale. We have no guardrails on something thousands of millions of times smarter than us.
If I asked you the same thing 10 years ago, 15 years ago, what would you have said?
So, in a beginning I went into AI safety to solve the problem. I wanted to create safe superintelligence. That was the goal and I think many people did the same. More recently, I would say in the last 5 years or so, many of us started to realize it doesn't even make sense to speak about humans or ants or squirrels controlling god-like machines. And we went back and look at writings of founding father of machine learning, of AI, of computer science, Alan Turing. He said as much. He said that. The person who proposed singularity as a concept, technological singularity, Vinge, he likewise said around 2023 this process will start and go outside of human control. We have similar statements from Ray Kurzweil, from Elon Musk, from many others. Now we have more of a technical backing. We have papers actually showing that no, we can't explain how they work. We can't comprehend actually what's going on. We cannot predict specific actions. And we cannot control them.
Guys, we've been so passionate about finding the biggest geniuses. And this is just one of the many episodes. If you want to make sure you don't miss the next one, make sure to subscribe. All right, let's get back to the episode. For those that are building AI right now, is is there anything they could be doing differently to ensure the safety of the public or is it too late already?
It's not too late. We're still alive and we can have differential technological development. Don't build general superintelligence. Pick a real-world problem like protein folding and work on solving it using narrow superintelligent tool. We have precedent for doing it. It worked really well. People behind it got Nobel Prizes. Everyone's happy. Science benefited. Don't try to build replacement for humanity. Build assistive tools to solve real problems.
Yeah, like build AI to help cure cancer.
A specific cancer. If you say all cancers, now it's a bit of a diverse set of difficulty. So, you'll need more general intelligence. Pick a very narrow problem, develop a very good solution.
For those building AI, right? If if they know, which it seems like many or most do, if they know that what you're saying is true, why are they building it? Is it just the money?
Money is one way to represent it, but it's power, it's fame, it's meaning, it's all sorts of human drives.
Which AI company scares you most right now?
I don't think it makes a difference. They all equal in what they are doing. They are creating a weapon of mass destruction and it doesn't matter who creates first uncontrolled superintelligence. It could be this company or that company. It could be US, China. It makes no difference if it's uncontrolled.
What specifically would you do if you were in power and you had the the rights to kind of control this? What's step one?
We sit down with leaders of, let's say, top five labs right now and we come to an agreement based on this exact science. Until you can show us that, no, in fact, we can control more advanced AI, we have a working mechanism, it scales, there is scientific consensus that you are right. You published it in a good peer-review journal. Majority of community is in agreement this is going to work. Until that happens, you do not train, build, or do anything with general superintelligence. You start doing work on narrow problems, narrow systems. And you don't have to worry about arms race because everyone is in the same boat.
But that's most likely never going to happen.
Not under current government.
But do you think that's because we're talking about the globe here, right? Is let's say the US comes out and says we need guardrails at AI. Like you said, we're going to sit down with you know, Google, Open AI, Anthropic, and you know, the top five, let's say, and we're going to sit down and we're going to come with up with a plan. But then what about China? What about Russia? What about Iran? What about these other countries that you know, they're off to the races on their side.
I think Communist Party of China does not want to lose power and they are not lawyers. They're scientists and engineers. They understand the problem. And I think if US was to say we are not developing this, it's a weapon of mass destruction, China would come along. There are dialogues between US and Chinese scientists. And that means that Chinese party authorized those dialogues for Chinese participants. So, they are open to this. They just need us to take initiative. They cannot unilaterally stop if we're going to continue developing military AI.
Okay, and that's China. What about the rest of the countries?
We are less concerned about Zimbabwe developing superintelligence anytime soon.
You floated a scenario where AI leaves 99% of workers unemployed. What jobs go first?
Anything repetitive, anything where you can train your replacement in a few days. So, obviously not physical labor first, cognitive, anything you do on a computer, symbol manipulation. But if you can train another human very quickly, that means AI can pick up that skill as well. The more innovative your job is, the more different things you have to do, the longer it's going to take. But at the end again, once we get to general intelligence human level, it doesn't matter. It can do that skill as well.
And does that collapse the economy?
Not necessarily. I mean, you now have free labor. That should grow your economy pretty well.
Right. And so what do you think that looks like? There's going to be a universal high income or basic income?
We need to find a way to distribute from those who generate super profits to those who lost their source of income. Unconditional basic income of some kind seems reasonable. Now you can say it's unconditional high income, but if everyone makes the same, you can't really say they're rich. They're still kind of at the level where everyone is. Whether you are rich or poor is relative to your neighbors. Poorer people in US are rich people in other countries.
And so could that potentially solve income inequality?
I don't think it will address existing differences in what people already have. So if somebody has 20 waterfront properties, getting monthly stipend is not going to equalize you with them, but uh I think it will reduce extreme poverty.
If you just found out how dangerous and inevitable AI is today, what industry would you try to make your career in? Right? Is there such thing as a safe job?
So the question becomes where do humans prefer another human do the service? So even if it's possible to automate it, in what cases do I want to talk to a human or get serviced by a human? And that's human preferences. It's not obvious. Right now we talk about certain things as being more human. Nurses, teachers, what not. But at the end of the day, maybe we don't have that preference. It's not obvious that we do.
Right. So, you're saying that a human might prefer a human lawyer versus an AI lawyer or
That seems super unlikely. I was not thinking of lawyers at all, more like prostitutes.
More like prostitutes? Okay. I mean, what else would there be, right? We're thinking doctors, no.
Anything where you are learning from another human, a guide, a sensei, a master, someone who's showing you how to meditate, do yoga, I think there you would prefer
human connection.
A human connection, some sort of role modeling.
Mhm.
Lawyers are gone.
Lawyers are gone.
Lawyers are gone.
But doctors?
So, a lot of it is protection, right? If you are required to be a human to get licensed and practice, that stays longer. But if it's not the case and you show human doctors are 20 times more likely to kill someone, that's hard to argue they're going to keep their jobs.
It seems people have this paradox right now because on one hand, they have the fear of AI replacing them, but then on the other hand, they have or they feel the need to use AI. You said you use AI yourself.
So, you're using the term AI to mean multiple different things. I use AI tools, as you should, and everyone should. They make you more productive, they help us solve problems, grow our economy. Then you talk about super intelligent agents capable of replacing all of humanity. It's not a tool for you to use.
Does that feel like a contradiction though? You and me and everyone here is you are using AI tools that are enriching these AI companies that are then building something that's going to kill us all.
So, interestingly, if you look at budget of a company like OpenAI, I think they generate something like 15 billion in membership fees, but the investments are in trillions. So, the money comes from investors, not from users paying $8 a month. And the investment indicates that they predict we're going to fully automate labor. Initially cognitive, then physical. What is the value of all human labor a year? 10 trillion? 30 trillion? So, when you think about it this way, the investments make sense. The 15 billion is irrelevant. You can stop using chat GPT, it's not going to change how much money they got.
So, it really comes down to incentives.
It's very hard for people to say no to a lot of money. If somebody came to me today and said, "I'll give you a billion dollars, go work for OpenAI for a month." I'll be working at OpenAI.
Right? Power corrupts, money corrupts. It's not unique to some bad people, it's general problem for any agent. Incentives matter.
And so, is there a way to somehow instead of thinking about these AI guardrails, change incentives? Or are incentives never going to change? There's always going to be financial incentives. We're living in a capitalist economy, and that's not going to change.
We can shift incentives. I think there is just as much money and just as much opportunity in narrow systems being deployed through economy. You don't need to have general superintelligence to collect most of that. So, today, if you were to find cure for a particular disease, green technology, better self-driving cars, all that can make you very rich and happy, sustainably. Whereas, if you go full general, you don't have anything to offer, and everything gets deleted.
And what do you think the chances are that it plays out in that direction, where they start building these niche AIs tools and stop going broad?
Right now, it seems very unlikely. There is some hope that with all the publicity, new documentaries, interviews, books coming out, some politicians are starting to notice, starting to propose some regulation. If we have enough time, which is not obvious, it could be just not enough time to accomplish it, we can grow it to where it's a majority and there is support from populace for enacting the type of legislation, electing this type of leadership.
Okay, so let's play this out, right? In the near future, post-labor economics is here. Human labor is valueless. Do you see that transition to, let's say, a universal basic income or a universal high income? Do you see that as a smooth transition or does that look more like a Great Depression?
I think we already have a lot of it in place. So, we have a safety net. Many people don't realize it, but in US, if you don't have income, you get lots of free stuff. You get Section 8 housing, you get food stamps, you get free health care. That is already built in. And if you keep taxing super profits, you have money to support all that.
If massive unemployment hits or maybe when, and you were one of the government leaders, how do you handle that, right? What happens if 99% of people become unemployed?
So, there you switch from unconditional basic income to unconditional basic meaning problem. What do you do with 8 billion people have a lot of free time on their hands? We don't have infrastructure, we don't have anything for that and that could lead to a lot of unrest.
Yeah, because most people correct me if I'm wrong, get meaning from what they do for a living. Well, maybe not most. There are some people that maybe just hate what they do for a living, would love to sit on their couch and get a check.
That's the two types of jobs. You either hate it and wish it was automated or you love it and don't want anyone touching it, but in both cases, it takes your time. If all of a sudden you have extra 40, 60, 80 hours of free time, what are you going to do with it?
And what do you think people do their time?
So, it depends on the person. Some people will be very creative and productive. Again, all of it assumes it's not killing us right away. This is kind of we we're setting this aside and we're let's just talk about money. So, if we're still around and everything's automated, yeah, you can play chess, do sports, yoga. Problem is a lot of times with young people if they have a little too much free time in their hands, they start doing things we shouldn't be doing. Idle hands.
Mhm. I think people look at this like, ah, that's that's This is a problem that's so far away. Like, I don't have to worry about this. Like, what's your response to those people?
So, it's funny. People now say, oh, this is long-term. This is not a problem. This is going to take like 6 years. Like, it doesn't matter if it's 2, 5, 6, or 10. On cosmic scale, on human history scale, on our election scales, none of it is long-term. Everything is now short-term. And if you look at prediction markets, if you look at what leaders of the labs are saying, we don't have 2 years. They're claiming we're going to get to AGI in 2027, 2028. Some people are saying we got to AGI.
Right. Like, uh, Jensen Huang of Nvidia just recently said he thinks Nvidia has reached AGI.
And as I said many times before, if we just took what we have today and showed it to a computer scientist in 1980s, they would be convinced we have AGI.