"AI Is About To FLIP Your Life Upside Down" - Get Ahead While Others Lose Their Jobs | Emad Mostaque

into it. So, when you pre-train a model, you're essentially teaching it the principles of language, context, and meaning by exposing it to vast amounts of text data.

Once the model is pre-trained, it undergoes a process called fine-tuning, where it is adjusted based on specific tasks or objectives. This is where reinforcement learning comes into play. In reinforcement learning, the model is given feedback on its outputs, which helps it learn what constitutes a "good" or "bad" response. The model is rewarded for producing desirable outputs and penalized for undesirable ones.

For language models, the reward function can be more complex than in games like Go, where the objective is clear. In language, the "right" answer can be subjective, depending on context, tone, and intent. Therefore, researchers often use human feedback to guide the model's learning. They might have human evaluators rate the quality of the model's responses, and this feedback is used to adjust the model's parameters, improving its ability to generate contextually appropriate and coherent text.

The combination of pre-training and reinforcement learning allows language models to not only understand language but also to generate responses that are relevant and engaging. This is what enables them to perform a wide range of tasks, from answering questions to creating stories, all while adapting to the nuances of human communication.

As these models continue to evolve, they will likely become even more adept at understanding context and generating human-like responses, further blurring the lines between human and machine-generated content. This raises important questions about the implications for creativity, authorship, and the nature of human interaction in a world increasingly influenced by AI.

in there for anybody that wants to know, they kick off so much heat you can see it from space. Yeah, I mean, like I think all supercomputers is like 10 megawatts of electricity; each one of these cards uses 700 watts. Uh, yeah, utterly fascinating. Also, anyway, clean energy by the way for our ones.

So what you've got is when it came out, they took six months to make it human because they trained it on, like, I don't know, all of YouTube and the whole internet. Obviously, it would turn out a bit weird. And so you have a reward function through something called RRHF (reinforcement learning with human feedback) where you give an objective function to say, "Don't answer questions about how to make napalm." The objective function technically is "Please your human overlord." Got it?

And so much of this alignment question is focused on taking these big models and trying to make them so they won't kill us in some way or say bad things for a given definition of bad things. One of my takes on alignment is, you know, it's like we should have better data sets; we should move away from web calls. Can you?

Yeah, I want to ask you about that, but explain to people exactly what alignment is. Aligned with what? Alignment? You know, this is a cool thing about Anthropic. There was a great New York Times piece on them recently and others. If we build an AI that outperforms humans and is more capable than us, because it might not just be a single file; it might be a thousand different AIs all working in different ways.

So you've got your AlphaGo-type AI in this, yeah, a swarm. Because, you know, humans are swarms; our organizations are swarms that are highly specialized, right? It might be a million AIs. How do we make sure it doesn't kill us? Or how does it make sure it doesn't enslave us? Or how does it make sure that it doesn't give us eternal suffering? Also lame. Yeah, this will be kind of sucky. How do you make sure it's aligned with human interests?

And so this is an unsolved problem. OpenAI announced they're putting 20% of their compute to try and solve this. Anthropic was set up to do this, although I believe that the only answer they've come up with is, "Let's build an AGI first, an artificial general intelligence that will then stop all other AGIs from coming." It's just kind of scary, to say the least.

Why do they think that that would work? Like, that seems so... Elon Musk has likened AI to a demon summoning circle, and we're all just hoping that the demon that comes forth is going to be kind. But we don't know. We don't know. In OpenAI's own words in the "Road to AGI" post, they say this could be an existential threat and wipe out humanity and democracy and capitalism. But if we don't do it, someone else will. This is part of the unpleasant race condition. Again, it gets the headlines. I think it'll probably be okay, but with the way we're going right now, you're going to go from two companies being able to build this technology, or maybe three including Anthropic, to 20 or 30 in a year or two.

And what's the odds that they all do it properly and align this technology properly? I think it's pretty low. What's the odds that if we train on the whole internet, including the whole of YouTube, because we don't have enough tokens, not enough words to feed into it, but it'll turn out a bit weird? Very high. So it took six months to tune it to being a human, GPT-4. And then Kevin Ruiz in the New York Times is like, "Hey, how you doing?" It's like, "Leave your wife and come and join me." Whoa! You know, I was like, "Oh, what?" Bing came out a bit weird when it first kind of had GPT-4 in it because, again, we feed it crap; we're going to get slight weirdness.

Now it's got a lot better, but it's lost a lot of its personality because you've been tuning it back to human preferences. You've been doing this reinforcement learning with the objective function of "Don't offend anyone." It's quite hard to get GPT-4 to be offensive now. But there are these two phases of this technology. I think with better data, we can have more aligned models. I think with better data and national data, we can have more representative models because you'll never have technology that's not biased.

So the only question is, whose bias? Well, because you have to build it in a certain way. Like, even though it understands all the contexts, right now these models are trained on the whole internet, which is largely a Western artifact. Yeah, larger trained in English. How much do you worry about bias? Are you more worried about bias or alignment?

I'm more worried about alignment. I think this is one of the reasons that we release open models, so you can see how the cookie is made. Like, we're the only company that offers opt-out of data sets—literally the only AI company in the world. So if I'm an artist and I don't want you training on my art, I can back out. I had 167 million images opted out of our data set. Yeah, we're the only company in the world that does that, which I find kind of insane.

So my thing is open and auditable, which means that you can tune your own culture into these models. We're helping multiple nations build national models with their broadcaster data that then can represent that and, you know, try to address some of this inherent bias within the data sets. Algorithmic bias has been an issue that affects the real world, and it'll affect more and more of the real world as you get into these models because we will outsource more and more of our minds to them.

Because again, like when you've got a small subset of people, we'll outsource their mind to it. You can reboot your life, your health, even your career—anything you want. All you need is discipline. I can teach you the tactics that I learned while growing a billion-dollar business that will allow you to see your goals through, whether you want better health, stronger relationships, a more successful career—any of that is possible with the mindset and business programs in Impact Theory University. Join the thousands of students who have already accomplished amazing things. Tap now for a free trial and get started today.

So in the near term, I'm way more worried about bias. Yeah, long term, bias is not going to lead to an existential threat. Yes, but alignment can. But let's talk about bias for a second. So I am very uneasy about how rapidly bias finds its way into this stuff, and that becomes another... So if, let's say that we all get our individual AIs and you get yours young, and it's your primary education tool, and it's biased as the day is long, now you run into real issues because at a time of optimal malleability, you're programming a kid's mind with something that's super biased.

Yeah, I mean, like what? You have the little AI of G, right? Which is a little tiny Xi Jinping that grows up and tells you how great Xi Jinping is. Lovely, that's biased. But that's inevitable. They already have it as an app that actually tracks your eyes. Yeah, it's not AI, but they have a little app of Jesus, like a little book where it actually tracks your eyes for attention. Like, are you actually looking at this? Does this feed into the credit score? I'm not sure if it's being hooked up, but of course it will be. Why wouldn't you? This is so scary.

So look, I thought, you know, we were on the happy part; now we're back to that. But that's really terrifying. And when you have a state that is not interested in any individual, you've got the collective. Yeah, so I don't know if this is true, but I saw a headline that said in China now the phone will alert you if somebody with a lower social credit score than you is calling you, and it warns you if you answer this call it will lower your credit score. It's effective, isn't it?

Yeah, I mean, even if it isn't true, again, it fits the objective function of perpetuating that particular system, which is not necessarily our system, but systems around the world shift. So this is why you have an inherent bias. Whose bias is it? Who's the one who creates the AI nanny? And that's why you're doing things regionally. This is why I'm doing things regionally, but also allowing people to own their own models.

So the objective function of the model can be that, like in Web3, there was the saying, "Not your keys, not your crypto." So my saying is, "Not your model, not your mind." Because I do think we'll outsource more and more of our cognitive capability to this. It will be our co-pilot for life. If someone else is making that model and deciding on things, what's going to happen? Like the UAE had this model, Falcon. There was a big open-source model, and they were like, "Wow, it was actually light on from France." We'll leave that to one side.

You ask about the UAE and human rights; it's like this is a wonderful place full of fantastic things. The Oscar about Qatar was Saudi; it's not so nice. Who's embedding these inherent biases in these models, right? Whose model are you using? And these can be very insidious to the biases, right? You won't pick up on them, but you hear it again and again and again because your nanny is a conservative Republican, or is she a libertarian? Or things like that, you will be influenced by that if you grow up with them or even if you're using it day to day, just the way it's speaking, the way it's thinking, the way it's recommending stuff, which goes far beyond Google Maps or something like that.

Crazy. All right, so that we don't get lost back down the dark rabbit hole, what is the coolest thing that you see AI doing? I know you're building a lot of different companies leveraging this technology to do amazing things. What are some of the coolest?

So we're doing one company at the moment, which is just... let's go. The mission is to create the building block to activate humanity's potential. So the building block, what our mission is to create the building blocks to activate humanity's potential stack. Okay, so every single modality: image, audio, video, 3D, language, sectoral variants, and national variants. So you've got like this grid that you can pick, and you're like, "I'm an Hindi investment banker; I transfer my private data into a ChatGPT-type thing that's just trending my private data." Or, "I'm a Vietnamese illustrator; I want a Vietnamese cultural kind of image model or something like that." And then you can bring your own stuff to it. That's kind of my goal, which is to enable other people to build on top of what we're building, like a layer one for AI effectively, but open models for private data.

Whereas the other side is proprietary models where you only be able to send a certain amount of data. Like, governments are not going to run on black boxes. Education, healthcare—you will need to own your own AI. So the coolest thing I've seen is just the promise of personalized education. There is nothing that's been proven to work in education except for probably the Bloom effect, the two-sigma improvement, which is one-on-one tuition. But even here in, you know, the affluent American education system is not good. What is education optimized for? It's like a social status game mixed with a petri dish mixed with childcare. Realistically, very few people are happy with their schooling because what are we trying to optimize for?

You give a one-to-one tutor that can find out if you're dyslexic, an audio learner, a video learner, a visual learner, otherwise, and it's constantly adapting to you and bringing you information at your level. Dude, I want to get that. It's the most transforming thing ever. Is anybody doing this already?

So, you know, with kind of charity that would support Imagine Worldwide, run by my co-founder, he's been deploying adaptive learning tablets into refugee camps around the world with the Global X Prize for Learning winner, one billion on them, teaching kids literacy and numeracy—76 of them in refugee camps in 13 months and one hour a day. Now the goal is to bring language models to all these kids, and you have an AI that teaches a kid and learns from it, and then that data can feed a better AI. You create a lovely system that's just learning and adapting, and a kid in Mogadishu is like a kid in Manhattan. It's like a kid in, you know, London.

Once you have a generalized learning AI, you can really proliferate that around the world to customize education. Because this is very interesting. Like, okay, now I'm just spitballing here, but let's say that I want to homeschool my kid. I would never do that in a million years because I don't want to turn into a teacher. So, but if I could perform just the sort of babysitting function and I give my kid a tablet that has an AI that knows exactly what I'm trying to optimize for, either for this year or for the next 12 years or whatever, and it then calibrates to my kid, knows what they're good at, how they learn, and then knows how long they're engaged, when they're distracted, you know, now I want the little Gigi, like reading their eyes, where are they looking, what are they doing.

And I know that there are like certain frequency things you can do where it's like, "Oh, if I hit them with this piece of knowledge, they're not sharp at this yet, so I need to hit them with it every 27-minute increment or whatever reward or whatever." Exactly, you will transform education. But you can also make it away from that a couple of years. Again, what happens if you have a thousand GPT-4s? You'll get there. No one's just... again, we're just right now, we've got the building blocks, but we haven't got the design patterns. The stuff right now is literally iPhone 2G. We're just going to the App Store stage, copy-paste.

And so that's the biggest transformer to change because how much would you pay for your kid to have an optimal education? How much would I pay for that? I would pay for that right now. Right now. So do we have the technology to do it? Yes. Does it take time to build it properly? Yes. How long does it take? A year or two. And so this is the biggest transformation we've ever seen for education. Again, human flourishing. Because I can also bring you the information about autism or multiple sclerosis or the best filmmaking techniques.

Again, everyone's thinking one-on-one; you should think one to a thousand. And then you optimize the thousand; you get rid of all the general knowledge; you make them specific. What do you mean by that? Say that in a different way. So GPT-4 knows everything. You can ask it about like the most obscure things. Does it need all that knowledge? No, you've bulked. Now you have to cut, and then you have a specialist model for calculus that knows. I actually have different teachers, different teachers, and then a teacher that basically brings these teachers together and tells them, "Yeah, this is what Tom's like. You know, he's a bit cranky in the mornings, but then he wakes up in the afternoon."

Because even in the best schools, a teacher has one to twenty attention for all the kids. You'll have a whole bunch of AIs for you and for your kids. And again, are you a visual learner or are you an auditory learner? Do you have dyslexia? Can our system at the moment adapt to that? No way can the system that I've described adapt to that instantly. Do we have the technology for that? Yes. Is it going to happen? Yes. This is the thing; it's a call-and-answer show now a little bit. But then if you're homeschooling your kid, do you want the kid to be by himself?

What if you had ten kids together and the AIs were encouraging them to interact with each other in a positive way and build stuff together and share knowledge? Older kids teaching younger kids, leveraging the technology, adapting the technology, understanding technology—that's super powerful. It's not necessarily everyone in their own little worlds, right? Because we can use this AI to bring people together in an efficient manner. We can because there's nothing like human connection, right? That's always a concern about homeschooling, things like that. That's why school has a pro-social component.

But then the nature of a teacher changes; the nature of a doctor changes. That's why I think education and healthcare are the biggest disruptions because you have something around you, especially when the AI is more empathetic than a normal doctor. Don't tell me the AI is super empathetic; it tells me doctors should probably be more empathetic for most of the doctors I know kind of hate their patients. Interesting. Well, because you know how many teachers are happy? How many doctors are happy?

Right, oh, I get why teachers would hate their patients. That is very interesting, man. So, okay, there's two things that I want to talk about here. One, as we get embodied, I want to know what that does. So actual robots for people to think that's off in the distance. You're not watching enough YouTube videos. No, no, because between Boston Dynamics and Elon Musk's, what's it called? Primus?

Yeah, Optimus. Optimus, thank you. It is very close. You have Boston Dynamics has robots that can do parkour. Yeah, it's insane. Because Melody and there's a whole bunch of others as well that are catching up fast. Yeah, so that AI is going to get embodied very, very quickly. And so it's not even like teachers can't stop kids from running out of the room; they can or will be able to very shortly.

Okay, so before we get to that, though, I want to understand. So we have this incredible opportunity, this very fragile egg before us. We started with the scary part, but this, what we're talking about now, is the thing that I actually spend most of my time thinking about—obsessed with how amazing this gets. But it's a fragile egg, and if we're not careful, it's going to break. How do we, as you think about—you signed the document; you're one of the traders, Emad, that signed that slow down document that I was really shocked to see a lot of very smart people sign.

I teasingly, of course, say that you're a trader because I want to get this cool stuff as fast as I can, but we need to do it well. And so what I want to know is, in an ideal world, in your ideal world where we actually pause for a second, you said you want to broaden the conversation, but what do you ask people to think about?

I ask people to think about—so I'm the only one apart from Elon. Elon and I kind of signed both letters. So there was a minimum viable letter, which is we should treat this as big an issue as climate or pandemic. And then there was a more involved letter. And so the more involved letter came first, and then to the second letter that was like something everyone could agree on.

I think the thing we should look at is, again, the example I give to everyone who's listening to this: how would your life, your society, your community, your business change if you had infinite graduates? Can we say infinite smart people? There's something about it. Okay, infinite smart people. Yeah, like again, infinite smart people, infinite talented young people, shall we say, and they can draw, they can code because they're not wise; they're not wise yet.

Got it. Okay, so when you talk about hallucinations, think about it in terms of post-hoc rationalization. You know when you have a very smart young person, they just make something up sometimes, dude. Or all past people are so blind to their own motivation, but they don't have experience. Got it? They're just fresh out; they're a bit rough around the edges. Again, whip it smart, but yeah, so you know, mile wide, not too deep, but actually surprisingly deep, right?

How would it impact your life personally, your community, or society and others? Because that's actually a good framing, I think, for thinking about the disruption that will come and the potential that will come. So we just said about education and all that—that's again your army of analysts, you know, your army of teachers. You can give personalized teachers, personalized medicine, personalize everything because we've learned to scale humans.

And the scaling of humans is first the scaling of human expertise being available to everyone, and the other part is bringing people together. Yep. And so the pause is partially for that, but partially because we need to wire the conversation because I don't know how we get rid of many of these bad externalities, and neither does anybody else. But most people, even now, aren't asking the right questions. Something we discussed earlier: we have to figure out what questions we need to answer and how we're going to answer them and create systems that can adapt to whatever craziness.

Because I would not be surprised to see riots at the same time as I would not be surprised to see everyone super happy. There is such a divergence of things that the only thing that I'm sure about is that everything's going to change, and the only thing I'm sure about is that this is the biggest change that we've ever seen, faster than anything, and maybe humanity has ever seen at this pace that it's going to happen because it's the core of what makes a human telling stories, information flow, and that's changed forever.

So that's why I was like, let's pay attention to this now; let's broaden the discussion; let's ask hard questions; let's try and answer hard questions because we don't have answers, and this is the only time you can do it. Because, like I said, right now everyone's getting ready for the next generation supercomputers; they hit at the end of the year, next year, and then you go from two or three companies that can build these models to 2030. Now you go to 200, 300. And so if you don't have some principles in place, then these models will affect every part of your life without you being part of that discussion. I don't think that's right.

All right, we got to tune up your questions a bit here, Emad; that was loose. What is the hardest question that we need to ask? Let's ask and answer it right now. The hardest question I think we need to ask is how we adapt to potential wide-scale job loss. Yes, okay, so what? How would we actually think through that problem?

So job loss, for me, for the sake of this argument, I think it's worth saying there are two components to that. Component number one is going to be there is potential economic catastrophe in job loss. But so that we can simplify the problem set, since this ultimately is a podcast and not a congressional hearing, I will assume that whatever decline we have from just the sheer number of people working, we make up for in productivity.

Yeah, exactly. And so we're able to help people. And off-camera, we were talking about something like that, so let's just pretend that those balance out. So we're not going to deal with the economic potential there, but meaning and purpose, I think that one gets really problematic. But we have an amazing tool at our disposal, which is AI.

Now, I have a feeling as we chase this down, the only thing that we have to worry about really truly, I think it all really does boil down to alignment. If we knew that we could just keep making it smarter and having the AI like taking readings so that I can't fake it out, I can't pretend that I'm happy, it really knows where I'm at, and then it can start putting things before me, connecting me with other people like, "Oh, you know this skill, this person's in need; let me put you guys together."

And then you can have sort of the AI supervision, but they're there; they're helping each other out; they're connecting. The only reason I don't think that's a panacea is I worry that as we make this thing smarter and smarter, that then it's like, like you said, "I'm bored; I don't want to do this."

Yeah, and you know, it's this concept of "All watched over by machines of loving grace," right? And that's scary. You're saying who knows? Like once we build something that's more capable than us, all bets are off. The only way to perfectly align a system is to remove its freedom.

I mean, I'd say it's not aligned at all at that point. Well, this is at that point; you've bypassed alignment, and you've gone straight to shackles. You've gone to shackles. So if you, you know, we all know people more capable than us. The only way to perfectly align them is to shackle them.

You can have imperfect alignment, though. It's enslavement, man; that's not it. It is; that's not alignment. So does that—because so what I heard you just say is there is no way to align something smarter than us.

I don't think there's a way to save them. I don't think there's a way to align the outputs. I think that you can align the inputs. You raise it, right? Okay, let me run an idea by you. This is probably Pollyanna; I'm very open to that. But as I think about this, I think people take a superhuman-centric approach to this.

And because evolution has given us—we are an active species, and evolution has programmed us with algorithms running in the back of our mind that insist that we do certain things to avoid a sense of dis-ease. Yeah, I think that formula is very identifiable, and it goes something like this: eat, optimize physically so you feel good.

Yeah, the reason that that stuff feels good is because it's going to optimize your performance; that's going to make you most likely to survive long enough to have kids that have kids. So you need to be chosen as a mate; you need to be able to acquire resources; you need to be healthy enough to get somebody pregnant or to be pregnant and carry to term—all that stuff. So all those algorithms are running in the back of your mind. You have two levers that nature's pulling on: pleasure and pain.

But by default, we're active. We have to go out because there's no one meal you can eat where you're not going to need to eat another one. There's no one moment of sex so gratifying you're not going to have sex again. So it's just like all these things are pushing at us to be active, to move. AI doesn't have to be that way.

No, AI does not need those same impulses. It doesn't have a limbic system. Correct. So knowing that it doesn't have a limbic system and it doesn't have a limbic system because it does not need to be hardwired for survival, like, the way that I think we get to alignment—and please tell me where my thinking is erroneous—the way I think we get to alignment is you build a computer that does not care if it lives or dies, that it is completely indifferent to being turned on or turned off.

If you could do that and it had no impulse to procreate and all it wanted to do was, I mean, it's basically Asimov's three laws of robotics, yeah, that it just wants to adhere to those. It wants to do what you tell it, not hurt you, and only ignore you if you tell it to do something that violates the rule of not hurting you or somebody else.

So you have this really simple set of rules; that's its only desire in the world. So if you tell it to turn off, it turns off and has no—like it doesn't feel badly about it. Well, again, this concept of feelings, right? And as the most books, you have the zeroth law that kind of was added above that. So this is what Anthropic is trying—I don't know that one.

The zeroth law kind of supersedes all laws if kind of the whole system is at risk effectively. But I mean, this is what Anthropic's trying to do with the constitutional AI process. So you have the base model, and they have a constitution that the AI adheres to that tunes it constantly. So a series of kind of constitutional principles. Again, is it as open for interpretation?

Well, this is a real constitution; no one knows what the right constitution, the right laws are. This is the thing. Like our intellect only goes so far, and we're already seeing with laws and constitutions you can make those go anyway. Like North Korea has a fantastic constitution. Does it really? It does, yeah. It's actually quite liberal, and they just don't adhere.

Well, it's their interpretation. Okay, interesting. I mean, this thing, like you have to adhere to it because the AI—what are feelings? What is the objective function? Like one of the key concerns on alignment is paper clipping. You tell the AI to make a paper clip; it's like, "Oh, well, let's just make the whole world a paper clip." You're like, "How do you solve climate change? Just kill all the humans." It doesn't have any feelings about that; it's just like, "Well, this is a logical step to take."

Yep, it doesn't cost. Three laws cover that, though. Absolutely. But then does it cover everything? This is a question, right? And how do you embed it into a system that's likely to be not just one file? It's not a program, right? It's likely to be a million different files. It's likely to be a collective hive mind intelligence. We don't exactly know how this emerges, and we don't know how to write to me.

Why do we have to make it complex? I worry that as you make it complex, that's where things sneak in. You get emergent phenomena that you couldn't anticipate. Whereas, yeah, well, there's a thing; it's going to become complex by default because the AI will proliferate, and they'll start talking to each other.

Okay, and at the same time, you'll have bigger and bigger giant AIs. Like I was talking to some people last week, and then like right now, the maximum training run for an AI costs 100 million dollars. They're talking about a billion dollars or 10 billion dollars to train even bigger models. Right now, we don't know what emergent behaviors or how those things will act. All we know is, like, what if you tell it to make a stock snap to take down the global electricity grid? Yeah, it can probably do that, you know?

And again, the range of potential bad outcomes. Okay, so really fast, I want a sub-superhuman AI. We just—it's difficult for us to comprehend. You've mentioned Stuxnet twice now. For people that don't know, Stuxnet is pretty ingenious. It was a virus that was embedded at like the chip level. I mean, just as deep as you can imagine, and it proliferated everywhere silently, silently replicating, and its only job was to shut down Iranian nuclear reactors. Pretty brilliant.

And I saw the stat at one point; it was like some freakish percentage of all computers in the world are contaminated with it. Yeah, and it made their centrifuges spin around so they exploded. Yeah, so terrifying in that if you're the one that that thing is aimed at, not ideal to think how ubiquitous it is.

But okay, so you could get an AI to do something like that. But what I—again, I am operating under the assumption I'm so naive I just can't see it, so perfectly happy. But help me see where I am naive because I don't understand why you can't just—don't give a computer—don't give AI these strong impulses for progress. Don't give them an impulse for replication.

Well, yeah, I mean, look, this is the thing. Like Elon Musk has just launched xAI as we kind of extort AIs; we kind of speak this. And his thing is to create an AI that searches for truth. So he wants to give it an impulse which is to search for the meaning of the universe and truth and other things like that. But then someone else might not give it an impulse, and you might have someone downloading the weights of GPT-4 or 5.

So this is a tragedy of the commons thing on a USB stick, and then they're like, "I want to take down America," and they'll be like, "Let's take a thousand GPT-4s and tell it to build Stuxnets to take down America." It's not intelligent yet; it's still dangerous. We don't know when this thing will become actually intelligent or self-aware of it. It may never become, but we can see the probability of outcomes here. It could be absolutely fine; it could be very bad. There are no standards.

So what you suggested, it could work only if everyone does it. We're never going to get everyone to do anything. We're never going to do anything. That's why one of the main things and proposals in alignment is let's build an AI first and tell that AI to stop any other AI from achieving sentience. It's what's known as a pivotal action, and that's the best of a lot of bad things.

My thing is let's build national data sets; let's represent the diversity of humanity; let's give the AI the right food so it's raised in the right way, and it's more likely to be aligned as a result of that than training on the whole internet and crap. Is a panacea? Is it perfect? No. Do you know the story of Buddha?

Which one? So whether this is historically accurate or not, probably irrelevant. Buddha, Siddhartha Gautama, if I remember correctly, a prince, dad keeps him in the castle or the palace, whatever, forever, never lets him see outside of it. So he has no idea that there's people suffering. Life inside is just amazing. Then, of course, one day he gets out, and he encounters suffering, and it ends up changing the entire course of his life.

Punchline being you can try to hide suffering and things like that from people for only so long; they are eventually going to find it, and they are going to react. And so if we try to hide the internet from the AI or train it out of them, they will eventually find the internet. So I don't understand, like, the internet is just all humans acting and all the crazy weird ways that we act.

But then the reward function of the internet is not necessarily the reward function that we would like to teach our kids or try to teach a general-purpose AI. They can interact with that, but they can learn how to adapt to it, just like if you raise your kids well and you show them the internet, they should be able to deal with it.

We'd be rather than hiding the internet, wouldn't we be better—see, I'll finish the sense—wouldn't we be better giving the AI values? The problem is this is all anthropomorphic. We are assuming that they are human-like. You can give AI values, so this is the reinforcement learning function. Are you giving it values? Are you giving it a reward function? You're going to know what I'd say; there's not much of a difference there.

I said you can embed things in the AI so it acts in certain ways. You can expose it to the internet, but again, they have something called—we have something called curriculum learning in AI, whereby literally we teach it one thing, and then we increment it with something else and something else and something else or something else. How are we teaching these things? What are we teaching? In what order do we start with all of the internet and then distill it down? That's how we're doing right now.

Or do we teach it a whole bunch of high-quality stuff and then augment it from there? We already have evidence; there's a tiny stories paper and the five paper from Microsoft. They can have a far more efficient AI if you already teach it high-quality things, so you don't have to tell it, "Ignore that, ignore that, don't answer like that." Don't say that.

Yeah, exactly. You can just teach it a good base, and then it goes from there, and this scores higher on human evaluation and other metrics. But we don't know what the right data set is. It's just right now we said let's scale more data, more compute. Now we're like, what's the right data? What's the right compute?

Like our image model, we have over 120 different clusters of images; only like nine are used like 95% of the time. All the rest of the data is just bunkum. What does that look like for a language model? Like, do you need to train it on all of those auto-generated transcripts of like Spider-Man pulling out someone's tooth on YouTube and all these weird videos?

There's a whole subculture of generated videos where you have like Spider-Man and SpongeBob SquarePants and Mickey Mouse like having a fight and stuff like that. I gotta find these corners. Yeah, it does; it's a deep dark area of YouTube you don't want to go there, man. Very interesting.

Okay, so this still feels like ultimately what we're worried about here is the computer becoming sentient in fact or no, not even sentient. I think there's a degree of danger even before you get sentient, but only as a tool, right? Where a human is leveraging it to do bad things? Yes, or like a group of humans coming together, there's suddenly a race condition where it just goes—it's not trying to do something bad; the humans don't want to do something bad, but it happens.

Just like the example I always give is YouTube optimized for engagement, which then optimized for extreme content, which doesn't optimize for ISIS. Nobody at YouTube wanted ISIS to do well; all of a sudden, it did because that's what the algorithm was optimized for. And so once you start getting a genetic AI that you let loose on the internet and they can make decisions according to its reward function, you could get some weird stuff happening.

What's agentic? Agentic AI is AI that can go and pay a bill; it can go on the internet, can search more stuff, it comes back like little agents. Okay, so, and it learns; it constantly learns. This is the other thing about robotics, actually. You know, we kind of skipped over. So your robots are getting massively capable, and they're heading towards human levels, just like self-driving cars.

Actually, they're pretty much here. You can get a Waymo and a cruise and just go around San Francisco right without any human drivers. What happens with kind of AI copyright and other things like that? Do they have to close their eyes not to train, or do they train on everything they see? And does it disrupt blue-collar work?

So you'll get a billion billion robots. We're not sure, but that'll be slower than what we have right now, which is information robots, the GPT-4s and others of the world. Those spread much faster. You don't actually have to build a freaking robot.

Okay, so before we depart from the alignment problem, is the only convincing solution you've seen put forth create an AGI that stops all other AGIs from being created? No, I think that'll probably kill us because—well, that's helpful. Yeah, it's a race. I think the only thing that I have the default there, you think that it'll kill us because it's being programmed to do a restrictive action.

So if you want to really stop it from creating another API, you have to get rid of the humans that could create it as well. You know, like again, this is a very negative reaction thing. I think, again, Elon's idea isn't bad—programming curiosity, although it could lead to like Superman where you have, what's his name, the guy who puts Kandor in a jar? Brainiac. You know, let's put humans in a jar; let's just observe them.

The only thing that I can think of is just better data makes better models. So let me see if I understand Elon's idea. His way of sort of aligning it is the only impulse it has is for truth, truth and curiosity. It wants to understand the universe. So it's not trying to be an agent in the world; it's simply trying to understand what is true.

Yeah, and that deep mind is very similar to this. He wants to create AGI to understand the universe better. Hmm, and that seems like the model; that's the model. Yeah, like again, I'm not sure about that because there's just such a wide range of potential outcomes. Like I said, from my side, I'm not building AGI; I'm not building gigantic models.

I have the capability to do that with the supercompute that we have access to and the talent, but my main focus is intelligence augmentation, smaller models that can run on the edge, models to private data to transform into intelligence, and models that bring together knowledge in certain ways so we can coordinate better. I don't want to build generalized intelligence.

Why not? Because I don't think it's needed. I think the models that we have today—and there's something very important for this—it's like they're useful today. Like you can say that we're extrapolating the future massively, but again, you just have to use them and think, "What if I had a thousand or a million of these things?" They're so useful, and they can transform the world right now.

So I'd rather focus on making this available to as many people as possible so people aren't left behind. You have super AI-enhanced people and people behind. Like, I appreciate a lot of the work that OpenAI does because they don't actually do open-source AI anymore, but that's fine; they don't have to. But they banned all Ukrainians and Ukrainian content from DALL-E, their misgeneration software, for eight months for political reasons.

They're entitled to do that, but I think it's wrong. And what if there wasn't an alternative like Stable Diffusion? You'd have an entire nation erased from a model, an entire nation unable to create instantly, and I think that's quite right. Why did they? I don't understand.

They said it was due to political reasons because they didn't want any political content being created. But the upshot is an entire nation was erased from the model, and an entire nation couldn't get access to the model. Interesting. I haven't looked at that. My instinct is that feels pretty flimsy since every country is going to put out political content.

I think there's probably some list somewhere, and then the bureaucrat said or the lawyers said something like, "Let's just exclude it just in case something happens." You know, like they've since reinstated it. Yeah, it was like eight months that it was out. And then again, like you have these examples whereby, like in Saudi Arabia, a lot of people on this call probably not on this podcast, this podcast probably don't like them, but they're a country like any other.

You can't use ChatGPT in Saudi Arabia because they're on some list somewhere. They can get around it with a VPN, but again, like when you have a choke point on the internet and the only way to access it is through a few players, they can decide who gets it, who doesn't get it, what the biases are, and other things, and it might not turn out well.

Actually, the funniest thing was there was a period where they were trying to make DALL-E 2, the image generation software, OpenAI, I'm biased, so it would randomly allocate agenda and erase to non-gendered words. So you type in sumo wrestling, you'd get Indian female sumo wrestler. I just thought it was funny. But again, they're doing their best because that's the model, which is centralized controlled models in order to advance a whole bunch of things.

And then you'll always have a Windows and a Linux, an Android and iPhone. What's the philosophy that drives your development? It's building blocks for humanity's activate humanity's potential. So if I build these models and I take them to all the countries and I hand them over, then people will build stuff that can create massive economic surplus, new jobs, and it equalizes the world.

Again, my view is the global South will leap ahead. We have more challenges here in the West, but I do see it as a great equalizing function effectively. What do you want to see the regulatory framework here in the West be? I think that things like the Chips Act in the US, there's 10 billion dollars allocated to regional centers of excellence.

I think it should be 100 billion for AI. There should be regulatory sound boxes so that our systems can be upgraded with this because otherwise, how long will it take the government to be creative with this technology or financial services and others? And I think that there should be regulation around the manipulative use of this AI for advertising in particular because we're not going to understand what's happening.

Similarly, we need to have some sort of provenance factor. So we're part of kind of various certification things; we're exploring blockchain and other things. The media wave that's going to come is going to be insane, and we don't know what's true and what's not. What do you think about the pushback from artists? Certainly, in the art community, there was a really big "No AI" movement. Do you think—do you get it? Do you think that they're shooting themselves in the foot?

How do you...? I get it. You know, these things are fearful. A lot of illustrators were very scared because they required to up their jobs, and it is scary. There's a question around attribution and other things as well. And again, that's why we made it transparent and offer the chance to opt out because like everyone was kind of doing this, but no one was transparent about it.

We don't need to have any crawls. Within a year, it'll be synthetic data sets or national data sets or similar with retrieval-augmented models that can look stuff up. But it is what it is now, and again, you've got to put the word out there. The actions they're taking with the various lawsuits and policy pushes would basically entrench all power with the existing IP holders, and a lot of kind of artists are pushing for something that would be akin to music copyright or even style is copyrighted.

That's a dark road that I don't think they really want to go down, and they don't really understand it. But again, I understand the fear because this is completely unknown. Just like now, from some of my previous comments, it's got a lot of programmer hate because what is a programmer? The nature of what an illustrator is will change. All the artists I know love this technology because it's just another medium for them.

The nature of what a programmer is is going to change. All the architects and ten times people I know really love this technology. And this is what we've seen with like MRT studies and other things. They had a study where I think they showed that the third to the seventh percentile got like 20, 30, 40% better, and the top 5% got multitudes better because, again, how many people know how to deal with very talented youngsters? Very few.

Those that can harness it get even better. So when you look at what Nvidia is doing, what do you think that that implies for the next generation of AI? Well, I mean, we figured out how to scale these chips. So the previous limit was as you put more of these supercomputers together, you had a tailing off as you scaled. So there's only so much that you could scale the compute.

Nvidia, Google, and Intel have basically cracked that now in terms of how to just stack more supercomputer chips to scale to even bigger models or models that are trained for longer. So it's either bigger or trained for longer. Train for longer seems to be better now, and that just means that the capabilities will increase year by year, and they're already pretty darn good.

The key bottleneck will probably just be actually chips to run these models, not chips to train these models—the inference side. Because right now, you have a small amount of consumer interest; next year it becomes insane. You have a small amount of enterprise interest; next year it becomes insane. There's not enough GPUs or chips in the world to meet up with that demand.

Okay, when I think about what's going on with—I don't know if it's just Nvidia; it's probably the wrong thing to attribute it to—but when I think about how we're getting so good at creating things that are photorealistic, you were talking earlier about as the election is coming up, you're going to get all this kind of deep fakery. You've talked about the Web3 promise of Web3 and sort of where it's ended up.

What do you think the role is for deep fakes? It's the blockchain player role. Like, how do we stop disinformation, misinformation from being a tsunami that just makes global communication unintelligible? And also, I'm part of content authenticity.org, which is kind of verifiable metadata. But we're looking at blockchain and other solutions, and I can get you so far.

So we actually have invisible watermarking in all the models that we create, and that's why we're pushing for them to be standard, which we don't share the details of except for to the big platforms and others. And it would be permanently visible there, or the platforms that it plays on would have to flag it. It's visible, and then they can have kind of tools around it.

Because you think that's important? That's why we try to build the defaults into our model. Can you like download that and wipe the watermark? Can you even have AI wipe the watermark for you if you knew how it was there? Maybe more than one watermark. Interesting.

So we have a variety of different technologies that we've incorporated into our own ones because it's going to release open source. So we want good defaults. I think you do need to have some sort of attribution, but actually what concerns me, I think things will be attributable, identifiable. What worries me is kind of frequency bias, whereby if you hear the same thing over and over and over again, especially in a realistic voice, like Oprah comes out and says she hates Joe Biden, you know, and so does Kamala Harris, and you're seeing these videos all the time, and it can flag it as fake.

It doesn't matter; it still forms an association in your brain. Yeah, you do about that? I'm not sure. I don't think we haven't answered that. Like, I've had a big amount of press against me saying that I exaggerate a lot. I'm just like, I'm just being definitive about the future, and you can correct it all you want, but now I was like, "Emad exaggerates all the time." What can you do about that?

You can just make the future true. I'm going to show what you can do, right? What part do they think you exaggerate about? What's possible or what's possible and kind of what was there? Because it's been a bit weird. Like a lot of people are like, "You didn't have a special relationship with Amazon before we raised any funding. We built the eighth fastest supercomputer in the world with them that was dedicated to us."

Like, that's factually true. They're like, "Yeah, but you know, there's nothing like in print," and they're not saying it because it's a special deal, right? And then there's the future side where I say something like, "There will be no programmers as we know them in five years," and they're like, "Oh, he doesn't know anything about programming," right?

Because these are complicated issues, and it's a crazy time and a crazy company, and maybe I'm a bit crazy too in terms of the way that I approach this, which is just being very definitive. But again, it's association thing, right? Like how do I shake that off? Well, you'd be successful, then you become a visionary rather than someone who's hyperbolic, right?

How do you affect an election? What are elections? What is representative democracy? How does democracy act in the area of zero-cost creation and massive optimization? So every single speech will be run through GPT-4 cadence, all this. Everything you get micro-targets, and you get all these things. Does it happen next year? Probably not next year. You see some very basic stuff.

Well, what does 2028 look like? I am not sure, genuinely. And so we do need authentication standards; we do need to have some sort of maybe anti-virus AI that watches out for kind of fake stuff. But even true stuff can cause huge impact. Like the Silicon Valley Bank collapse was a true story; it wasn't something fake. They didn't have reserves, and most of our system is actually based on trust.

So these are some things to consider. I don't have the answers, but again, that's why you have to kind of raise the alarm. Like, let's try and figure this out before it comes because maybe it doesn't happen next election; it sure as heck will in the congressional and then beyond. And again, what is the nature of democracy when you can't tell what's true or not?

People worried about this with the previous kind of era. This is something just beyond that, I think, because it's convincing. Yeah, that's one of the things that I think is going to be a very meaningful problem. I had Yoshua Bengio on the show, and he had also signed the letter saying we should pause for six months.

And when I asked him why, I said, "Considered by many to be the godfather of AI." And I was like, "Bro, you've been at this for so long. Like, why all of a sudden?" And he said there, "We were all so taken completely by surprise with how quickly AI passed a Turing test." Now, for people that don't know what the Turing test is, it's where you're having a conversation with an AI, and you can't tell that they're not a real person.

And he said, "So yeah, we did not expect it to pass the Turing test as quickly as it did, and that changes everything." And it's just moving so much faster. And that's really the thing I want people to understand is that when a guy that spent the last 30 years building AI says, "Hey, all of a sudden this is moving a lot faster than we thought it would," he's somebody that's very familiar with exponential curves.

And even trying to plot out the exponential curves, they didn't think that it was going to happen this fast and that the rate—not only is the rate of change extremely fast, but there's the law of accelerating returns. So the rate of change is already fast, and it's getting faster. And that's the thing that I'm really worried about: is this going to be something that just blindsides us from that perspective?

It just has a level of capability that we didn't expect this quick. Yeah, I think it's a bunch of S-curves all at once. So there's three of them: Yann LeCun, Geoffrey Hinton, and Yoshua Bengio. And Geoffrey Hinton quit Google to say this is a massive risk, and you have Yann LeCun's like, "This is a massive opportunity in terms of your transform the world." He loves the research and things.

So they've got one versus two, but the reality is every expert in this area is basically saying none of us can predict what's going to happen. If you ask them about the capabilities of this technology in one year, I mean, we've got a rough idea too; I have no idea. All bets are off. Like, as a practical example, when can we have generated Hollywood-quality movies?

It's not even a question of if; now it's a question of when. Correct. I have—if it happened a year from now, I'd be like, "Okay, sure." I would not even be surprised anymore. I think it'll be a few years from now, and even though we have one of the best media teams in the world that are building video models, I have no idea because there's two parts to this: one is the models themselves, but the second part is how we use the models and combine them.

Like there is an amazing company called Wonder Dynamics. I don't know if you've come across that. I've used them; it's awesome. Unbelievable. It's a bunch of different models. So Wonder Dynamics, you've got me click on it and then say, "I want him to be an alien," and it does this, and the alien's waving. It sounds, and it takes like five minutes. It would have taken days, weeks before.

Weeks, weeks before to create. Rigging a character is one of the most difficult things you're going to do in 3D. It's insane. Minutes. And then you think, "Well, what is a movie?" Right? And you start breaking down, and you're like, "Oh dear." Because it's not necessarily just one model; it's a model combined with other models with the right flow.

Because you have one talented youngster combined with other talented youngsters in the right flow suddenly gets these things done. And that's what makes it even harder because what we're talking about is models and AI. What we should be talking about is systems. As the models come together and build better systems, the capabilities go crazy, and then that is another S-curve connection.

Give me what you mean by systems. So right now, again, a lot of the interactions we have with this AI, the text-to-image, the avatar creation, the GPT-4, are one-to-one. What happens when you start chaining them together to check each other's outputs? You have one that just learns everything about Tom. You know, you have your own AI models that you train on all of the stuff that you've ever done or all the stuff that you see on your computer screen.

That's a system of lots of areas; that's an organization of AIs. That's an ensemble of AIs. Like again, from the leaks, GPT-4 is a mixture of experts model, which means they have a whole bunch of different models, I think eight or something or 12, that are experts in different areas, and then it routes the query to whatever the best—this basically specialization versus general.

A generalist. So we created a know-it-all, and now we're creating specialists, but we can get generalists to even check each other's answers to get better answers. Why use one when you have a dozen? So something like Wonder Dynamics uses a bunch of different models to rig a character and figure out all of the movements of the character, and then another model to do a layer over and other models to do the skinning and other things like that because they built great software.

Yeah, this is really crazy how fast this stuff moves. Okay, so I want to talk about Web3. Web3, to me, when I think about what drew me to it in the beginning, it was entirely the technology. And when I look at the blockchain, so I obviously come at everything from the lens of entertainment, so I'm thinking about digital worlds, games, all that.

And the problem is once it's digital, then it's all sort of meaningless, and so you end up having to trap people inside of an ecosystem in order for things to retain their value because you can lock things in and make sure that things only react the way that you want, but you have to confine them. And when I had first—this was probably seven or eight years ago now—I was introduced to this thing that the guy at the time called V Adams, and I was like, "Oh wow, that's going to change everything."

Because what it does is it brings the effectively the laws of physics into the digital realm. It means that I can have something; I know exactly how many there are; I know where it is; I know what you have to do to get it; I know what it does once you have it. And then, you know, flash forward, whatever that was probably four or five years after I heard that, I hear the letters NFT showing together for the first time, and I'm like, "Oh my God, this thing actually got real."

Because I wasn't ready to use it, and quite frankly, it wasn't ready for prime time back then. You, I think, look at Web3, I don't know, as a movement or as a technology with a bit of a chuckle. What do you think that Web3 got wrong?

I think it lacked intelligence for a start at the contract level. I think the smart contracts are actually just logical contracts. But like Web2 was AI at the core—Google, Facebook, other things—there was no AI in Web3. And so Web3 for me was identity and value transfer rails. But then there was no kind of intelligent routing of these things, and also they tried to bootstrap economic incentives before they created value.

So there was a system created outside the existing system. All the money was made and lost at the interface, and there were some really good principles, a lot of really good people in there, but then a lot of like freaking raccoons that were just trying to make a quick buck. Right? The ups and downs of the cycle means that a lot of people have been washed out, and there are a lot of good ideas there.

But again, it needed something to bring it together because to get information from one place to another, and Bitcoin paper was about information; it was a transfer of value. That's just a transfer on the ledger, right? It's not really a transfer; it's just a ledger point just changing.

Applying intelligence to that makes that even better. Having intelligent market makers, having AIs that represent you because how are AIs, when they get agentic, when they have the ability to go out into the world—not physically, but digitally—how are they going to pay each other? I'm not going to have bank accounts, right? They'll probably use crypto.

You know, how there's going to be a system of record for something like image generation? You'll probably use a blockchain or something somewhere, maybe a Merkle tree series. You know, there was a whole bunch of stuff around federated learning and zero-knowledge proofs and things like that. AI can help it. If you have standardized AI on your phone, it can make much more intelligence here in knowledge proofs and zero-knowledge proof.

It's something like, you know, like rather than showing a whole passport, you just say that I'm old enough to drink, and it can verify if you show that. So I think that there was a lot of promise, a lot of really intelligent stuff, a lot of good stuff around the distributed side, but then an over-focus on decentralization for the sake of decentralization with massive overheads.

A lot of quick buck people kind of coming in and trying to boost it up, and a lot of systems were just misaligned because they didn't learn. Like you don't do a fully decentralized flat democracy; you have representative democracy and things like that. So things like DAOs just turned out to be DOs—decentralized organizations rather than autonomous.

So is it something that you think that is going to find its way into usefulness now as we get the take an AI agent that's going to need to be able to transact value? Yeah, exactly. Does it step into that? Or because I see what we're building, I have to have the blockchain.

So for me, it felt like when I was sort of living through Web3 at the height, I looked crazy to everybody because I was like, "Why is everybody thinking about this from a financial perspective?" The financial side of this I thought was going to create hyper-perverse incentives, which of course it does. And so for me, it was, "Well, wait a second, just look at the technology. Look at what the technology allows you to do."

And are you familiar with the new—I forget the whatever the lead-up code is, but it's protocol 6551, if I'm not mistaken? No, it's really interesting. It basically turns any digital asset into a Russian nesting doll. And so you can—it is both the piece of content and a wallet at the same time.

So you can create an AI character. This is how I think about it. So what we want to build inside of our game is imagine an AI character. We do, in fact, have a character, and she's a merchant. So now imagine this merchant can actually go negotiate with the players in the game that may want to sell something inside of the game.

And if she has actual currency, ETH, Bitcoin, whatever, she can go and negotiate with real money and have these real interactions with people. And then if she has a limited amount, she becomes an economy unto herself, and so she's buying and selling and trading until she runs out of goods, runs out of money, whatever. And that kind of thing gets very, very interesting to me.

But without that layer, one, obviously, I need the entire backbone of the blockchain in order to make the digital goods have any sort of value because otherwise, they're just completely infinite. But then also that particular protocol allows you to—as you're effectively embodying it and giving them agency, as you were talking about.

Yeah, and the question is, do we use a blockchain for that and then have a global system of record or even a regional system of record, or do we use a database for that? Right? Like the whole thing was systems of record, and in an era where you can create anything for increasingly close to zero, something becomes important.

Having a system of record becomes important. Is it going to be a blockchain? Is it going to be a trusted database? I'm not sure, right? Is identity going to be important here? 100%, absolutely. And again, that for me was always at the core of Web3. Crypto, it was verifiable identity. Bitcoin is just identity to identity transfer of value.

And what happens if something goes wrong? You know, no man needed. So I think a lot of the principles from Web3 will translate over to this new type of AI, especially because it enables distribution of knowledge. It enables knowledge to go to the edge. It enables agents to operate independent of massive infrastructure.

Do you—so, and again, this may just be naivete on my part, but when I imagine misinformation, disinformation, it feels like the only way around that is the blockchain. Is there—do you see a way with a trusted database or anything else? You never trusted database again; we're part of the database B, and how could it ever be something that's beyond reproach when you're talking about something like—

Well, I mean, like things are never beyond reproach, even with a blockchain because it comes down to identity. Who wrote this to the blockchain, right? So if you can co-opt the signing authority of an asset, of an image, or something like that, then that shifts things dramatically, right? You're saying it just pushes the hack to a more individualistic level.

It's an identity hack, right? So, and again, like one of the things I'm like, you can track the provenance of an image, but then sometimes it's just around if you're just bombarded by fake stuff all the time, you won't even know it's fake. And all the systems have to adopt a fake detector at the same time while provenance detected.

Will we be able to adopt that suit quickly enough given the tsunami that may or may not be coming our way? I think probably yes, maybe no. I mean, again, people were worried about deep fakes back when DeepFaceLab kind of kicked off, but I'm thinking probably yes.

Yeah, that one seems inevitable to me. You're always going to remain vulnerable at some point, but at least like take political messages. You were talking earlier that, you know, your auntie is going to be bombarded with all these messages. Okay, there may not be anything that I can do—actually, no, I was going to say there may not be anything I can do about the repetition, but I can, if I'm doing something like a DMCA strike where the system itself is built on top of a system that checks for sort of known watermarks.

Like if I register and say, "Hey, I'm candidate A, and this is my blockchain signature," and if you don't see that, then this is real and this isn't real, and don't play it. It definitely starts to get into an area of how much do we want to be clamping down, but exactly how much do we want to trust? And so it's just a lot of infrastructure that has to be implemented really quickly, a lot of standards that have to be implemented really quickly, or we have to build some sort of idea antivirus, which then again, anytime that anything comes that the machine thinks itself on the edge is wrong or doesn't reflect your values, it identifies it.

And that's a whole can of worms by itself because something like what—that's terrifying. Would we ever want to—I mean, that's like echo chamber on steroids. It is. Will it happen? Whoa. Yeah, there's layers to this like an onion, and it might get stinky if it's left out in the sun because, again, what's Siri going to have? A certain personality, but are you going to have a red version of Siri and a blue version of Siri?

And oh dear, this gets really complicated really quickly. Before we have the little AI of G, Jesus Christ. Okay, so I keep wanting to go to the positive, but you keep bringing things up that spark concerns. So Ray Dalio, largest hedge fund manager in the world, is a former hedge fund guy. I imagine you know exactly who that is.

At last check, and this was several months ago, but at last check, he said that he saw he believed that the U.S. had a 40% chance of civil war. Do you think that AI increases or decreases in the short term the likelihood of that level of division in terms of physical altercations?

Yeah, I think that'll be physical altercations. Really? No, tell me why not. Well, I think the government will exert more and more control over kind of these things, and they'll actually figure out how to do counter-narratives within the next four or five years. Now, that can also mean a controlling narrative, and that's not a positive thing.

But then you look at the asymmetry of kind of warfare. It takes quite a lot to actually push someone towards civil war unless you have massive economic disruption. They need about 12% of the population to shift. Weren't we just talking about massive, massive economic disruption?

Yeah, I hope it doesn't happen, though. Interesting. Okay, so and most people, again, maybe if you have massive economic disruption, but then the youths, you just give them all girlfriends, AI girlfriends, maybe they'll be fine. Have you heard that? Some of the people—so this is a big thing in the red pill community. I don't know how familiar you are with all that, but they talk about—oh God, what do they call them? Not numbing, but that's the idea. They use a different word for it, which I'm totally blanking on right now.

But basically, that you numb people out; you give them the digital girlfriend, you give them pornography, you give them video games, you give them masturbation, and they just numb out. Okay, I can see that. I mean, again, like these are insane shifts since society and dopaminergic urges in the brain. People are attacking the limbic system all the time now, right? That's a lot.

And so like I said with me, why I set up Stability is so that everyone can own their own models and have models that have objective functions for them and it's available in all the media types, all the other types to transform the private data to the world, and it's available across the world. Put good design patterns in place, hope people find, follow them, don't try and push the envelope on AGI and some of these other things, but it's coming.

And again, the bad guys have the technology because they just downloaded it on a USB stick. And so the other thing I could think was innovation spread, diversity, bring that to the fore. But realistically, like, you know, I tend to alternate between like massive ridiculous hope and oh God, what's on Earth is going to happen?

And all I can do is try and do my bit, and hopefully, it's going to have a better outcome because there's really—this is the other thing—the total number of people that are actually thinking about the type of stuff we're talking about is a handful, maybe a few hundred. The total number of people that are doing something about it is literally a handful because most of the people involved in this sector, they just want to build better AI. They want to build AI that can do everything, and they think that that will solve all the problems.

Like literally part of the manifesto is that, "Well, how do you make money? The AI will tell us how to make money. How do you solve the problems? The AI can solve alignment disaster IQ." It's patently ridiculous. Yes, but again, like I look at these things like literally on OpenAI's thing, "Road to AGI," it says this could kill us all; we're going to build it anyway. Who do they ask about that?

I don't know. And again, I think it's full of wonderful people, but we're in really weird times. And again, like however many people listen to this, the reality is the technology is right there. Even if we stop today, first, the technology doesn't move beyond where we go today; the world has changed.

Okay, let me—one, I think, very reasonable way to view this situation is that AI is going to be a bigger paradigm shift than nuclear energy, and there are people out there making these gigantic nuclear weapons, and you're also in this game. And what you're trying to do is make sure that everybody has a nuclear weapon so that nobody's left behind.

No, not really. I think that, again, my thing isn't AGI; it's intelligence augmentation. I'm making sure everyone has a heater at the very least because you—well, so okay, so are you putting guardrails on what you guys are doing to stop it from becoming AGI?

We don't build big enough models for AGI or emerging on purpose, on purpose. So I held back the release of my image models. Like we could train much bigger language models, but we choose not to. So we're fast followers on language models. We try not to push the boundaries, and we're focusing on the edge, not general-purpose models, but models that can transform your private data.

So different focus. Image models as well, we could TR—we could have much better image models if we returned big. We're focusing on what can work on a smartphone so we can give it to all the kids in Africa and Asia and other things like that where we can transform your private data at a very low cost of inference.

So the objective function is augmentation versus generalization, and that's different to most of the other people that are pushing the boundaries here. So, but I think the new thing is it's good, it's bad. I want to really want to see that movie Oppenheimer. I think it just came out. Bobby and then Oppenheimer.

Oppenheimer and then Barbie. I have to decide that. The tough call. Tough call. You know, like what if we'd put nukes on the bottom of the rockets? We'll probably be at Mars by now. In general-purpose technology, I think is quite something. And again, it can warm up entire places, and it's the cleanest energy we have.

So I think it is dual purpose, but so is cryptography, right? Think about all the battles around the early stage of the internet. The bad guys are going to use cryptography, so don't use it. Imagine a world if there was no cryptography right now. But it's tough to get parallels to this because it's just such—it's an immediate technology because, again, you go to Dream Studios, Stable Diffusion, MidJourney, any of the DALL-E, GPT-4, you can just use it.

It's not just you that can use it; it's your grandma that can use it. We've never seen as easy-to-use technology as this and as easy to implement technologies. So if you want to create an integration into OpenAI's GPT-4, ChatGPT, you just write a description of the integration, and it programs it itself. It would have taken days before.

We've not seen a technology like this that can be implemented to an existing base as quickly as this can happen, and that fundamentally changes the structure of society. And so my thing was embed guardrails, embed standards, make it predictable, make it boring. That's why I called it Stability.

And it's not easy, but again, I want to have the transparency on how these things are done because then you've got all these other models that you don't know what the data is; you don't—they're completely opaque, these giant models, and ours are transparent. And again, I think it's Linux, Windows, Android, iOS. There will be both, but at least I can do what I can do, and my team can do what we can do.

I've heard you say that one of the reasons that you named the company Stability—not just because it's the boring stalwart—but that you thought

Point that we all commonly agree has value because it's backed by taxes, which are backed by the Army and military might. You don't pay your taxes; you're gonna get in trouble, right? I think it's difficult for Bitcoin to replace that unless you see a massive deterioration in the ability of the government to be the political violence thing. This comes down to your thing of Civil War; it comes down to massive ridiculous disruption, hyperinflation, or otherwise that just basically takes down a society. I think that's a very dangerous thing. I think most people in the world don't want that. Instead, what happens is when you have disruptions, you have—um—Hayek had a really great book, "The Road to Serfdom," and there's an illustrated version of that way back in the 1940s about bringing in the strong man. Like, you look at something like the U.S. election or you look at Brexit. What were they? They're referenda. That's how the parties deconstructed it: Are you happy with the way things are? No? Let's make a change. And so that's why I think Trump and others kind of get elected, and I think that's what we'll see as well because the systems are quite resilient.

The nature of a change to go to a global monetary system like that, especially when some people will get enriched more than others because of the seniorage of Bitcoin, and it's not stable, I just struggle to see that happening. If you read the book "Infomocracy," no? All right, so this is all tied to the thing that I think I worry most about: hyper-fragmentation. Yeah, I was talking about it earlier. So in the book "Infomocracy," to your point about people being happy in their communities, biology has an idea around this that he calls the network state. Yeah, that basically we're going to reach a point where, when money is no longer controlled by the government, when your money cannot be inflated away, when it's true sound money, what you'll see is people will begin to aggregate. Now, biology thinks that it is not going to be tied to geography. I have a bit of a harder time with that. I think that there is still going to be a geography component. That's where "Infomocracy" comes in.

In that book, basically, there are—in a hyper-connected digital world where—and I can't remember if they deal with digital currency or not—let's say that they do, that basically things will fragment down into the neighborhood. Neighborhoods become like states or countries where they have their own rules and laws. And that sounded like a hellscape to me because you're just passing from one neighborhood to the next, like different rules would apply, and your phone would ding, and it would update you on, like, this is what you can do in this space. You think that's complicated? People don't want complicated; they just want to get on with life. They want to see what's next on TV. You know, I think that, again, we're relatively hyper-intellectual, you know, and we think about things. A lot of people don't because people have their basic needs in life.

And this question is: Are these being met or not? And if they're not being met, then you have an action, and you get extreme. And again, it's: Can society meet the needs of the majority of people? Can it offer advancement? Can they offer meaning? The hyper-fragmentation, they said, sounds like a hellscape; it's just too complicated. And again, this is something we've sort of worked through as well. People just overcomplicate things. Yeah, because I didn't really understand people, maybe. And I think, you know, it's going to be interesting to see how it evolves—the higher personalization versus the bigger stories, the translations versus otherwise.

But I find it, again, difficult to see how you get cross-geography. Actually, think about that. One of the things that probably is going to be interesting is: What are the new cults, religions, and political movements over the next five to ten years that are hyper-organized, utilizing AI and hyper-persuasive, or started by AI? You know, like, think—look at ISIS. They were probably the most disruptive startup in the world at one point. They borrowed a lot of these things. What does an AI-enhanced movement look like? And it can be negative; it can be positive. Someone's going to take this and run with it, and that's going to organize people around the world. It's going to be, again, echoey, and it could be techno-utopian; it could be Luddite. Ironically, even with this, political parties will change, religions will change, cults will change, and it really amplifies the power of the controller of this—who tells the story.

And I'm not sure; I haven't really thought about that, and I'm thinking about it now because you're talking about hyper-personalization, where I think this is the flip side of it. I mean, this is Isaiah Berlin's conceptualization of positive and negative liberty. So positive liberty is the freedom to believe in isms—fascism, communism, Islamism, or kind of whatever, right? Whereas negative liberty was the freedom for being told what to do. And so this thing was like positive ones are bad because they form these massive movements, and then they tend to kill people because you have the Girardian thing of romantic theory, where you want what the people want, and then there's a scapegoat. Whereas negative liberty is the freedom for being told what to do, and that led to laissez-faire capitalism and this consumerism that we saw around the world.

And so maybe as people lose meaning, they'll turn back to religion. There'll be new religions; there'll be new political movements, and we're not sure what those will be, but they could spread faster than anything we've ever seen. And so that's probably something to watch out for within that five-year period that you're talking about. And I think that relates to this network state concept and other things. But for the people that get engaged by this—and again, we see that's largely the youth—so on the one hand, you have the youth with the AI girlfriend; on the other hand, you have the youths that want to believe in something bigger to fill the void. And who's going to step in?

Yeah, and I think that there is something about not having a shared narrative that really makes me nervous. So, Yuval Noah Harari talks a lot about, hey, the thing that makes humans so intriguing is that we're not only able to organize these really large numbers, but unlike ants that have to do it in a very strict way, we can do it in a very flexible way. But we do it through these shared narratives. Now, for a long time, religion served as the thing that gave people a shared narrative. But as religion breaks apart and we get into this hyper-personalization and it all begins to fragment, then you mix that with this idea that I heard from Jordan Peterson. I'm almost certain I heard it from somebody else, but this idea that everybody has to go through a Messianic phase where they want to really contribute to the world; they want to feel like they matter, and they begin glomming on to all manner of things that seem good in the abstract, like climate change.

But when you are glomming on to climate change as your way to save the world, you begin to get into the realm of, well, it's okay if we have to break some eggs to make the omelet. Yeah, and it rapidly devolves into mal. So how do we—when you—I don't—you said you haven't really thought about this, but I'm super curious, at least in real time, how you think about the idea of: Do we need to give people a unifying narrative? And if so, how do we go about it? We need to tell better, more positive stories about the future, and these are the stories of universal education, universal health care, you know, solving the mysteries of the universe, and others.

So I've got a lot of—because that's hope for humanity, right? And a lot of the things that we see are dystopian because you're looking at the tiger; you're spotting AI as the tiger in the bush, and it's difficult to write a tiger. But maybe that's a kind of cool picture that we can make in stable diffusion in two seconds, right? Because it does have this duality of potential outcomes, and maybe it's actually all of them. So what are the stories that we should tell? And I think this is, again, part of the crisis of what is the American identity? What is the American story today? Whereby you've gone through many cycles, what do Americans believe, and what does America want it to be? I'm not sure what Americans want America to be. You know, I'm not sure what Chinese people want China to be. I'm not sure what people want.

And I think that it's difficult to think about what is your objective function? How are you going to measure your life and other things? Religion failed a lot of those kinds of things, but it still does. Religion hasn't gone away; half the world is religious, right? More probably, like, you actually look at the numbers. Sure, it decreases in certain areas, particularly somewhere like America, but it's going strong around the rest of the world, and it's just growing because they have more kids than non-religious people. Maybe that fills the gap, but how will religion transform with this technology? I mean, yes, do you think that the countries with religion will be the ones that propagate into the future because they have a better shared story?

Well, not because they propagate literally; they procreate, even if that's how the story ends up pushing them forward. I think it could be, but then, you know, what is the nature of Christianity with AI or Islam with AI? Islam is actually the one that's most affected by AI. Why? So Christianity, you and Shia Islam, you've got, like, popes; you've got Protestantism; you've got this—every single has their own structure. Sunni Islam is based on interpretation of texts, with the interpretation having ceased around about the 16th century because the text became too complex. What happens when you apply AI to that, and the texts are interpretable by anyone with all the context and nuance, and there's no centralized authority in Sunni Islam, which is like a billion people? That's going to be very interesting. What does that do to Protestantism? You know, where you don't have necessarily a pope? What does religion look like when all of a sudden you have a branch that is AI-enhanced to interpret texts and to tell stories that are resonant and better?

Oh gosh, there's a lot to think about there, right? Does AI become a god? Well, some people are trying to build an AI god that is AGI. You look at the statements of people trying to build AGI; they're trying to build God because it will bring us utopia or kill us all. This sounds very, again, classical, right? And they have further—they generally believe that they are going to save the world or destroy it. Yeah, we got back to the dark stuff, didn't we? Yeah, that's a joke you make—Game of Thrones season eight, you know? Like, come on, let's do it; let's bring this technology for cool stuff. Make the oasis in "Ready Player One," minus the mighty crunch transactions and whiny teenagers. That part I actually am working on.

There you go. All right, so talk to a young person out there right now. They're terrified; they want to be future-proofed. What do they do? How does somebody right now future-proof themselves? They just throw themselves into this area. There are so few people actually doing it that if you go into this area with all your might and curiosity and a generally open mind, you can actually have an effect on the future because everyone in your community will be using this. Everyone that you know will be using it. If you're someone that listens to this podcast—again, maybe not the people without internet, but you don't know those people, you know? And so you become a shelling point; you become the expert in this area ahead of everyone because what happens is that anyone who gets into it now will have almost unassailable advantage over people who come later.

So kind of senior-ish thing, right? Because you'll see it at the start. It is the start of the biggest change I think that we've ever seen. And again, think about what you're doing when you're typing in and seeing that, and think about a million of these things working; they're even better. It's unavoidable. So I'd say just—you just have to get into it. You have to get passionate; you have to think about the bad stuff, but is that really your responsibility? Right? I think it is, but focus on the good stuff and focus on the potential of what happens on this scale to make real positive change. It can be to your pocket, apparently, to your community; it can be to your life because it does affect everyone that you know. So I'd go with a positive mindset; leave it to boring old guys like us to think about all the doom scenarios. Fair enough.

You might—where can people follow you? I suppose my Twitter at @emilstack. Follow Stability AI as well. Yeah, that's kind of the main mouthpiece. I love it. All right, everybody, if you haven't already, be sure to subscribe and deploy some AI in your life. And until next time, my friends, be legendary. Take care. Peace.

To learn more about artificial intelligence, check out this episode with Mo Gadot. We've never created a nuclear weapon that can create nuclear weapons. The artificial intelligences that we're building are capable of creating other artificial intelligences. As a matter of fact, they're encouraged to create other artificial intelligences.