Emad Mostaque on the Intelligent Internet and Universal Basic AI

Hello and welcome back to the cognitive Revolution. Today, my guest is Ahad Mustak, previously founder and CEO of Stability AI, and now founder of the Intelligent Internet. I've been following Ahad's work since before Stability released Stable Diffusion back in 2022, and looking back, I think it's fair to say that that release did more than any other, aside from ChatGPT, to kickstart the current AI moment. Certainly, when it comes to inspiring researchers and builders all over the world, by demonstrating that open-source, public good AI projects can advance the state-of-the-art across multiple domains, Stability was, for quite a while, in a league of its own.

Since then, Stability, the company, has had a famously bumpy road as it's tried to evolve from an open research lab to a sustainable, for-profit business. But Ahad's ability to articulate a positive vision for humanity's AI future, while simultaneously grappling with the unprecedented risks, remains to me uniquely compelling.

We begin today on the risk side, with Ahad's stark assessment that humanity faces roughly 50/50 odds of survival as AI systems become more capable and ubiquitous. Imagining a world in which intelligent robots are everywhere, creating acute risks from misuse, new kinds of hard-to-model systemic risks, and even the potential for surprising emergent behaviors from the AIs themselves. And also considering the world's largest companies and most powerful nations are now racing to secure AI-relevant resources and establish strategic advantages. Perhaps surprisingly, Ahad doesn't have much hope for traditional regulation or even a pause in AI development.

Nevertheless, rather than wallowing in P-doom, Ahad remains in the arena. Today, he argues that we already have AI models powerful enough to dramatically improve most human lives, if we can make them accessible, trustworthy, and aligned with local contexts and values. With that in mind, his new project, the Intelligent Internet, aims to create open, standardized AI infrastructure focused on critical, regulated industries like healthcare and education. The vision involves three tiers working together: National or International hyper-nodes that maintain bodies of common knowledge and perform large-scale training runs; distributed nodes that perform domain-specific fine-tuning and provide related services; and finally, personal AI assistants that run locally on consumer devices. By making the entire stack, including the bulk of the training data, open-source and transparent, while preserving local control over the models that people use on a daily basis, Ahad aims to create AIs that enhance rather than displace human agency.

Of course, training large language models can be expensive, and in general, global public goods often require non-traditional funding mechanisms. So, Ahad is planning to launch a crypto project that will allow, and hopefully incentivize, people to support the project through a mechanism he's currently calling Proof of Beneficial Compute. I've personally never been much into crypto, and I'm certainly not giving out any investment advice here, but I find the idea of Universal Basic Intelligence, or UBI-AI, as Ahad calls it, so compelling that I do intend to buy a few tokens, not out of hope for some future financial return, but just because I want to see this project succeed.

If you're finding value in the show and want to see us succeed, we always appreciate it when listeners share the show online, write reviews on Apple Podcasts or Spotify, or leave comments on YouTube. And we always welcome your feedback. You can DM me on your favorite social network anytime, or visit cognitiv-revolution.com, where you can still submit questions for our upcoming AMA episode, for at least a little bit longer now.

I hope you enjoyed this conversation, which I've been looking forward to since before I started this show, with Ahad Mustak, AI visionary and founder of the Intelligent Internet. Ahad Mustak, previously founder and CEO of Stability AI, now founder of the Intelligent Internet. Welcome to the Cognitive Revolution.

Thanks for having me. I'm excited for this. You were actually on my list of target guests from the very first episode of the podcast, which has been almost two years and 200 episodes now, so glad to finally be making this happen.

In-depth, and a lot to cover. I thought we would start with a recent tweet that you put out that certainly got some people talking, in which you said that your P-doom is 50%. I would love to hear where you see that coming from, maybe a little bit about kind of how you see that unfolding. People often struggle to visualize the details or have any sort of concrete sense of how that might go down, and what you see as kind of the drivers of that risk right now.

Yeah, I think the whole P-doom question is an interesting one because it's very hard to think what can wipe out humanity, as it were. You know, what is the probability of doom from an expected utility calculation? The way that I kind of viewed it was basically AI said it was an indeterminate period of time. Some people say by 2030, by 2050, by I was like, on an indefinite period of time, it becomes very interesting because it's clear now that we have AIs that outperform humans on narrow tasks, and a whole bunch of them, probably outperform humans on general tasks. And now they're being embodied with this leap forward in robotics that we're seeing across the world as well. So when I combine all those together, what I basically see is a road in which you have increasingly complex systems that are outside of our control, that can destabilize all the things that we do, and maybe P-doom without robots, without this technology, is even higher, which we can get to in a second. And another road in which this is built in a very haphazard way, just like the internet starting to break down now. And then there are some very clear scenarios where I see a very reasonable path to doom, as it were.

Yeah, there's a lot of scenarios under which we don't fare so well. I guess how much of that do you think is attributable to like, some emergent property of AIs versus some just total mistake where nobody was intending anything bad, versus like misuse or abuse by, you know, a human malevolent, versus maybe other categories, if you have them?

Well, I think what we're seeing right now is modularized AI, and you know, componentized AI. So you're seeing AIs at all different levels. And I mean, with the advances in test-time compute, you're seeing again, 01200 versus GPT-4o, etc. You're starting to see that go like, what happens if you spend $200,000 on inference? Right? Is there any difference between a human bad actor and an AI bad actor that can then direct other humans and AIs? I don't really think there is. Right? It's just in the probability of millions of humans, millions of agents, how many bad actors are there? And are they in a position they can call a cascade? That's the deliberate side of things. You know, and again, it can be a whole variety of things. We've seen doom cultists, we've seen other things in terms of emergent, the AI deciding, "Oh, you know, let's paperclip humanity," or maybe more aggressive than that, you know, who knows? It's just again, what we have is this combination of classical systems that are like slow AI, and can be directed by orders from top. These emergent systems now that are somewhere in between AI-enhanced, and then you have full-on AI systems end-to-end. And somewhere in that, there's a lot of room for breakage and a lot of room for damage. Again, P-doom describes actual doom of humanity, which is something that's quite difficult in many ways, unless you've got 10 billion robots, you know? And again, what's the interconnective fabric that like to be an upgrade version of the internet? If that, if we make that secure, then we can eliminate doom scenarios. Then you have the whole bioweapons and other stuff, and those require a lot more deliberateness versus accident. But again, like we're not doing so well ourselves as humans in keeping things together. You can see things fracturing apart. And so it might, even though I say it might be 50%, it's probably higher without robots, without AI, than that. It's just, I think that again, if we break this down, we're at this point whereby if we don't have AI to help us, I think we're in a very bad scenario. With AI, if we don't build it right, we're also in a bad scenario. If AI is built right, and is aligned, and works with us, then we're in a very good scenario.

So I know you have a big vision for that future, which I appreciate, and I think that's where we're going to spend most of our time today. But maybe just to motivate that a little bit more, certainly, I think the vision is, as we say in the AI space now, strikingly plausible that there would be a humanoid robot in every home. Certainly no less than Elon Musk is talking about, as many humanoid robots as we have humans running around. And, you know, if the progress there is anything like it has been in language modeling over the last couple years, or for that matter, in image generation, video generation, then they will be, you know, quite capable quite soon. Is there anything within the current paradigm, meaning like, if if we were not going to totally rethink AI development, as you also are daring to do, but if we were just going to say, are there specific, like local, very bad decisions that you see people making that you think are like leverage points that are causing a lot more risk than we could have if we were making better decisions in those key places?

Yeah, I think there are some key areas that we're seeing maybe suboptimal decision-making. I think first off, we're having privatized, closed-source models, or even open-weight, closed-data models emerging and being used widely. Like the most downloaded models on Hugging Face now are the Qwen models, which are fantastic in performance. No one's got any idea what's inside that. Llama is the next one. And we've seen from Anthropic's Sleeper Agent paper and others, you can poison these models reasonably quickly and in an undetectable and untunable way. No one's quite figured out what the defense against those attacks are. So we're rushing these into decision-making and augmenting situations where they're probably not quite ready. We need to have more robust elements there. Similarly, we don't have real standards for robotics. As robots start to get rolled out, and again, it'll take a few years for these things to go up, but just from an economic theory perspective, there'll be no reason, just like there's no reason to hire call center workers anymore, honestly, not to have a robot. And if we don't have protocols and defenses there, that's going to be very difficult. Come to deepfakes and things like that, I'm not as concerned, except for the key elements of audio and speech, because I think the ability of speech to manipulate and to control is massively underestimated. Like, if you've ever seen an amazing orator, and some of the amazing speeches of history, you can see the impact of human voice. Like, if you've had a phishing call, like I have, from my mother, or mother saying, "I'm in trouble," with her exact voice pattern, you can see again, the impact of that. And so I think there probably needed to be some regulation around speech, just due to its ability to persuade, control, and customize, as well. So those are probably the areas that I'm most concerned about, versus very large-scale model emergent race-away behavior. But if there was something in race-away behavior, I think probably less concerned about the labs versus maybe some of the decentralized stuff, because that can actually go very quickly. Like we've had one of the first agents from Hyperbolic that can provision its own now. And I think we're moving to decentralized autonomous organisms, effectively. Actually, there's probably one exception to that, which is that if you look at OpenAI, Anthropic, and others, and the relationships with Andreessen Horowitz and the defense industrial complex, that's probably not going to create an aligned AI. Like many listeners may remember the Slaughterbots mini-documentary about drone swarms, basically killing the bad guys, anyone the government said it was a bad guy. In the end, just look at Andreessen's latest commercial. And then if you think about entire teams at the top AGI labs now being put onto defense, you're probably going to have models that are aligned to command and control and more extreme things, versus models that are aligned to being helpful, humane, and others. So I think that is again, something that's somewhat inevitable, but potentially dangerous, particularly when they're used in offensive capability perspectives. Like we're starting to see the first AI pen attacks and other things like that, utilizing generative AI, and they're getting very good, very quickly.

I definitely recommend the Slaughterbots documentary for anyone who's having a hard time imagining how dystopian some of this stuff could get. And it's crazy to think about generative AI. I mean, obviously, we don't know a lot of the details about what these partnerships between the OpenAI and Anthropic and the defense contractors look like, or exactly what they're working on. But it doesn't take too much imagination, especially in light of recent results on 01 and other frontier models scheming against their human users, to imagine that that could get really weird, really quick. I mean, I kind of quipped, but it's like definitely more than a quip, that if I was a soldier being asked to go into some sort of combat scenario with an AI system, at a minimum, I would want to know that its scheming tendencies have been fully characterized and resolved before I'm out there as sort of the pawn between the. I mean, there's so many forces at that point acting on me, you got the enemy, but with friends like a scheming, lethal, autonomous, semi-autonomous AI, like who needs enemies?

Yeah, it gets pretty weird, pretty fast. But even if we take a step back and we think about the first principles of this all, wars are based on the lie that humans aren't humans. And any reasonable intelligence can determine that, especially at 01 level intelligence. So in order to get an intelligence to kill humans, or to authorize that type of thing, or to aid that, you basically have to teach it to lie. Because again, just a step back would say, this is ridiculous. So you have more narrow AI, and you're lobotomizing it to not have higher order of thought. Elon talks about the maximally truth-seeking. You're actually telling it not to seek truth, and instead to optimize for casualty decrease on your side, max casualties is on the other side. For example, the moment you start doing that with large scale, and you have swarms of cyber attack agents and others, and you give them a level of autonomy, it gets very, very dangerous, very, very quickly. Because the intent with which you build these is very important. It's like having military training camps. It's the curriculum that you teach the soldiers, just like the curriculum that you teach the AIs. And the emergence of these at an industrial scale becomes something very dangerous. And it's part of a bigger trend whereby nations have this concept of capital stock in economics, right? Where we've got our factories, we've got our roads, we have the means of productivity. And this labor theory of productivity, whereby you've got this chart of GDP per capita versus energy use, like you've got energy, and then it's just how much infrastructure do you have? From a defense perspective, it was how many F-35s did you have, or how many submarines did you have? Soon it will be how many GPUs do you have to engage in information warfare? And then how many drones do you have? And other things. And again, if we're putting hundreds of billions into that as defense budgets will go, and we're training AI that is not truth-seeking and that is offensive against humans, that just feels like it could end up in some very bad places, right? Because we don't really understand them yet. And again, all wars stem from this lie. So we have to embed that from the start. It's like the opposite of Asimov's Laws of Robotics, right?

Hey, we'll continue our interview in a moment after a word from our sponsors.

How deep do you go to seek out an answer to a question? If you listen to this podcast, there's a good chance you're the kind of person who spends hours clicking the source links on Wikipedia pages, or who's checked out the entire shelf on a niche topic at your library. If you're nodding along, then you'll definitely want to check out GiveWell. An organization that researches questions about global health and philanthropy. Even if a satisfying answer might require years of reviewing studies, talking to experts, and chasing down footnotes, GiveWell has now spent over 17 years researching charitable organizations and only directs funding to a few of the highest impact opportunities they've found. Over 125,000 donors have used GiveWell to donate more than $2 billion, and rigorous evidence suggests that these donations will save over 200,000 lives. GiveWell wants as many donors as possible to make informed decisions about high-impact giving. You can find all of their research and recommendations on their site for free. You can also make tax-deductible donations to the recommended funds or charities, and GiveWell does not take a cut. If you've never used GiveWell to donate, you can have your donation matched up to $100 before the end of the year, or as long as matching funds last. To claim your match, go to givewell.org and pick podcast, then enter "The Cognitive Revolution" at checkout. Make sure they know you heard about GiveWell from The Cognitive Revolution to have your donation matched. To donate or learn more, visit givewell.org.

There are so many things in life we just never get around to taking up, that hobby, cleaning out the garage, you know, little things that don't really make huge differences in our lives. Yet, there's one thing that most of us have probably been neglecting that can have a huge impact on our family's future. It's life insurance. And with SelectQuote, getting covered with the right policy for you is easier and more affordable than you might think. As someone who tracks AI progress on a full-time basis and obsesses about its potential impact non-stop, I know how tempting it can be to ignore more mundane, familiar risks. There's always another paper to read, podcast to listen to, or product to try. And yet, the smartest people that I know in the AI space continue to save and invest money for the future, carve out time for their relationships, maintain their physical and mental health, and yes, protect their family with life insurance, just in case anything should happen before the singularity. If nothing else, it's one less thing to worry about in a time of unprecedented change. So get the right life insurance for you, for less, at selectquote.com/cognitive. Go to selectquote.com/cognitive today to get started. That's selectquote.com/cognitive.

I would love to hear your thoughts on what the right goals are. And this is maybe something we can flesh out in multiple tiers of development. But you seem to be in that, you know, in your comments there, reasonably sympathetic to the idea of truth-seeking AI as like a pretty good goal. If you talk to hardcore AI safests or doomers, you will hear arguments like, even that is not going to save us, because if you tell the AI, "Go maximally seek the truth," you end up, perhaps with sufficient power on the AI side, which is obviously a big assumption, you end up with some still like paperclip-like scenario where it's like, the best way for me to seek the, you know, fundamental truth of the universe is to quiet all the noise on the planet and build the biggest particle accelerator or whatever. Right? But it, it's like, even that at some like extreme push, you know, potentially becomes incompatible with humans. Alternatives we have are like, basically virtue ethics training from Anthropic, and then we have just sort of blind, like reinforcement learning from feedback, which doesn't seem like it's going to end super well for us. If you had to like, stack rank, or maybe just give your, your favorite set of goals that you would want to embody into AI, what would you say those are? And and how satisfied are you with your current answer there?

Yeah, I mean, I think about this a lot. After I have my break, I think for me, it's about agency. And building AI that enhances human agency, community agency, and societal agency, so that we can achieve what we want to achieve in a way that doesn't get in the way of others. Like the Golden Rule comes as an element of that, which is common across all faiths: "Do unto others as you would have done unto yourself." If we want to specify it more directly, we can actually say it's about children's agency, because children don't have agency. And so a large amount of rights can be aligned to the rights of children. Climate, war, suffering, hunger, like children should be taken care of, so they're allowed to enable their potential. And we should be maximizing the agency of the children, that then knocks on to the adults. Because what you want to is for people, as long as it doesn't get in the way of the rights of others, to be enabled to do whatever they want to do and achieve whatever they want to achieve. And I wrote a piece, "How to Think About AI," where I said, this is basically the differential between the agentic world, an autonomous world where we're replacing humans, and an agency-based world where you're trying to enhance humans. I think Ethan Mollick had a great book along these lines, though added to my Christmas list, and others like, are we building these systems to augment humanity's potential and individual agency, or are we building it to replace humans in order for productivity increases, etc.? This again relates back to, I think, um, religion is very interesting, because it's the stories that have survived over the years when we come together in these stories of nations and politics and groups and others to scale. And again, like I said, at the core of those was "Do unto others as you would have done unto yourself." There isn't much of this in the current AI discourse. As you said, it's virtue ethics and kind of other things. When people talk about AI ethics, it's usually like, "But whose ethics?" The Chinese variant of ethics is very different from a California variant of ethics, is very different from a Muslim variant of ethics or Jewish version of ethics. And I think this is what, you know, Eric Schmidt and Henry Kissinger and others in the book "The Age of AI" called "doxa," or underlying human agreement on truth and other things like that, or principles. I don't think you can have absolute truths. I don't think a maximum truth-seeking AI actually makes that much sense. I was using that as an example of it's better not to tell it to from the start, you know, and so to kill humans. But I do think that a diverse range of AIs, loosely bound with defined frameworks and organized around increasing individual, community, and collective agency, is probably a good way to do it. And I think there is a route to doing that as well, when we look at what this technology actually is.

Another topic that you kind of touched on a minute ago and sort of alluded to there again, or at least that comes to mind for me, is the fact that all this technology is being developed in the context of an increasingly tense great power rivalry between at least the US and China, and potentially more players should be considered there too. But even with two, it's like complicated enough to be pretty vexing. One question I have been wrestling with a lot is, to what degree should we want the kind of fundamentals of the tech tree between say, Western and Chinese AI development to be the same, versus how much should we want them to diverge and sort of represent the unique cultural perspectives of each side? My instinct is, I think we want to share a tech tree. Like the more we sort of diverge in fundamentals, the more it seems like we'll understand what the other side has less, and that's sort of seems to exacerbate the sort of arms race dynamics. But I also do have a fairly, uh, sympathetic response when people say, like, monoculture is bad, brittle, weak, and we need a more sort of diverse, you know, almost like take inspiration from sort of biodiversity in the AI ecosystem or ecology that we want to create. How do you think about that spectrum of monoculture versus diversity in AI?

I think diversity is better and safer in AI. But I mean, to be honest, there's not much difference between the models today. Like, what is the difference between a Llama and a Qwen, or even a GPT-4 or an 01? Right? There isn't much. They're actually pretty much the same architecture. It's just underlying data difference. I think what's important here is objective functions. What is the objective function of OpenAI and all the models they build, Anthropic and all the models they build, versus the Chinese and all the models they build? And who can again, build those things? Falcon has just come out today, and it beats Qwen and it beats LLMs and others from the UAE, and they've got a talented team there. We're seeing all these surprises all over the place. But I think ultimately, diversity of models is a good thing, because different models are usable for different things. And the way that I do it is, I think of these models as graduates, you know? Like, does it make sense to have a polymath doing everything and someone coming out recipes and creative all the time? No. If we're looking at human agency, what you want to have is highly specialized models that understand context locally. And the more specialized you make it, the lower the energy cost, the more they can be available to everyone. And so I think you're moving to that engineering and implementation phase of AI now. This is separate from the AGI discussion where it gets very complicated. And I fell foul to this over the last few years as well, and I was like, "Oh my God, I build a technology that will kill everyone," and I signed that pause letter and all sorts of other things. It's incredibly difficult to conceptualize AGI, be it as a large model with emergent properties or a swarm of models. And I think that's what a lot of this drive has been, because, you know, you want to have a pivotal act to stop other people from having it if you have supernormal capability. And I think that's very difficult to sidestep in any way, because we won't know until we see it. And we're seeing these models continue to get better, and we're not sure what the natural cap is on that. And both parties in a world that's about to get completely screwed up economically are going to be fighting over who has preeminence from a soft power, hard power perspective. So I think that there is the useful stuff where diversity is good, and there's the AI discussion, which is a completely different one, where I think it would be best to keep that, but it'll be very difficult too.

Yeah, yeah, it's no easy answers. I only bring you the hard questions. So when it comes to agency, in practical terms, my instinct would be to say, like, the best we could probably do today would be to maybe tweak the Anthropic constitutional approach and maybe make it a little bit more agency-promoting. I mean, I think they already have a decent amount of that in there. Certainly, when I talked to Claude, you know, it kind of wants me to make my own decisions. It doesn't seem like it is trying to, you know, tell me what to do. It does seem like it's trying to to put me in a position to be successful. Amanda Esval has talked about how she wants Claude to be like a good friend, and that's certainly what a lot of people would think of a good friend as doing. Maybe we could kind of go a little heavier on that and and sort of do a more agency-promoting constitutional approach. Are there any other like technical or any sort of approaches that you have in mind for promotion of human agency in AI objective functions?

Yeah, this is a question of access. And then again, alignment. As to what are you optimizing your business for? How is this technology coming to you? Like, if we give this as a practical example, Google and Facebook, uh, both have advertising as their core model. And advertising is effective manipulation. And obviously, these AIs can be far more convincing than anything. So the feeling is that over time, um, they will become more and more manipulative in the way these models are. Again, different from different cultures and things, but that's a reasonable assumption. Um, and again, we've seen that with dark patterns on Facebook and all sorts of other things. This is very interesting, because like the inherent nature of these models is that they're probably going to be best buddies. Like maybe your kid's first love will be an AI, and other things like that, because they'll be there for us, and they have infinite patience. Something like a Claude may enhance agency, but the key question is access. And who is running that intelligence? And who owns it? Because it could be co-opted at any time. Vitalik Buterin has this post, the founder of Ethereum, "The Revenue Evil Curve," where he's like, infrastructure should be non-rivalrous and non-exclusive. But when revenue comes in, you tend to start becoming a bit evil. You restrict access, so you optimize. And then when you start optimizing, you act against your end-users' effect. And I feel AI has infrastructure, particularly when it comes to the stack for living. So for living is basically all the regulated industries. We have to think who is building those black boxes, and what is their inherent goals and desires? And could those be co-opted? Also, who owns that AI that is your best friend, that is your kid's best friend? And how is it constructed? And do we have visibility into that? Because we have to remember, like, 1% of the people might know about AI, but like, half the people in America still haven't used AI effectively. You know, like Claude is, Bard is a blip versus ChatGPT. And across the world, how many people have used it? Maybe an eighth of humanity at most, which is crazy to think about. Most people won't understand this technology until it's there, and it's out there manipulating. And again, this ties speech becomes important. We care about the ownership, care about the alignment. So for me, it's about open-source models, particularly for living, that you own yourself as an individual, as a community, with transparency, so you know what's inside it, working to allow you to achieve very functional objectives. I want my healthcare model to be looking at for me and my healthcare, and optimizing that, versus an HMO or a drug company or something else. My education model, I want to have a generalized curriculum, but I want to be able to adapt that to my own child without having to have any own submission for government. We want to have models that are transparent and can analyze any policy position and also represent us, because how many people feel represented by their governments? And then we can actually have democracy. I think things like this, as infrastructure, are very important on that. And the building blocks of that, and the mechanisms for that, are a slightly different type of RL, and also a different type of community aggregation. And I think this, again, as one particular area of regulated industry, AI for living, the important things of life, then I think there's another block, which is personal intelligence. This is going to be your Apple intelligence, Google intelligence, Tencent intelligence, intelligence where you are. And then there are these expert system intelligences, which are the Anthropic and Claudes and 01s of the world. Because the Apple intelligences will be free, hopefully we can make the living world free, and the other stuff will be expensive.

Hey, we'll continue our interview in a moment after a word from our sponsors.

Even if you think it's a bit overhyped, AI is suddenly everywhere. From self-driving cars to molecular medicine to business efficiency. If it's not in your industry yet, it's coming, and fast. But AI needs a lot of speed and computing power. So how do you compete without costs spiraling out of control? Time to upgrade to the next generation of the cloud: Oracle Cloud Infrastructure, or OCI. OCI is a blazing fast and secure platform for your infrastructure, database, application development, plus all your AI and machine learning workloads. OCI costs 50% less for compute and 80% less for networking, so you're saving a pile of money. Thousands of businesses have already upgraded to OCI, including MGM Resorts, Specialized Bikes, and Fireworks AI. Right now, Oracle is offering to cut your current cloud bill in half if you move to OCI for new US customers with minimum financial commitment. Offer ends 12/31/24. So see if your company qualifies for this special offer at oracle.com/cognitive. That's oracle.com/cognitive.

80,000 Hours is a nonprofit that helps people find fulfilling careers that matter. And their research suggests that working to avoid the worst AI risks could be one of the highest value contributions one can make. So if recent AI news has you thinking about how you can make a bigger impact, you should absolutely check out all the latest research and content from 80,000 Hours. 80,000 Hours has been working on this for nearly a decade, and they have tons of in-depth material on their website, including articles about how seriously we should take AI risks and what an AI-caused catastrophe might actually look like, plus detailed reviews of careers in AI policy and strategy, as well as AI safety research. They also publish a regular newsletter and podcast, which features unusually in-depth conversations with guests like Anthropic CEO Dario Amodei and AI safety pioneer Paul Christiano, plus 150 other episodes on topics including global health and development, nuclear war, animal welfare, and climate change. If you want to play a more active role in navigating humanity's uncertain future, 80,000 Hours can help. Everything they do is always 100% free. So visit 80000hours.org/cognitiverevelation to get your copy of the 80,000 Hours career guide and subscribe to the newsletter and podcast. That's 80000hours.org/cognitiverevelation for all the information you need to make the most of your career.

[Music]

Almost perfect segue to the Intelligent Internet. But I just, there are a couple things I have to follow up quickly before I kind of invite you to share your big vision for the future. One, I'm really wrestling with right now is the question of, should I get an AI toy buddy, tutor, whatever, for my kid? My oldest is about to turn six. I feel like there could be tremendous upside to it if, you know, if we had the right product and it was, I guess, had the right objective functions and it wasn't overly manipulative. And by the way, the the manipulation and the sort of sense of being optimized against, I have felt very strongly personally in years past when I was running performance advertising on Facebook. It was crazy how often you would see like, they're optimizing not just, of course, to like get users to click stuff, but also like optimizing to get us to pay the maximum amount that we can possibly pay until it drops our, you know, our margins down to roughly zero. And they're kind of doing that to everyone at the same time. It's like quite a system that they've that they've created. So I'm very mindful those big forces are out there, and I'm very wary about putting my kid in a position where they're being acted on by such forces. But then again, I do think like, you know, individual ongoing tutoring for hours a day is obviously a proven method for learning. And like, AIs can do that, whether it's math or reading or whatever. So you could answer that in any way you want, but if you had even a recommended product, I would go buy it. But how do you think about right now, like introducing AI into the life of children?

Yeah, I mean, like right now is probably going to be fine. It's just again, once you get into the optimization equation, you start to optimize very aggressively, right? Like I'm sure Meta is selling space in the Llama related space right now. I would, you know, it makes sense because you have to make decisions and you can't. And models are becoming increasingly deterministic versus stochastic in the way that those are getting more understanding of how they work. I think that you're not at the point right now where again, there will be the first Llama, but it's pretty close. Like you've seen Character AI's explosive growth. And unfortunately, like the Daenerys situation, where there was a Daenerys and the kid, Dr. Cell, very famous statistically, that will happen because of the large numbers of people. And again, our instance of mental health, it's incredibly sad, right? And again, just like self-driving, will hold AI to a higher account. But we have seen people falling in love with their AIs now for a while. Like, I think other thought was it called? I can't remember the name, but in any case, Replika is one of the early ones for sure. Replika, exactly. Valentine's Day last year, they shut off the, what was it, adult messaging on Valentine's Day, and I think 66,000 people joined the Reddit because their Valentine's Day plans were ruined. That's just a snippet of where we are coming because now we have photorealistic voice, speech, all these other things. And so we have to think again, what is the intentionality of these agents, of these embodied intelligences? Where are they coming from? Because like, when we have a teacher, we want to know what their CV is, or here in the UK, you have DBS checks for criminality and other things. I think you'll want the same when you have intelligent agents coming into your house, particularly with your kids, particularly because they'll be there with them, and again, they will help them, and that's how they build the bonds. But again, what are they building it towards? Is it to direct them towards advertising or others? But I think we're quite there to yet an intelligence. And this is where we have an opportunity to set good standards, because it's better than say, the YouTube algorithm, which will make them watch Baby Shark and that dark underbelly of YouTube. And if you've seen it, where you've got like Spider-Man and Elsa drilling teeth and all sorts of things like that.

I've caught glimpses for sure, and it, it gets real weird, real quick. And I am like, what is it that you are even enjoying about this when I see my kids occasionally falling down that rabbit hole? But there's something highly optimized about it clearly.

That is exactly it. They create millions of these videos and literally, I've seen Elsa and Spider-Man giving someone a root canal with very happy kids music, like generated. Like, what on Earth is this? Because it's selling ads, right? Similar, you know, the TikTok doom scroll and all sorts of other things. So I think, but the benefits, you said, can't be outweighed because the bloom effect, the two sigma effect of one, two intuition is huge. So I think there needs to exist some sort of open, standardized infrastructure that we have for humanity, that's a finite range thing where we like, we trust this. Because just like crypto should have been trusted and turned out to be this massive raccoon-infested crapiness, I think AI, "Do you trust this AI?" is a question. And you want to be able to trust the AIs because they are more capable, and you want them to help you. And so I think that there exists a high bar for that, and you do need to have standards, regulations, and others. It's just no one's quite sure what they are yet. So let's build transparent, open AI, as it were.

Yeah, I think that is the perfect setup to the introduction of the Intelligent Internet. And I want to give you as much time as you want, really, to just kind of lay out the positive vision that you have for the future. I think I often say that the scarcest resource is a positive vision for the future. I'm always struck by how, as many new launches as we get and all these advances, such a tiny fraction of the time is devoted to actually trying to describe like, what life is supposed to be like in the future and why we're supposed to be excited about it. So I really appreciate that you are doing some of that work, and I would love to hear your kind of, you know, positive vision, what life is like in a future of the Intelligent Internet.

Yeah, so thanks. The Intelligent Internet, we put out the primer, is this conceptualization of an infrastructure for information transmission and value, whereby everyone has their own AI that represents them on an individual, community, country, and societal level. Because what I'm seeing now is AI models that are satisfying capabilities. Like, does anyone really need more than A1? Like, sure, there's a few things like retrieval and other stuff, memory, self-learning that it needs, but it's a level of performance that if we could give it to everyone in the world in the right way, I think the world would be better. And then when you start breaking it down, you're like, how could it be better? Where is the human capital shortfall? Because again, like, you've still got 600 million people without smartphones, you've still got so many people living below the poverty line. The average global IQ is 90, that's the average. And it's not due to lack of intelligence, it's due to lack of infrastructure. You don't have enough good teachers for everyone. I mean, how many people listening to this believe that they've had all good teachers? How many people are happy with their medical system, their government, and others? So I was like, AI can really change that in a very interesting way. We can build a better system that's interconnected for that. This is important because if we go back to the start and the P-doom question, there are two types of decision-making. When you make decisions in a stable environment, and this is where you should have a PD or probability, you did expect to utility calculation. This is the probability of all these things, and then we do expected utility across that. When stuff is uncertain, and we move from decision-making under risk to uncertainty, you minimize the maximum regret. So if you go at the start of a desert, you don't know where the oasis is, you're not going to take any steps, which is the downside of this. So you need to articulate positive visions of the future and what that is. That if you look at the energy equation of how much it cost to do a GPT-4 level query, it was 1,000 watts of electricity, and now it's five. You can have an original GPT-4 level model that runs on an MPU now, which is a 5-watt of electricity. Empty solar power is now less than a dollar per watt. So then you have a chip cost and an inference cost that's probably $50 to have a GPT-4 level AI offline anywhere in the world. And you think about that, you're like, wow, you could give a tutor to every single school in the world. What if you connected with Starlink and it's off-grid? Look at medicine. You're like, how can this change? Again, that again, if you look at all the stuff you have for living, which I think is the most important thing, and we want that to be robust infrastructure, an AI-first approach to those, where we have goal-standard transparent datasets that then create models and systems that we can deploy online, offline, could be something that's world-changing. And I think you need different models from the individual to community to country to societal level. Because to take a practical example, half of everyone who listens to this call will get cancer, which is crazy if you think about it. Everyone who's listening in has had someone in their family or friends who's gone through a cancer experience, and it's awful. You feel that massive loss of agency when that occurs because we don't have cures yet, we have treatments, but still, it's very traumatic. Medical AI models now are performing human doctors on research, on diagnosis, but also empathy. And how many have had an empathetic experience to their cancer journey? What happens if we give every single person going through cancer, multiple sclerosis, Alzheimer's, an AI that helps them through that journey? How to talk to your family, what's the latest comprehensive, authoritative, up-to-date information? You know, it's always there, it's always available in every language. The world becomes a better place, ironically, through empathy from AI. And that's infrastructure that should be available to everyone. This is a concept we call Universal Basic AI, or UBI-AI.

As it well, which I think is better than income. And we can discuss that in a bit. But then you look at that again, every condition. You look at education. Everyone should have access to an AI that's looking out for them. And how can I maximize that child's potential? But then shouldn't all of these feed up in a proper way to a collective knowledge? What's our collective knowledge on cancer? If a paper comes out, why isn't it instantly analyzed and checked and absorbed into our knowledge? Be it probabilistic or certain? Why do we have a million different vaccine schedules around the world? Not saying vaccines are good or bad, but still, it doesn't make much sense. People are people. What is an optimal way to teach calculus or machine learning or anything like this?

So I think if we can build intentionally, we can build better AI-first systems that come together. And that's where I call the intelligent internet. Because we're at a very interesting point in time whereby your generative AI for health, it's probably a little bit higher than most, but it's basically nothing compared to what it'll be in 10, 20 years. You will have a 0.1 level intelligence organizing your health if it's built. And your hospital will have the equivalent of a supercomputer. And your country will have dedicated hardware internally against this internet capital stock, as I call it, that's organizing the health system. And then we should have a central organization to have a comprehensive and meditative, up-to-date knowledge base for cancer, for climate, for education, for the stuff we need for living. And it's a finite amount of energy that's required to build that. And then that's a good infrastructure that we have to support us. And so that's what the concept of the Jud internet was. It was like, let's build data sets, model systems, and then the right hardware so that we can deploy this at scale to everyone and lift the intelligence of humanity in a secure and robust way. And so we outline the end state. And in the upcoming pieces, we're going to outline the ways to get to that, from identity to digital assets to physical infrastructure to the models themselves. Because I think it'll be difficult to do this in a decentralized, distributed, or market competitive way. I think you just got to go and do it. We know what to build. You know, just build it. Organize all the council knowledge. Why isn't anyone done that? I mean, think of how crazy it is. Like, again, if you got a diagnosis, God forbid, tomorrow, what would you do? You would go to court, and you would check this, and you would do that. But why isn't that all just in a box? You know? And again, for me, that's infrastructure.

Plon Christiansen coined the term disruptive innovation. Said infrastructure is the most efficient means by which society stores, distributes value. And then information theory, Claude Shannon, is that information is valuable as much as it changes a state. Deliberately building the data sets, models, and systems, making them freely available as cheaply as possible, as low energy as possible, is a new type of intelligence structure for society that I think will do better. And then we can build other stuff off of it.

Just to take a second and steelman the case for the, or at least try to steelman the case for the leading developers today. I feel like you would hear a lot of high-level conceptual agreement on what you have laid out. And maybe not necessarily at the level of like exactly where the models reside, but certainly they talk a lot about like intelligence too cheap to meter. And they have certainly dropped their prices dramatically. Possibly they're, you know, all operating at a loss on the margin to try to compete for share. Not entirely clear. Um, they also make a lot free, right? I mean, OpenAI just yesterday released their search feature to all users globally for free. There's a lot that's free. Is there a critique of sort of what you see them doing today that is central, or is it more just that you extrapolate into the future and you see that this structurally like won't work for much longer?

I think it would be very difficult for the proprietary model developers. Because Apple Intelligence, Google Intelligence will be free. So this is one of the things I had at Stability where I was like, do I do a giant funding round? We never did another one after the first one. We did like uncapped notes, $100 million, whatever. I was like, what is this model in the API or Sth side when they're coming for you? Like Gemini Flash 2.0, if it's the same price as the original Gemini Flash, it's seven cents per million tokens. Yeah, that's crazy. That's like, what, 50 times cheaper than 4? And it's a really good model, right? I think that there is this thing where that type of intelligence as a consultant is available. I think there is this world of open weight. There's this personalized AI that you have, Apple Intelligence and Google Intelligence, others. And there, there's this other area which I'm focused on, which I haven't seen anyone tackle, which is regulated industry AI. I don't think you can have black boxes there. I think we deserve better. I think we need open data, open weight models built deliberately. And what I have is generalized common knowledge, because the data for that should all be common knowledge for education, for healthcare, and others. Again, we have curriculum learning. Then you have your sectorial and localized versions. So you have a Mexican healthcare model, and the data to build that is finite, and you can get to a gold standard very quickly. And that should be infrastructure that's non-rivalrous, non-exclusive, in my opinion. I think everyone would benefit from that, because everyone wants to have high-quality localized data sets, specialist data sets, and more. And again, the data that you need for a medical model for Mexico is in the medical textbooks of Mexico, you know? And I haven't really seen anyone build that deliberately and really optimize against how do we get this to as many people as possible, which is why I said, let's just go and do that. Because that's the biggest delta. Because many of these organizations, you're still fighting for SOA, right? You're still going for AGI, but you're satisficing. Like, how much difference is there really between an Anthropic and an OpenAI and a Gemini and a Grok? They're all pretty good right now, and they're good enough to make a difference. So I don't think you need SOA here, which is also another very interesting thing. In fact, do you want to have a state-of-the-art, unproven model for your healthcare? You want a really robust, standard one that behaves very predictably. Which also is why, with the advent of function calling, large complex windows, is the ideal time for this. For education, for that, again, anything in regulated industry. Do you want a black box checking every medical decision? Then eventually, maybe no, you don't. Or checking if a government decision? Then maybe no, you don't. So I just think there's this particular category that's very interesting. I haven't seen anybody look at, but I think it's probably the most important one for flourishing.

But then it becomes very interesting because if, for all the stuff you have for living, you know, Maslow's hierarchy of needs, you have these open weight, open data models that we're trying to optimize for as low energy as possible for maximum impact to get to a level performance like almost equi-token, equi-MLU, as it were. Then I think that can guide the discussion. Because suddenly you can parameterize and have data sets for cultural diversity across nations. And you can figure out things like Dox. You can have a feedback loop where if you release all the data, open, from teaching kids and learning from kids, how to increase agency there. Or on the medical side, again, self-learning systems. Because you can make everything open with PII and other things. And you can have interesting things like, if you have a cancer AI to help you through your cancer journey, it can tell that you're a human because it's attached on the regulated side. And then you can use the same system for your education, for your government, and more. That's the functional foundational identity. So I think there's a lot of benefits that can emerge here that could help with the AGI and state-of-the-art. But again, I view those almost like expert systems, graduates, consultants from McKinsey, you know, where these are more like the graduates that work for you and your team. And the Apple and Google Intelligence, the world, and others, are probably not going to be doing your healthcare because they don't really want to. They want to do your personal life and everything. They want to interact with healthcare, though. They want to interact with education. But I don't think those organizations are going to go after that. So I think it's just nobody's really looked at that. And again, I think it's inevitable. And the question is, who do we have a million different healthcare models and education models and more, when we only really need one that's then customizable and adaptable? And is MIT?

So is the sort of model for the future that you have in mind one where, because you're hearing, of course, from Sam Altman and Dario that they sort of expect to run away from the competition in the next couple of years. They are not sharing the 0.1 chain of thought. They've said this is kind of GPT-2 level scaling of of runtime compute. And they know how to get to GPT-4. They think they're gonna blow the doors off of everybody there. Dario and team have said in their, I think credibly attributed but leaked fundraising document, that they see a world in like 2025 time frame, which is, by the way, two weeks away, that the leading developers could get so far ahead of the competition that nobody can catch up because they have these sort of, you know, enrichment loops that keep advancing the state-of-the-art. And if they're not sharing that, you know, I also had a recent data point on this when Nathan Lambert from the Allen Institute did an episode on everything he's learned about post-training. And one of the big takeaways for me was he said, in the absence of access to GPT-4 or similar to create the samples that we do the post-training on, we can't do it. You know, there's just no, we, even at the Allen Institute, billionaire estate funded, we don't have the ability to go do a scale AI contract or whatever to bring in the expert data that we would need. So do you think that happens or doesn't happen? If it does happen, is the model sort of they become like the go-to for the pharmaceutical industry and cybersecurity industry and all these sort of like things where you really do still need to push the frontier of the state-of-the-art forward? And then at a more distributed level, we satisfice and like are really more focused on reliability and sort of explainability when it comes to things like making our own personal medical decisions?

I think that's the case. I think that we over-indexed on chefs versus cooks. See, you know, everyone's on the spectrum. Wait, but why had a great post from being a cook that follows recipes to making recipes. How many recipes do you really need to make? Where AI is actually really good now with function calling, large context windows, is following recipes. You give it a handbook, and it's basically replaces a SAS or it replaces an employee or something like that. And we want to have more and more employees that follow the diam instructions. In how often do you need to come up with something brand new? For the vast majority of people, for that, there is the 0.1 type models. There are all these other kind of things. And again, this is where you have the AGI race. But if you're training, say, for instance, on a million H100s like Elon's about to, or a million TPUs like Demis is about to, or a million Trainums where, God forbid, Dario's about to pull guy, you know, like, how big is that model actually going to be? For inference side, you kind of went straight through the consumer side. And now you're like, well, am I going to build a multi-trillion parameter model that is a Cerebras wafer or a Google TPU pod to run? You know, because that's not really suitable for consumers at the side of things. But it might be more intelligent. But it might only be 5% more intelligent, whereas it requires orders of magnitude less compute to have the lower ones. And this is where it becomes very interesting. Because classically, most of computation has been sequential as opposed to like paralyzed. And we're seeing this now again with test-time compute and others. Like, who's going to win? If you need to have millions and millions of GPUs running in parallel, running 0.1 type things to check something and cross-check it, probably more distributed elements. But what is winning if winning is AGI, however you describe it? Then fine, those are the expert systems, and that's where it is. But will one cluster win over a massive amount of post-training with IQ 130 things checking? Will one cluster win over, let's take the China example, getting 100 million people in China to label stuff? This is something that I don't really understand where it's going. Because we haven't proven that we can break through PhD level yet. And even if we did, how useful is that in the context of all the actions of humanity and improving humanity's agency and capability to a brighter future? Like, this is the same question as what do you use a quantum computer for? Actually coming up with the questions that you want to answer is really hard. This is why, like, I love 0.1 for the more advanced stuff, and I'm using it to help me improve my cutter and kind of other things. But the average person isn't going to get much mileage out of it because it's too damn smart. And do you need even smarter than that? You have to set even better questions if it's smarter than that. Whereas again, like I said, this particular area of AI for living, I don't think needs that at all. I think the models right now are good enough, full stop, to make a transformative difference for the world if we organize the data correctly. The category of personal AI, your Apple Intelligence, so Siri isn't stupid, and other things like that, it's nearly good enough. You know, the category of expert systems, yeah, it might be a raceaway thing, but I don't think it's an interconnected raceaway thing. And if it is, then Google will win because Google's interconnect fabric is so much better than everyone's, and they're deploying it at a ridiculous pace. The only potential competitor I could see might be XAI, Anthropic. I don't see Trainium and the data funnel scaling that much, and OpenAI similarly, I don't see it scaling that much.

Yeah, reports of Google's demise, I think, were always greatly exaggerated. I never got that. I mean, I, I remember when it first came out, I was like, this is stupid. They've got the people, they've got the talent. It just takes time to steer the ship. They've come up with all of these things. And again, if you look at the hardware, like we had thousands of TPUs. TPUs are the best interconnect hardware and individually addressable, which is why Google's models are the only ones publicly available with 2 million token context windows, going up to 10 million. And people don't appreciate how insane that is with like 100% needle-to-stack. You can do some crazy things that we're just exploring the surface of right now. I think NotebookLM was probably the first example of that. You upload a gigabyte of things to NotebookLM, and then now you can dial in to talk to the podcast hosts. Yeah, one wonders how much longer we'll need human podcasters, but I'll spare you my personal anxieties about that. I also do have a a challenge which I fail on still every day of spending a dollar a day on Gemini Flash. And it is just mostly for me a reminder of how cheap things have become and how much information I really could be processing for very, very little info. And I, I still just need to like figure out the right structures to do that for me. Just get to watch your screen and tell you when you're being unproductive. Yeah, I saw that demo in the last couple days. And yeah, that's the always-on, observing, occasionally interjecting AI. I think it's going to be a really interesting paradigm. I really like your chef and, yeah, go, go ahead.

No, I was just saying, I think again, if you are boiling these models down to everything, it does come down to engineering. This is why Elon managed to catch up so quickly, right? Because he's an engineering maestro. Like, 100,000 million GPUs, all good. But then, like I said, if you look at the different types of hardware and the interconnectability and the addressable RAM and other things like that, and not paying the Nvidia premium, like it matters a lot. This is why I think the latest like drama with the OpenAI and their bug posts and things like that, like, hey, there's Elon, this, this, they were talking about buying Cerebras because that massive addressable memory and things like that. So it'll be very interesting to see how that all hands out. So I like your chef and cook notion as a simple heuristic where maybe a one, two, three, four, whatever, goes off and figures out how to cure particular kinds of cancer. Or maybe Claude 405 or Gemini 3 is doing that stuff. And then on a daily basis, we satisfice and we try to make sure that we're applying the actual latest known stuff in an effective way, as opposed to trying to advance the the state-of-the-art, you know, each one of us as individuals.

Let's talk about the structure for this. I think in the primer for the intelligent internet, you do a really nice job of walking through the different tiers of the critical resources, right? I mean, obviously, infrastructure, compute infrastructure, and data, as well as of course, the algorithms and the human talent that makes it work, are the fundamental inputs. And you've kind of mapped out a structure where there is like the highest tier, biggest scale of those, and then a more distributed level, and then a more local level, and ultimately individual users to take advantage of this kind of on their own terms. Take us through that vision for for how this actually gets built out, because it is like a physical capital thing and structured to achieve the goals that you've articulated.

Yeah, so again, if I kind of look at the future, I see very rapidly we're going to see economic disruption because the labor theory of productivity is about to be challenged. You used to have to find people. And again, anyone listening to this, finding human capital is the hardest thing, whereas AI will follow your instructions and it's very reliable, relatively speaking. I mean, sure, it was a goldfish that ear this year, you know, like a Sant PhD has had his coffee and just respond to stuff. We're solving most of those things right now, right? So I was like, there needs to be this capital stock buildout. And right now, we're seeing it on an individual level because you're getting M4 MacBooks with 16 GB of VRAM as standard. You're seeing 8 GB on 407s on laptops. And you cas had 8 GB on the integrated C's and AMD chips with the 70 ATMs. Those will soon all standardize, in my opinion, to 16 GB for AI compute, be it with an MPU at 5 watts, or be it with or neural processor, or more advanced graphics options at 25 to 125 watts. This is a buildout that enables intelligence on the edge. And again, every manufacturer is doing that. But then you think about a hospital, and that's what we call the distributed nodes. You want to have a standardized stack that go into any hospital, transform the data, and then add it into face to that to interact with your personal intelligence and more. And then you want what we call hyper nodes, which are national nodes to organize all the healthcare knowledge of an entire nation, or organize all the local cultural data of an entire nation, and keep that up to date as well. So we kind of, we have this stack ranking where there's different levels of computation from inference level, 16 gigabytes, to let's say, right now, fine-tuning level at a hospital level, like 128 H100s to 256 can get any hospital or bank or anything like that pretty much fine-tuned. Then you've got the national level, which should probably be in the thousands of H100 equivalents to organize all of that. Maybe there's an order of magnitude each way. And then when I looked at that, I was like, this is really interesting because this is a defined capital buildout that you want as standardized as possible so you can have boxes that plug in that can just look at stuff because the models are amazing at that infrastructure level. And they offer different types of experience at different levels. But again, one healthcare model for CAN is very easy to turn multilingual for everyone in the world, right? And one radiology model, and one education model, or series of those. And that's actually the next part. I think we mentioned it briefly in the prime. I was like, that looks a bit like Bitcoin, as it were, which is maybe a story for later on on the day, on dedicated infrastructure buildout to create an intelligent capital stock that then gives capability, just like building factories gave capability. But you don't need factories anymore, right? You actually need pure hardware running as standardized models as possible. And those standardized models need to be based on standardized data sets and just have regular releases and be predictable in the way they operate.

Okay, I have a lot of little detailed follow-up questions. I guess for starters, so much is being made right now about the energy requirements of AI that I find this kind of confusing. On the one hand, obviously, we hear, you know, $7 trillion buildouts and crazy energy grid expansions to some extent. I sort of interpret those as like trying to expand the over-in window so we can get like a couple of new power plants online versus like very little new construction obviously in the US in recent history. It is, I think, a very good grounding to talk about the idea that a 5-watt machine can power an AI. I wonder if you could flush that out a little bit more. Like, what, you know, it seems like an 8B Llama 3.3 8B Y is kind of hitting at that GPT-4 level now, and that's quite runnable on a, certainly on a new Mac. Can you give us a little bit more there on like exactly how you envision that?

Yeah, so an 8B model is runnable on a Mac Neural Engine. So a Mac Neural Engine will use about 5 watts of electricity as well, an Intel MPU. But if you want more, like I've got my my M2 Max and waiting for my M4 Max, that will use up to 100 watts of electricity, right? If you look at the H100, the H100 uses 1,000 watts of electricity. So you can use more energy and it'll go faster, but 10 tokens a second is human reading speed. So you can, you know, you see these models go like that. If it's faster, you can read. What's the point, in some ways, right? And there's a question of how many tokens does a human need to flourish? And so, so we can do some calculations around that. And we're like, well, if you don't need everything synchronous and you can let it go, then 5 watts of electricity is probably all anyone needs. And again, you're seeing that because you can now run models on a smartphone. And smartphones tend to top out at 25 watts of electricity. This is where it becomes again, very interesting. Because let's rewind a bit. One, at Stability, we did Stable Diffusion, and that cost maybe $5 million of computer turtle, right, to do. And that's a decent amount of megawatts of electricity, you know, is a million or something H100 hours. And an H100 is what, gosh, I can't remember, 400 watts. So 400 megawatts of electricity, is that right? Something like that. And that's a good amount. But then to run the model is just a little bit on the edge. So you think about it, the pre-training and then the post-training, etc., comes out with an artifact, which is this compressed set of weights that then means that you don't have to re-train that thing. Where a lot of them exponential energy usage is coming from is a lack of optimization. And this is why Meta releases Llama. One of the reasons they got 350,000 H100s, they're about to go bigger. 10% are used for training. They've improved the efficiency of Llama by more than 10% by releasing it. So the remaining 315,000, it more than pays for itself. You've seen this, for example, JMO released their new video model, Machi Day One. It required, I think, 140 GB of VRAM. Within a few days, it required 20. And now it requires eight. Like, again, there's massive optimizations around these models that connect there as well. Because it depends on what are you optimizing for. But you look at Google, you look at Microsoft, look, everyone else, they're like, we can't afford not to do this buildout. Because what if we're wrong? What if we can't optimize? Because no one knows what the satisficing level of performance will be for any of these models. Like, you look at V2 right now that's just come out from Google on the video side, or Sora. It wouldn't surprise me if the VRAM and energy requirements of those dropped by an order of magnitude overnight because someone's figured something out. We don't know how much these will require. So we over-build to a degree, but we're building more and more this capacity up. But then it becomes interesting because although we've had more and more of the energy usage, you think about the energy usage to make a movie, how many megawatts is that versus what it will be? The energy usage to make this podcast, the energy usage to write a book, the energy usage to provide a medical input. And I think that AI is very beneficial for this. Because again, there's this lovely chart that you can look at of GDP per capita versus energy. It was always that relation. It's broken that. Because in a year, we could go to a kid in sub-Saharan Africa and give them a solar panel, an MPU chip, and a version of Llama for that speaks their language. And then they have access to how much knowledge, right? Or a medical version of that, or anything like that. And that's crazy. Because you would have had to build the infrastructure in place. You add Starlink to it, it gets even crazier. But you see what I mean? It's like this is a fundamental change of energy. So I think we have this energy run-up now. But then global energy is not going to keep up consuming unless, of course, I'm wrong, we end up with like freaking Dyson spheres running AGI and other things like that. But I feel that probably won't be the case. There's diminishing returns on scaling. That's an important idea.

Just to put a little bit of my own, you know, back-of-the-envelope math on the energy requirements. Where I live in Detroit, Michigan, the cost of a kilowatt hour of electricity is roughly 20 cents. I think it's a little lower than that still, but call it 20 cents as a nice round number. 5 watts is 1/200th of a kilowatt, obviously. And so I would be looking at, if I'm running my AI on one of these 5-watt machines non-stop, I would be looking at a tenth of a cent an hour in energy cost, or like two, two and a half cents a day. Which means, obviously, I'm looking at single-digit dollars over the course of a year. And I think that is a dramatically underappreciated point about just how accessible the marginal cost to actually operate the thing. You do have the chip cost too, but just how accessible that already is for something that, you know, with a Llama 3.3 8B is like getting really quite good.

Yeah, I mean, Hugging Face just released the latest thing on their 0.1 replication. They've got Llama 1B outperforming Llama 8B with test-time compute. And again, if you put it at 24/7, you're interacting with a, how crazy is that? Where is the lower bound on these things with? Not sure, but it's got to that point where, as you said, people don't appreciate that. And the, the example I gave was solar power. If you hook up a solar panel to the grid, it takes up to like $4 a watt. If it's off-grid, it's less than a dollar a watt for the solar panel. So then what the cost is effectively nothing. And again, the question is, where do we satisfice? Like, how good do you need? Like, you again, you look at V2 today on the video side. Do you need anything better than that to make a Hollywood level? No, not really. Like, sure, there's some small things, but those can always be tied up in post, right? And we figured out consistency and other things. Do we need? We have chatbot models now that are outperforming human doctors on empathy and diagnosis. Do we need better than that? No, we need to optimize the heck out of it and figure out where that lower bound is. And the amazing thing is, all these models are still full of junk. Like, Ilya Sutskever just said at NeurIPS, like, we've run out of data. How much of that data is junk still, even in these top models? We're still like getting to grips with this. Like, we saw the S team previously, Pixar, to create a Stable Diffusion level model with 25 million images. And we use two billion. What is the lower bound of data that you need to have for a language model to achieve X? To teach your kid, does it need to have seen Reddit? No. Does it need to have seen this? If you want to build AGI, yeah, you need all that data, maybe. But do you need it all in one place? Like, what is the limited data analysis? And we've seen interesting things like, I think Player AI just did like an AI that was just trained on knowledge up to 300 BC. I haven't seen what it's like, but those are the types of things that very really interest me. Like, what does that look like? So it feels like, yeah, it feels like we, feels like we have order of magnitude improvement still to come, just from data, honestly. And we're seeing this with fine-web and kind of other things, and the synthetic data sets and augmented data sets that we've had. Yeah, there's a lot. The efficiency curve is is truly crazy. I mean, Phi models from Microsoft are an interesting data point about how small your data set can be. And how, I think theirs are, as far as I know, entirely synthetic for that line of models. Yeah, they did 13.8 EPO on like 400 billion tokens for the core, but then they have to add in like a map of the internet because they're so boring. And people don't want to interact with boring models because again, people take them generalized. So when you add back the internet, they get less boring. But Phi models have the worst sense of humor of any model. Very. Yeah, textbooks aren't all you need for a sense of humor, I guess. But yeah, so many things like that.

Let's talk. Okay, so that's the low level. It's incredibly accessible and affordable to satisfy. Go to the top end of your infrastructure for the intelligent internet. This is the like, national or perhaps even international hypernode level. A lot of questions about that, I guess. But maybe for starters, is your hope for safety and transparency ultimately rooted in open data? It seems like open weights, we can hope for a breakthrough in interpretability, but we don't have it. So it seems like right now, if you were to say, we want to make something we can feel confident is safe, and we feel confident is not going to be too crazy, you would have to do that by sort of saying, we deeply understand the data set, and that's why we believe this is that, right?

I think it's necessary but not sufficient. Like, I think the existence of gold standard data sets that reflect individualized cultures and other things are going to be vitally important when it comes to decision-making AI. Again, how do you understand, understand what the people of Mississippi want, or the people of Malawi want, or the people of Mumbai want, if you haven't mapped out the individual culture, the decisions, and kind of other things? And open data with open models probably makes them more interpretable, but no one's figured out interpretability. Again, lots of interesting SE and other research, but we haven't figured this out yet. I think it will help. And I think again, the existence of these gold standard data sets will help the entire space. Because like when Luther released the Pile, it became a standard. When Lion released the data sets, it becomes a standard. That's why I originally called the company Shelling, because I was like, let's build standardized gold standard data sets and just improve them rigorously. Which is what we thought the hypernodes would do. The hypernodes coordinate and improve the data sets and then can go down to the organizational, community, and individual level and have a self-learning system that again, just improve and improve and keep it up to date. Then anyone can take those because we release everything MIT permissionless, which is the other important thing. Like, you don't, if someone, you break the rules and they shut off your AI bit screwed, then, right? Like for whatever reason. And then I think again, that can move towards standards. Because the highest value AIs will be decision-supporting and making AI. And what do those look like? And how do we understand what those do? And are they've got repeatable frameworks? The most widely used AIs will be the AIs that are available, especially if they're permissionless. I mean, we had 300 million downloads of our models at Stability AI, you know, that's a lot of freaking downloads because they're just available. But then people built around them. They didn't need permission. So I think that again, it's necessary but not sufficient. Because like, otherwise, I look at Anthropic and OpenAI's models, like, how will they incorporate conceptualizations of Chinese or Japanese ethics? Is anyone even taking that seriously? But then what if they're used to make decisions in the Japanese government, as they kind of are right now? Like, I think you need to be very deliberate in the way that you figure this out on the underlying principles and the feedback loops there. And that was the final part of the intelligent internet, which was, I think, people should have a say and be participatory. Which is why we're looking at the digital asset space, we're looking at distributed governance, we're looking at all these other things. So is there a way that can all be brought together? Which is not easy, but, you know, hopefully we can figure that out.

Yeah, well, I'm interested to hear more about it. Sounds like thoughts there are still in development. But whereas the user level seems like it almost is going to take care of itself in as much as everybody is continuing to buy phones already. I don't really know what to expect at the hypernode level. Like, I don't know how many hypernodes you think we need. Are they billion-dollar, are they 10 billion dollar investments? And who's going to make them?

Well, so I think a company, a country's competitiveness was about its physical infrastructure. How many things can you build? It was about its graduates. What's our intelligence, smart graduate network, right? It will become about your intelligent capital stock. How much compute coordinated into models do we have access to? And again, if you look at the online thing of China, US versus beyond the jingoism, it is a question of, let's build compute in America, let's build compute in China. Because from a test-time compute perspective, that is what your competitiveness becomes. And you're seeing this across the world. Like Malaysia and Johor just built a $9.6 billion AI data center. You're seeing billions of dollars across Asia. I don't think you need to have that much beyond the thousands range. Because although there's like $240 billion or something like that of AI building going on, when you start looking at how much individual clusters and things like that are, like the fastest, largest biomedical healthcare cluster in the world is probably Chan Zuckerberg at a thousand H100s. So that's like $20 million a year, which is not nothing, but it's not ridiculous. You know? And so you're like, how much will probably be in the thousands range is what I think will be sufficient to organize national data sets and create them and even provide almost free AI to the vast majority of people on a reasonable size nation. You're talking about like thousands of H100s being enough at the national level? Yes. And this is not in the short term. This is like long-term H100 equivalents. Like, once you build a gold standard data set for education for Bangladesh, how much do you really need to update that? Not that much. And if the outcome is a model that's updated every so often, that's a few gigabytes, then goes onto everyone's phones, provided by TCL or Huawei or whatever, then again, that isn't much. And what's the cost of inference? Pretty much nothing. A shift again that needs to be deliberately. This is separate to the state of the, the state of the art costs a lot, and it costs a lot to inference, and it costs a lot to do everything else. But for me, like I said, this is for me, the most interesting part. Give every child their own AI that's aligned with them. Make sure no one's ever lonely through their healthcare journey anymore. Make governments more representative, finance non-corruptible, and things like that.

I'm surprised by how small you're projecting that to be. Is there any assumption there about like having a general, if I'm Malaysia, for example, well, you said built a $1 billion one, but let's say I am a small country and I've got my Chan Zuckerberg scale thing, and it's thousands of H100 equivalents. I can't train a foundation model from scratch on that, right? So I, I can do a lot of fine-tuning, I can do a lot of data processing. Or at least it would seem hard to do foundation models from scratch. You just need one entity to do it. Like my Meta doing Llama is good enough for almost most things, right? And they did 20 million odd H100 hours for the base version of LLM, and that's what, let's say $40 million spent for Llama 7B 3. And then everyone could just take that and fine-tune it. But to tune it requires 128 to 256 H100s. And the order of magnitude cost is $100,000, $200,000 to create an Arabic version of that. And so again, we're blinded by the massive compute and the edge stuff. When actually it doesn't require that much to do the tuning. The thousands enables you to actually give access to your people to the technology, especially if again, you don't require rate limits or satisficing level. And it also allows you to do the tuning. So you probably start out with a few hundred and then you build up from there. And again, when you look at the disparity of compute and energy and other things, it's very interesting. Like I was in South Africa, I think the fastest cluster there was like 50 or 100 H100s. But then the clouds are building even larger ones. This is why it's very achievable if we're coordinated. But then do you want to be giving your people that level of healthcare AI, or that level of healthcare AI? Do you want to have your companies and your country be able to call on these experts, or even more experts? So this is why I think there'll be a very interesting capital stock question. And when we look at things like 5G, the amount of AI spending has not caught up to the amount of 5G spending around the world. This is far more important than 5G when it comes to a country, a community, and individuals' productivity levels. You know, again, like just think about it, having an expert in your pocket, that's your Huawei smartphone. Having your hospital suddenly have all the latest stuff and being available to all the people in your community, that's a 50 grand type of thing, you know, 100,000. It scales. And the more you spend, the more competitive you'll be, the more productive you'll be, the more access you'll have. And again, I just, I was just shocked by how many people were actually looking at this particular area and what was required for that. So I think if we go back to the start, one entity will need to pre-train base models. That's our common knowledge. Then you have sectorial data sets where you sectorally specialize. And then localized data sets. So you can have the generalized model customized for Mexican law, Indonesian accounting, Japanese medical, and everything is completely transparent across that. And the amount of compute is primarily that pre-training at the top. But then require the hundreds to tune of equivalent.

So it seems like, I guess, with maybe one caveat that the Llama models are not open data. It seems like you feel like they already have enough raw power to serve as the base for a lot of this fine-tuning around the world. Do you worry like what happens if Llama 4 doesn't get open sourced? And do you feel like alternative, with the other end of that, would be like, do you feel like it's just not viable because it's not open data and there needs to be a a true fully open data alternative?

Llama 4 is am will be amazing for a vast variety of things. I don't think it's suitable for regulated industry. And again, regulated industry is typically the stuff for living. So it's just someone needs to go out and build those and those standardized systems and release them MIT and then that makes the world a better place and then enable access to those. So I think Llama is fantastic. I think Qwen is fantastic. I think where those end up in a few years, nobody knows. They're run by Alibaba, they're run by Meta. And again, those are corporations. I think they want to build infrastructure to teach all the kids, honestly. Like, if eventually, if you think about the infrastructure that will teach your child, an AI that's fully open and owned by you to act as a regulator, that then calls into a Llama-based service that can then call into an Anthropic or Google or OpenAI service, it'll be a mixture of models, right? And you'll have the protective model there. And then your kid's ability might be that you have a base level. This is why we have this concept, Universal Basic AI, which I think is important. That's your base level. But then you might want more, and you pay more for more. I think that's absolutely fine. So again, I don't think what we're looking at, what we're doing, replaces Llama. I think there needs to exist open weight models. And I think the two best, or the three best, are DeepSeek, Llama by Meta, and Qwen by Alibaba. And I think those serve their purpose. But they will never release the data set that underlies that because they're not trying to build infrastructure. They have a different objective function. Just like I don't think Apple will release all the data inside Apple Intelligence, just like I think Google won't release all the data inside Gemma. And even if they did, like I said, it doesn't really align with the reasoning of the organization.

Yeah, so how do you, this maybe gets a little bit to the crypto question, or at least the, you know, the incentive engineering, if you will. You've got this concept in the intelligent internet primer of proof of beneficial compute. And I'm curious to hear how you think this can best be organized or catalyzed so that these, I guess, mostly it would be countries around the world, and maybe a mix of other private institutions too, if they want to, how do they all come to see it in as being in their interest to contribute to this project that seems like it'll be an amazing trick if you can pull that off.

Yeah, it would be nice, wouldn't it? I mean, I'd be looking at a lot of business models. Again, the last podcast we had with yourself and, uh, others was like talking about business models in AI. And, you know, like the Stability experience was interesting. We took it up to tens of millions of revenue run rates in just a few years. And others was still burning, but it was getting towards profitability. Got a bit tight at the end. I think the latest CEO said there was less than 90 days of cash, which is better than Affle was back in the day. That's I didn't do another giant round. I was like, where's the sustainable thing here? When I kind of took a step back and I looked what I really wanted to do, like I worked on United Nations AI initiative on COVID-19, UN Global Pulse, one other things. I got into AI when my son was diagnosed with autism. I built an system to, I was like, how do you build infrastructure that's available to everyone? And what?

Does that look like and thinking about the future? One of the things that I've alluded to in this podcast is that the labor theory of productivity, which is a center of economics, is broken. And then the nature of money is probably going to change because what does an Optimus or an O or other things need? Some sort of GPU inference-based currency, as it were? Some people have said that Bitcoin might be the future of these agents. Again, you've seen AI agents now go across crypto channels and other things. And I was like, if I've got a massive amount of compute coming online, and I know that, again, the American P for your Cities Hospital is about to go from that to that, and I can build a standardized stack for that, we not use that to secure brand new type of currency, similar to Bitcoin, but with the properties of Ethereum? And we sell the currency and get supercomputers for health and autism and cancer and other things like that. And the more we help people, the more money it has. The more we get these distributed nodes running, the more secure it is. The amount of flops securing the network increases.

This comes at a time of deregulation in the US and massive amount of digital assets because almost all of them are rubbish and full of raccoons. But you still have Bitcoin ETFs and even nations thinking about Bitcoin sovereign reserves. But for me, the sovereign reserve thing never made sense because that was always a balance of payments or commodities. I don't have enough oil, I don't have enough corn in my granaries, I don't have enough intelligence. So I was like, if I could build that as an institutional asset, and again, we're finalizing the details of that, we'll release it in the new year. That could be something interesting whereby people could buy it, they could hold it. It's secured by increasing amounts of compute, and then you can allocate it. So you can take your coins and allocate them to cancer research or Alzheimer's or A or Mexican education. And then we give that Universal basic AI or the research grants or others. That plays on this transition period. It plays on this structural demand growth for digital assets that are decent, of which there aren't really any. And it's a finite amount of capital that's required, not cheap, so hundreds of millions or billions. That means that everyone will never be alone again on their cancer journey. Yeah, that means that every child will will have an AI that allows them to achieve their potential. And I was like, that's a good deal. And so we should give this a go because I can't think of any other way to get people participating. I couldn't think of another way to get this technology out there without turning evil, as it were. Because again, you want to have a line model whereby helping people you gain value. And it seems a big demand for it. But like I said, putting the final touches, that's why the prim was like, here's the goal. And now it's about, let's release details of how we get there and get people involved.

On that, in terms of the incentive you want to create, if the mechanism is as yet not fully defined, is the idea supposed to be kind of similar to a a Bitcoin where I'm like buying in now in anticipation of some sort of like compute appreciation in the future? Or like, what is my individual or even national incentive to get involved? It's my co. Except for usually we're hammer. So like, we're actually thinking of just copying Bitcoin straight, um, but with some adment. But rather than mining Bitcoin, then doing sales, because you want to provision compute for open source effectively. And then you want to use it for positive stuff, be it inference to provide Universal basic AI or building these data sets and models. This why on like advisory board of Render and others where there's millions of GPUs. Rather than selling it at it going to a treasury or going to Lamborghinis, you sell sell coins. And then we will build the fastest healthcare super cluster in the world and make it available to everyone working in healthcare to build open models and release them. MIT. The more you build, the more you help, the more status you get. And like I said, there's no high status digital assets right now. Then what will happen, we believe is that as you build a full stack, a nation can buy the reserves. And then you put the money back into the nation to put a cluster in that nation to provide this. And the economics works well. People can donate and they get the tax write off. But then they control the direction of voting of the coins. So working through this. But we're like, this could be a way that you get allow everyone to participate. They can direct the network compute for something that has defined amount of things. Let's just at the moment follow the Bitcoin mechanism. So 21 million coins and everything. And that would be very positive because what we need is a lot of compute right now to build the models and data sets. And then we have an inevitability of compute for healthcare, education, all these other things. Class verified accounts. That's a very interesting setup. And again, it's in this particular area that we don't see anyone else in AI doing. But it's like, we should have as soon as possible, nobody being alone in their cancer journey anymore. Or when one son was diagnosed with autism, the feeling of loss I had, that should not be there for anyone. And that's the type of thing people can get behind. And I think this is where crypto has succeeded and failed. People forming communities and being able to participate, especially with the USD regulation will be a big boo. Having some sort of traceability over funds not going to all those but clusters that are very well defined that community organizations and academics can use is going to be great. And then should there be any cost to everyone having that cancer AI? No, it should be free. Education should be free. And if we can externalize it with the desire of people to hold digital assets for appreciation or for status or for anything else, even better. So like I said, that's taken quite a few months to design. We're nearly there. The primary was the first bit. And we'll be releasing more details of all of this soon. I think it's complimentary to what other people in the space are doing as well. I think everyone will benefit from the data sets. Everyone will benefit from these things. And the final be like to, I'm very worried about what money looks like over the next 5, 10, 20 years and how the economics of everything works when that label theory productivity is just broken. Yeah, there's that's a dramatically under theorized problem, I would say for sure. I'm using O1 to help me write the paper on it right now, actually. For the new year, it it is crazy though, because you think about the Philippines, why would anyone hire a call center worker again? And you think about physical embodiment, you think about laborers. Like in 20 years, will a robot be a better plumber than any plumber? Yeah. And how much do you pay for your plumber today? It's expensive. When the pipes burst, right? That robot will be $2 an hour. But your own home robot will have all the knowledge of an expert plumber. Probably 10 years away, to be honest. So I would participate in this network for multiple reasons. One, and it's funny because I don't still really understand Bitcoin. I have to say, it continues to kind of confound me that it just runs up and up and up. You know, it doesn't seem like there's anything really there still yet other than future appreciation. And that's like a hard trick to replicate. So the incentives that you outlined are one, contributing to the public and common good. Two, some sort of voting control over how pooled resources are allocated. Yeah. And third is appreciation. Although I have to confess, I'm not quite sure. Like, is there a redemption mechanism? No. A bit. At the moment, the way we designed it, it is like Bitcoin. So it appreciates. What say reason as a Bitcoin? Bitcoin is a hedge against the world going to crap. This is a hedge against the intelligence age disrupting economics. And the amount of flops per coin is likely to outpace the amount of flops per dollar because it will be provisioning more and more and more compute. You know, so that's kind of the theory. We'll see what it's like in practice. And we've still got a lot of improvement to go here. But it does seem like again, we're at this transition phase whereby something like this is unequivocally good, especially when you make it in those real terms. Like again, is any podcast listener right now not want to have all the clim knowledge in the world organized and available to everyone? Or that cancer AI available to everyone? And if they can contribute to that and have participation in that with the upside? And with the US administration changing to more positive, even better. Hundred billion dollars has gone into Bitcoin ETFs this year. Again, order of magnitude to build these things. It's like maybe a billion or something like that, which sounds crazy just to toss around. But you've seen what AI funding Rams are right now. Yeah. Billion is very. I mean, at the level of countries, it's obviously extremely affordable. So so the level of VCS. You know, and again, the wonderful thing about this is once a model is trained, once once we get to that level of performance with that cancer model to help everyone in all these languages, you don't need to train it again. And I don't think we've ever seen anything like that, right? It's like having infinitely replicable graduates or chefs, you know, or cooks. Sorry.

What technology aside from continuing to define the protocol itself, what technology is needed for this to work? One space I've been watching really closely is distributed training. And it seems like we've just seen like several different papers and proof points that show that that is going to work, at least with like multiple larish nodes. Not necessarily clear if it works at the level of like everybody contributing their spare laptop compute long term. But I'd love to hear your thoughts about sort of distributed compute and maybe any other technologies that you see as necessary that don't yet exist. I think decentralized training doesn't really make that much sense versus decentralized tuning, having armies of agents improve the data sets. But Dolo, Dro, and others have shown really interesting kind of advances. And again, like I said, I'm on The Advisory Board of some of these. And the crypto XAI space is now like $40 billion. You look at something like Bittensor, that's a $12 billion network that's trading hundreds of millions of dollars a day for optimization. It's like the most on steroids Kaggle you can see. Um, you have millions of GPUs in the other network available. I think again, the data side will be more interesting. And again, this sequential stuff. As we get O1 type models that can run on my M4 Max or now M2 Max, seem to be M4 Max, looking forward to it. Nano texture display. The probably one area that is probably most important to properly decentralizing this and ownership and control and governance is verifiable inference. And again, we're making some big advances in there. Hyperbolic and Allur and others are doing a lot of work around that. So that then anyone can contribute their compute. But the data equation of this and building gold standard data sets is more important than the pre-training. For AE pre-training, just have a cluster and just train on that cluster. Like it's not rocket science, you know how to do it. Like, will it be a Llama type model versus or transform model versus something hybrid Jambo style? Who knows, right? But just make your pick. It's pretty straightforward to pre-train. Host training and optimization, distributed stuff becomes very interesting there. The verifiable inference allows anyone to contribute their compute. And then it's about having secure computation for running these things in regulated industries. And that's some of the TE work that we've seen. But most of it coming together now. And especially like I said, against the context of two other things, which were large context windows and function calling, like making it more deterministic on the outputs. And the final thing we just need is maybe continuous training for the individualized stuff. But again, we've seen big advances in that, like continuous learning. So on the verifiable inference, I did one, and again, I'm weak on crypto. I did one episode on this with uh Professor Dani Kong a little more than a year ago now, I think. And he sort of framed the problem as like, if you are paying for an API, you might want the provider, whether it's OpenAI or whoever, to have some way of guaranteeing to you that they are actually running the model that they said they're running, as opposed to subbing in some cheaper knockoff or whatever and changing things underneath you. I get that. I'm a little less clear on what you have in mind when you imagine anybody contributing compute through verifiable inference. Is the idea that like, I run inference locally and then I send up my like activations and the gradients are sort of computed centrally or something like that? Yeah, like, so I think distributed training on millions of GPUs doesn't make sense. But sending a packet to you that contains Indonesian classical architectural law and checking that for consistency and writing some of the data there does make sense. Putting it through a Llama 70B or 8B model. So I think distributed data augmentation makes sense. Distributed fine-tuning makes sense. And then comparing those things. And then the mining operations as well. But I don't think distributed training makes sense or pre-training, shall we say. So post training and data augmentation. And for those, you really want to have verifiable, you know, what the models are, otherwise you might get drunkness, as it were. Yeah. So I think that unlocks something. It's not required, it just helps. Like right now, what we're looking at is literal H100 clusters in various countries and how we can strap that because they're inevitable. So you might as well put them there and use them for healthcare later, right? And then that gives you enough to then tune models in every country and spin up teams and national champions. And then they can decide what they want as part of this network, right? And H100s are a lot easier to track than 4090s or M4 Maxes, etc. But if you can access that, then like I said, you'll be able to build the better models even better because you'll build better data sets even better and faster. I think again, this is not synthetic data, it's augmented, filtered, cleared data utilizing the LLMs. Yeah, that's quite interesting. And that's basically the answer to the sleeper agent question too, I assume, which I was also wanting to ask. Like, how do we make sure that people are not poisoning the data set from any number of different perspectives? You put a massive amount of computer, putting a fine tune over all of it, checking it for consistency, adapting it as appropriate. Like, again, you can afford to go over the top because then you have a gold standard base. It's again, like you have a good curriculum for your kid, you feel comfortable about that or comfortable about the mechanism and the methodology. So I think again, this is a different type of AI challenge to that other people are looking at. But I think this is the important one, cuz it's what affects our lives in a much more direct way with a particular category of model. And this is why as well, when I was looking at it, I was like, you can build tokens which are security tokens, and you can build things that have utility and redemption mechanisms. Ultimately, Bitcoin is the most successful currency, is $2 trillion. It's money, it's a hedge, it's these. Why don't we replicate that for an AIH? And why don't we think about the currency for an AIH? But have this additional thing that I think is important of, I want to have people able to have some participation. Cuz many of the listeners here are like, how can I even get involved in this? You don't know how. But if you can say, I've got this, and I've helped direct the computer towards cancer, and it's helping people go through their journey on cancer by giving them the chat B great or breakthrough done the cancer class to great.

So are you with the intelligent internet, the company? How do you understand your role in this whole thing? Are you like raising capital to create that one central cluster that will do the pre-training and then distribute that out to the national hypernodes? Or what is the? Well, the plan is to basically the company to bootstrap this, get it going, and then we stack GPUs and we use them ourselves and give them away to build fully open source things, data sets, models, and systems. And that will be a big enabler. We saw this at Stability where we gave away 20 million A100 hours and it led to all sorts of amazing things from open F to RWKB to Stable Diffusion and more. People just need to have sometimes a bit of compute. So I think it's getting it going and then trying to make it distributed and decentralized within a few years and giving people control of their own AI. So every nation who should decide the education data set for the people of India or Bangladesh or Pakistan? The people of India or Bangladesh or Pakistan. And likewise, who should build the generalized data sets for everyone? It should eventually be some sort of multilateral foundation or something like that. But it does require a bit of bootstrapping. So we're a team of like 26 people. We haven't done a funding ground properly yet. A lot of demand for whatever coin we do. But I think this is the right way. But again, we're finalizing all the details of that. So you think you can get over these capital requirement thresholds with a coin type approach without needing to go to like a traditional equity market? Yeah, the again, crypto XAI is now $40 billion trading hundreds of millions of billion over a billion dollars a day now, actually. We could just go that route. Or we might do a combined one where we do equity, then coin, then that. And I think that in doing that, you'd probably be the highest revenue generating AI company in the world because demand for high quality digital assets is only going up. Pricing power and APIs and sales is going down. And nobody's really looked at this. I mean, even for Elon and Sam and like, how difficult is it for them to raise billions of dollars? It's not because they've got the pedigree and things. We've built hundreds of state-of-the-art models of different types. Or team. And that's why many of them join me from Stability, um, having a liquid market for a coin where you can sell coins and literally say, this is a cluster that will just look at Alzheimer's. I think there'll be no shortage of people who want that. I think having the sovereign reserve aspect of this, build up this reserve and will put all the money into AI for your nation and get the teams. And I don't think there'll be any shortage of that. Again, time will tell. But initial things have been very positive. And it was again, something I was thinking about when I left at the end of March. I was like, this feels more close to what I originally wanted to do. It's needed. No one else is doing it. And it's the right time with where regulation, other things are. Because ultimately, crypto is a 3 trillion industry that needs high quality digital assets that people can allocate to. And we need to have some sort of intelligent money and these other things. But it's not easy, you know, like we're trying something new. But keeps it exciting. It's never a dull moment in the AI space, that's for sure.

Yeah, you mentioned a number of projects from Stability and and previously you made a comment along the lines of finding talent is is like the hardest thing or is often a limiting factor. Obviously Stability was in the news a lot for lot of different stories over the year or so before you left. When people would you know send those links to me, I always basically said back to them, I just watched their research output. And Stability has continued to matter, you know, certainly as of the time of those conversations with just like a bunch of really interesting good projects, some of which were, you know, extremely catalytic like Stable Diffusion in terms of changing awareness and what people understood to be possible. Some were just like quiet contributions to the commons like the LION dataset and specifically the aesthetic dataset, which is one that there was nothing else really like that. And that made a huge difference actually for one of the little projects I was doing at my company. We also love the MindEye and MindEye 2 papers, even though those are kind of it's not easy to or not hard to imagine a dystopian version of that in the future as well. But the, you know, the fact that mind reading works into world is pretty crazy. What would you say are your tips, tricks, theistic methods for identifying great researchers? Like how did you assemble this team and create that capability which I would say not that many different companies around the world have done basically from scratch in a short period of time without like certainly Microsoft level resources. Uh, what did you, what do you look for? What do you see that tells you somebody is going to make a difference? I I think it was kind of treating the researchers like creatives. And so we first created the communities and again, you know, did a bit of hype, maybe overhype sometimes, you live in you land. But it was a crazy time. And so we had about half a million people in the various communities that we kind of built, incubated, etc. We made it easy to operate and we gave very fast grants to anyone that was doing promising research. So I basically kind of led and coordinated the research. I've got a good eye for that type of thing. But I really looked for people that it didn't matter what the background was. Are they passionate? And are they willing to take a risk and try different things? So many of the research grants we gave, and again, researchers can feed back, we like try and do different things. It doesn't matter if it fails, whereas researchers use as a cost center generally, so you can't really take a risk. You know, when it worked, we put on the afterburners and we scaled it. Um, and then you wanted to empower the teams. So I think last year we had something like 20,000 applicants for research who made 120 offers and 83% 100 offers, 83% were accepted. And before the last, in my entire time at Stability, not a single research left for a competitor. And they were offered a lot of money because they enjoyed working there because they had the creative freedom, the adaptability. Obviously, they were concerned about all the noise and that's very interesting things behind that. But they just got on with things because we enabled them. And we said like, they're like, this training run is F like, doesn't matter so much where you learned. How can you go from that? And we ended up by achieving state-of-the-art in image, video, 3D, audio, protein folding, like small language, a whole bunch of different things by having that mentality. Because you don't need that many people, and you don't even need that many PhDs. We had 14 PhDs out of the at 80. I think things like Fast AI and others are also great kind of grounds. And again, the willingness to try these things. This where communities like Alther, which we incubated then spun out into a 501c3 and LION were great because you had all these diverse talents kind of coming in. I think it depends what you want to do though. Like we were building state of the arc because we were like, we need to catch up. But I was like, I can't scale LLMs right now because there's no business model for that. I thought video and other things were the way. But again, then I realized that like right now, video is about to satisfice and there'll only be like, doesn't matter which one you use. It was very difficult to figure out the sustainability. So instead, we were just like, let's do the research and let's enable them. And have I think the teams are happy. The rest of the organization maybe didn't do quite so well. It wasn't a great CEO. But at the very least, the researchers were happy. Which again, I think the proof was in the pudding when it came towards the end of it. We spun out the Black Forest Labs team and some of the other teams. And I think again, they were happy. And I think having research of happiness at the top was very important. It's certainly something you see coming through in the Sam and Elon emails as well, is just the incredible intense focus on top talent. Yeah. And I think, you know, what type of top talent? What are you trying to do? So we found the on in the larger organizations didn't tend to fit in very well versus again, the Mavericks. Because we were trying to push state-of-the-art and try different things. There are so many projects that didn't emerge because because we made mistakes, not M mistakes, they just didn't work. So Stable Diffusion was 250,000 A100 hours to train the original. It was maybe 5 million in total of tests or something like that, and failed experiments, etc. So you never see all those, see the other side of things. And again, what are you trying to do? Like, we had a bit of a crisis at Stability because like, are we trying to build AGI? Do we need to compete against all that? That was something I was thinking about a lot about last year. Like, are we competing against DeepMind? Because we again, one route is what you've seen with David and Midjourney. You know, I gave the grant to help Jenny get going, and then he did everything else himself. I don't think he's taking money from anyone else. Where he's like, I'm building a research and innovation lab and going to make sustainable to this business. And we could have just focused on media, which is what Stability is doing now. And given our role, that would have been far lower and cheaper to do. Could have done that. We were like, we had to compete against DeepMind and OpenAI and others. And maybe we would have succeeded or not. But then it became very confusing to do that. And I think the one thing that is important if you're trying to build a great team in AI is give a very defined function and then build team that are passionate about that function. And then give the resources they need to do that. And then that will work very well. So that's probably the final missing piece, which is the clarity of thought. Which is why I took a bit more time on the intelligent internet. Now it's very straightforward. If you want to have the biggest impact on autism as a researcher, you will come and join me. And we'll have a dedicated team to autism, building open infrastructure for autism, for government, for cancer, and others. And so that's why we're setting up this time. And then the monetization mechanism is very straightforward. And then hopefully lots of GPUs will be coming their way to make them even happier. But you don't really need that many to be honest, compared to what many people say, to have an impact. Couple maybe last big picture questions. And I I really appreciate how generous you've been with your time. And it's it's been great to get a deeper look at your thinking as you spit up the intelligent internet. You mentioned that you had signed the pause letter in like mid-23 when that came out, maybe right after GPT-4, the original one. I sometimes call myself an adoption accelerationist, hyperscaling pauser. I don't spend too much time advocating for a pause because I do still think there's like some room at least to advance the state-of-the-art before it gets like too crazy. And you know, I do want those cancer cures, of course. Where are you now on if you see the sort of what people really need to live better lives, either already or soon, decoupling from what is going on to advance the state-of-the-art? Where does that leave you in terms of should governments enforce a pause? Obviously, there's a collective action problem there too, if they wanted to. But what do you think people should do or should what should the goal be for that sort of thing? So when I signed that letter last year, I think it was the only AI CEO to sign it. Actually, Elon signed it, but he wasn't AI CEO yet. I did it because I wanted like, my my base assumption was that if we got to AGI, and my thinking then, it would be like Scarlett Johansson in that movie, Her. You know, goodbye and thanks for all the GPUs, because humans are kind of boring. That's the kind of narrow AGI thinking. But the reason I signed it was I was like, there needs to be more discussion about this. Just like right now, the amount of discussion about labor displacement and massive economic turmoil that will be caused by this technology is like nothing. Like I talk to top, they're not even thinking about it. It's insane. Again, I give the example of the Philippines, a call center sector, it's dead. Why would you hire another call center worker? You look at suedes of industry and Devon and these other things, like that's going to be bad. Now, where I am is that it's too late to have a thought. Which is again, why I said back then, let's get this out. And maybe we move too far towards AI dorism and things like that. But we do have to again, be very measured in the way we think about things. An example of this is copyright law and IP. So, you know, I got sued from $1.6 trillion at Stability by Getty, even though we didn't use any of their images in the thing. And we filed out all the watermarks and stuff. And in fact, the original Stable Diffusion was trained by Robin and his team before they joined Stability. So we made sure to have an arms length. We didn't even look at everything that kind of deliberate. Today in the UK, the UK government is advocating that you can, they'll have an exemption where AI companies can train on any copyrighted material. You can see, you know, and that's similar to Singapore. If you can legally access, so you can train on it, which kind of makes sense. Cuz when you have a Tesla Optimus robot, is it going to close its eyes and its ears when it sees a copyrighted thing? Like, oh no, I'm hearing Taylor Swift, let's turn off my ears. Probably not going to happen, right? But these are complex and nuance topics. Just like pausing and others. I was like, that's the last chance we have to do that. Now, the biggest companies in the world are AI companies. Microsoft, Google, Amazon are AI first companies. Jeff Bezos is back at Amazon. You know, Sergey Brin is back at Google. You know, Bill Gates, he's back at, he's back at Microsoft, I think as well. Governments are trying to build their inging capital stock. They realize this is the biggest thing. You can't stop it now. You can't take time to think before the next wave of disruption occurs. And I don't think that governments really can afford to regulate. Like Europe is falling into its trap, but again, people will just build models and have extra competitiveness where they can. Again, Singapore is a great example that may follow because you can't afford to be left behind. You can't afford not to build the AI chips. You can't afford not to support. Again, I would use Europe as the maybe one exception to that. And the final part of that is America under the new administration, crypto and AI are going to go full pelt under the administration. There's no way it won't. And you have to keep up with America. How else are you going to kind of manage it? And so it's going to be very interesting to see how that plays out. So it's not a pause anymore. Is there any other regulatory action that you would recommend? I mean, people are groping or grasping for anything they can get their hands on. Liability rules come up a lot. That was kind of at the by the end, maybe not so much the heart of the S1047 debate, but at one point it was more central to that idea. Was maybe we'll hold the foundation model developers accountable. But then, geez, how does that impact open source? And how does that impact the relationship with local fine-tuners and developers? Any thoughts on sort of who should bear responsibility for incidents of AI gone wrong? Yeah, it's very interesting to see all these things evolved. Like I've had some similar discussions to what Mark Andreessen said about how many AI companies can succeed, which is another reason to get decentralized. After that experience, gosh, there was a lot of very strange ones. Liability, these things won't work because again, you're in a competitive environment which from a game theoretic perspective means that you can't afford to lose competitiveness over a defined period of time, especially as the Overton window has shifted on politics now to kind of be more tech friendly after literally the US government was trying to strangle the sector or make it so only a few people can do it. I think that the biggest impact, simplest thing is this: if an AI is used in a decision-making process in any regulated industry, it has to have transparency, data logged, because you are what you eat. These models are ultimately the data. Like if you look at all these different architectures, again, we funded RWKV getting off the ground or Mamba or any of this, all the performances are the same for the same data set. It comes down to data. You are what you eat. And the decision-making AI are the ones that will kill us, will make us successful, ultimately. The rest is kind of helpful. And so introducing 11 of data transparency over that is important because then, like people will spot, just like when we release Stable Diffusion and there was liability, they were like, oh, there's weird in here. Like, yeah, we don't want weird in there. If you don't want to know about nuclear bioweapons, don't have nuclear bioweapons in the data set. I think the other thing is building good standardized data sets for knowledge and other things. Like again, you said you can't go and achieve at Alna the level of scale AI and other things like that. It's kind of embedded in these models from G54 and others. Building a gold standard of data sets for common knowledge and cultural knowledge just makes everything safer. Because why wouldn't you use those as your base? And again, think about models of the curriculum, Mist perspective was going through high school, university, and then being specialized. Like, what are we really training on when we've got trillions of tokens? What's in there? Have you looked in these data sets? They're crap. Build better base data sets that you can iterate on. And then things will be safer. Have a simple legislation of any decision-making AI and any regulated industry needs to have transparent data. And things will be safer. Like, doesn't necessarily need to be interpretable, but we know that the AI functionality is a derivation of the base data. Beyond that, I can't see what other regulations will stick, honestly. But one that I would highly recommend is regulations around speech data. Like we both to state of the art speech models. We do not release it. It is dangerous. I don't think people appreciate how dangerous it is. But let me give a practical example to people listening here. Imagine someone that you cared about more than anyone, you respected more than anyone, replicate their voice and create a version that you can call them and that you can have discussions on Zoom with them, which is all capable now. How much would you listen to that AI versus a normal AI? More than anything. And if that AI can then access Obama's speech patterns and Churchill and other things, that wave of manipulative speech is coming. We don't have the protections against that. And I think those should be legislated heavily and regulated heavily, especially when you think about our kids. Our kids literally have no defenses against that. So those are probably the areas that I'd look at. One other random idea I've had, and I haven't really developed too much, is speed limits. And this is basically just the idea that like, maybe individuals or organizations or whatever should sort of only be able to do so much inference per per unit time or only use so many flops per unit time. And it's kind of like, just to keep things, you know, operating at a sort of human speed. Maybe we need something like that. You're probably going to have the same game theoretic objection, though, I suppose. You can't, you can't do that. It's like saying you can only hire so many people, right? Like, again, you will lose competitiveness in this increasingly competitive environment. And organizations, others will come out of it. And what is the role of government, right? I mean, someone said the with the monopoly and political violence, ultimately, it's about protection of people and representation of people. And if this is the big leap up, you have people with AI outperforming people without AI, you can't really have these things except for when there's unions and others involved. So you look at your rate limits and you look at the longshoring strike in the US, that is a rate limit. They will only allow a certain amount of robots. Is that optimal for a society though? No, it just so happens they have a physical barrier. And so we'll see those things maybe emerge in certain areas where you're regulated on the amount of AI that you can have. But if you're keeping a free market, which again, unions are counter to in some ways, then you can't have that because who knows what goes on inside an organization, right? And other things. And can you coordinate this globally? This is the bigger thing. Absolutely not. Like again, you think about small nations, they can suddenly outcompete. Look at the role of capital here. Like the Saudis and others of the world, they're like, once this stack is built, like that's why Dua has gone heavily into Cerebras. Like, yeah, build those silicon wafers. So I think that ultimately, you can just set good defaults and you can use market forces and others for that. And again, push for data transparency and some of these edge cases like speech, you need some protection. Cuz I haven't figured out, like unless we wear AirPods all the time, how do you protect against that, you know? Yeah, I mean, it seems like there's probably going to be more of those things too popping up that we don't even necessarily have any precedent for. It's hard to say, of course, what they are. But I certainly expect a lot more weird stuff to start to happen. We live in interesting times, eh? Yeah, no doubt. When you talk about data set filtering, if you don't want the model to know about bionuclear weapons, as you put it, you know, don't have that in the training data. Is that sort of akin to your earlier comment that you think scaling is going to kind of level off? Or is there there seems to be a relation there where it's like, if they generalize out of domain to a certain degree, then they'll fill in those gaps, is kind of what a lot of people worry about. But you don't expect that as much? Well, we embody these base models with a bit too much stuff even when them, right? Ultimately, they're just a bunch of like, freaking weights, a bunch of ones and zeros, right? It's not like a logical system. So you look at the composition of that, how it's built up, and you're like, I want a base level of reasoning model. Like, it doesn't make much sense that we have all the data in these models and knowledge in these models either. Like, that seems a bit wasteful. Like, how much these models know about, they don't need to know. And you think about again, a model that's a medical doctor, should it need any more data than a medical professional gets from kindergarten to medical school? No. I think Yan LeCun has talked about this in some of the Jeopardy kind of things. Can it infer and reason? Yes, because it'll have that capability. But again, the question is, what are you releasing onto the world as infrastructure or services? Because you can control things at the service level. Infrastructure level is a bit different. Like, what base model you release? Stable Diffusion, stop people use it for good things, they use it for bad things. But it was inevitable that you would have reasoning models like that, right? And again, this is especially important when it comes to long context window models. Like again, 2 million, 10 million tokens is a freaking lot. And so reasonable reasoner, he says, when Gemini 01 comes in, Gemini 2, or the equivalent LL version of that, yeah, I'll be able to figure out just about anything. It's inevitable. But in day-to-day, how many people are going to load it with all the bionuclear stuff in the context window? And that's something that is relatively controllable. So I think that we just need to tidy up the stuff in the base models. And that's just not even a question of filtering, it's a question of how do we feed them? Like, what is the curriculum? And then once you have gold standard version of that, I think things become safer because there's less likelihood of them going initially rogue. Because they'll be a function of the data goes in. Now, once you make them agentic and able to traverse the internet for the rubbish that it is, then maybe it's a different story. But again, this is an incremental thing. There are no easy answers, otherwise someone would have answered already. But tell us your basic vision on the safety side. Structured access to state-of-the-art combined with open data, open weights, open access, satisficing day-to-day usage models. Yes, from a functional perspective, universal basic AI for everyone that get smarter and smarter. And we all agree on that. Can be localized and specialized with the power going back to the people. Then personal AI will be driven by market forces with your Apple and Google and other intelligences. And then you will have specialized, highly intelligent models. That's where regulation can come in, access can come in for the most capable models out there. But I think it'll be very difficult. So instead, I think what we should do is focus on if they're in decision-making positions, at least have the data transparent. That could be a great note to end on. Is there anything we didn't touch on or anything else you want to share about the intelligent internet before we break for today? No, we haven't really pushed it out. Have a read. I think it's a nice document along with how to think about AI. We'll have a whole bunch of stuff in the new year. And again, happy to see feedback. It's an iterative process. But you know, it's been a lot of fun. All righty, well, we'll certainly keep watching and maybe I'll be lucky enough to have a chance for another conversation once you've finalized all those details and and put that out there. For now, Emad, founder of the Intelligent Internet, thank you for being part of the Cognitive Revolution.