This Is How OpenAI Goes Broke

When you have a very high valuation, a down round is super painful, right? You know, you they're valued in the last round at 852 million billion dollars uh post money. Um and in the secondary market, they're trading for a lot less than that. And if they were to sort of just say, okay, we accept we're really worth 600 billion. Um you know, the hit to everybody's equity options inside OpenAI would be horrible and they would lose people. And the hit to investors who had believed in open AI would be bad and they would get pissed off and the whole momentum machine that Samman has built would really go through a convulsion.

Welcome to Crafty Markets. Cracks are forming in the open AI story. Last week, the company reportedly proposed giving the US government a 5% stake worth roughly $43 billion as a way to share the upside of AI with the public. Critics, however, argue it would amount to a government bailout and see it as a troubling signal for both OpenAI and the broader AI boom. That news came after reports that OpenAI had pushed back its IPO plans until 2027, adding to concerns about the company's financial position.

So, we wanted to speak with someone who has spent years studying the history of AI and who also believes that open AAI could run out of money in the near future. Sebastian Malib is a prominent journalist, author, Puliter Prize finalist, and senior fellow at the Council on Foreign Relations. And today he's joining us to discuss what is next for OpenAI, what is next for the AI industry, and what investors should be watching. Sebastian, thank you so much for joining me on the show.

I'd love to start with an article you wrote back in January that was titled, "This is what convinced me OpenAI will run out of money." And you said back then, quote, "My bet is that over the next 18 months, OpenAI runs out of money." Uh, we've been seeing a lot of red flags since then, the delaying of the IPO. uh later we saw this proposal for the US government to take a stake in the company. I guess I'll just start with what do you make of the recent news and do you hold to your prediction?

>> Yeah, I do hold to my prediction. Um back in January the burn rate was just crazy. Um, so that although OpenAI had good products and quite a lot of traction, 900 million consumers, they won't be able to charge money for the product. Like 5% of the retail consumers were actually paying if you look at a chart of where these users are. You know, the US is the number two market. India is first and the next three are kind of Brazil, Indonesia and so forth. So these are not rich consumers. You can't charge them very much money. And so they had a business model that imagined that they could throw money in all directions. You know, a collaboration with Johnny IV to have a new form factor which would supplant the iPhone serving Sora video generation models and all this stuff. All of which is very expensive. And yet the revenue side simply wasn't there. So the burn seemed to me to be totally unsustainable. And even though Sam Alman is a magician when it comes to raising money, he wasn't going to be raising $660 billion, which is what the internally projected burn rate was for the next 5 years when you looked at the documents back in January.

Now, since then, what's happened is some good news, right? Because OpenAI, I think, has recognized that it had to get the burn rate down. It's pulled out of a bunch of data center building products, you know, Stargate, all that stuff. It's cancelled Sora, the video generation model, which was a total money loser, and it's tried to impose some sort of order on the chaotic management, but it's only been kind of half successful. And in the meantime, open AI is squeezed between anthropic, which is much better at the frontier enterprise applications like, you know, coding assistance and uh and and cyber security stuff and agentic stuff. Uh and then on the other hand, it's squeezed by um the Gemini model from Google DeepMind uh which has now reached more retail consumers and is way better at monetizing from that because uh Google has plugged AI into its search advertising business and that business is now doing more revenue than ever. So, I I just think that, you know, OpenAI is technically a good lab, but it's very hard to monetize when you have a product where there's a lot of competition and it's kind of a commodity and they're not terribly well managed and they've relied too much on the fake it till you make it Silicon Valley tactic of kind of weird Smoke and Mirrors fundraising gambits.

If you look at the fundraising they did and announced earlier this year, the headline number they raised was $122 uh billion, which is an astronomical amount. But when you dig in, and I'm amazed the press didn't point this out more, about 2/3 of that amount was kind of promises in the future conditional upon having a successful IPO or payment in kind like you know access to compute. The actual real money was a small share of the total fund raise, which raises the question, why announce this massive 122 billion headline number when anyone who digs into it can see it's rubbish. Well, the answer is they're trying to head fake investors into putting more money in. They're trying to persuade people they have momentum. They don't. And this news that you pointed out just recently that they have delayed it seems their IPO into next year is just the latest icing on the cake. The latest evidence that uh they took a big game but they are behind where they say they are.

Do you think that the the delay of the IPO was in large part because of all of this? Because perhaps Sam Alman and the company know that as soon as Wall Street actually gets like an audited review of their financial statements, then suddenly the tide will turn on this company and suddenly people will say, "Sure, you might have a great product, but this is not a sustainable business model." Do you think that that was the concern that people might actually see how the company actually works?

100%. I mean, everybody remembers the Wei Work story when we work was this, you know, rocket ship back in like 2019 and it went out with a prospectus to do the IPO and people looked at it and said this is a joke and nobody wanted to buy the shares and the IPO never happened. Um so you know you can fail in going for the IPO and OpenAI is in this very tough position where on the one hand it needs the IPO because it can't hope to raise enough money if it stays private. On the other hand if it tries to do the IPO it may not succeed and then it's really cooked.

>> Just looking at how much they they are spending at the moment. We just saw the financials that were released by um Ed Zitran who's this independent journalist. He's he got his hands on the numbers. They uh generated $13 billion last year in revenue. They spent $34 billion which means that their operating loss was $21 billion. Um I mean we could talk about the the net loss which was even higher than that than that number but that seems to be like a good roughly uh estimate of of of how this business is actually doing. And you mentioned that they're stuck between on the one hand Gemini and then also Anthropic. Uh Anthropic is an interesting one because we also don't know much about that business and we know that they're unprofitable and we know that they're in a similar business to OpenAI and as you say this technology is becoming increasingly commoditized. They said there were reports that maybe they were coming up on a on a quarter of operating profitability, but I think we probably have to take that with a grain of salt because we don't know how they're doing their accounting. My question, how does Anthropic compare to OpenAI from a business model perspective?

>> I think you're making a good point and I agree with you that we don't know as much as we would do if Anthropic were a public company or if you know the prospectus was public. Um what we do know is that Anthropic has always targeted enterprise customers which means the type of customer that actually pay for the product and we know that it's been ahead on stuff like coding assistance and cyber security AI. Uh not that open AI is bad by the way I mean it's not far behind. uh but I think Anthropic is the cutting edge on those particular applications that enterprises are really willing to pay for. Um and meanwhile um Anthropic has not been sort of distracted into announcing a whole suite of retail-oriented business initiatives which came to nothing. I mean OpenAI announced it was going to do shopping at one point and that doesn't seem to have happened. It said it was going to do ads. I'm not sure they've got terribly far with the ads. It did generate the, you know, Sora video model which just was a huge money loser. Whereas Anthropic never went down that path. So I think Anthropic has been way more laser focused on the part of the market that makes sense, which is the enterprise part um and and just better managed. The other point I'd make is that Anthropic amongst all the frontier labs is known as the one where the churn in terms of the scientists is the lowest. People go there, they believe in the mission mission. They believe in Dario Amade as the leader and they tend to stay there. They don't hop around. Whereas all the other labs are subject to job hopping and that's obviously disruptive.

I think one of the questions that is in investors minds, especially if you're worried about the potential of an AI bubble and the potential that an AI bubble might pop, is is it an open AI problem or is it an AI problem? Is it that open AI is just bad at managing their finances and they pursued all these side projects and they don't really know how to get their spending under control? Or is AI as a business model just too expensive relative to the amount of revenue that could be generated by charging customers for using chat GPT uh or charging enterprises for these larger enterprisewide AI contracts? What is your view on that debate? And just for context for our listeners, you wrote the power law which is one of the most famous books ever on venture capital and it's kind of about how venture works as a business model where you do lose money for a number of years and then you figure it out eventually. Is AI going to be that story or is this different?

My view is that we have an open AI bubble but not a general AI bubble. So I think open AI for the reasons we've discussed is is a 50/50. Look, it might work. I'm not saying I'm I don't know that they're going to fail. I'm just saying there's a 50% chance that by next summer we'll find they couldn't really go public in the private markets. They can't raise enough money and they have to sort of sell themselves at some sort of discount to another company. It could be, you know, Amazon or Microsoft or some other big company that wants an AI team because technically OpenAI is a good team. Right now on the more general issue, yeah, there's debate at the moment about whether enterprise customers are having a oh my god moment where they think, oh, these tokens are just so expensive. Now, I've spent the last 18 months telling my teams that they should just go out and go wild with AI and experiment um and do whatever they feel like and token max. And the more tokens you use, the better of an employee you are because you're showing that you're AI forward. And now, wow, this is expensive and I've got haven't seen a productivity gain yet. And so, what am I doing here? I have to rationalize this. And there are lots of stories out there about how companies are imposing a sort of middle layer between the user in the enterprise and the models. And the middle layer is there to switch a query so that you know if it's a simple query that I'm asking it gets rooted to a cheap low token consumption model. Um and then only if it's a seriously difficult one will it go to a fable or something expensive. So I think there's some sort of sensible rationalization about how the AI customers are spending money on this technology. But fundamentally, fundamentally, if you look back at what's happened since the release of ChhatgPT, the clear story is this is unbelievably exciting fast progress in the tech. I mean, when CHPT came out, the thing hallucinated non-stop. Then when GPT4 was plugged in 6 months later, it basically stopped like 80% of the hallucination. Then you got very long context windows so you could put a whole toll story novel into the model and then query it. Then you got you know these reasoning systems that could do math and logic which had been impossible before. Then you get agentic system. Then you get coding assistance. Then you get cyber security systems. Now you've got like bespoke AI autonomous scientists emerging. This is unbelievably fast progress. So I fundamentally think that you know AI as a sector and therefore the demand for the semiconductors, the data center businesses, all these things that people worry about, I don't think that's a bubble. I think that's for real. And it's going to take a little bit of time for for companies to figure out how to ration their people's use of token so it's sort of sensible, but basically they're going to consume a lot of tokens.

Another data point that the Bears might present that we saw last week uh is Meta launching their cloud business. And this would be the argument against what you're saying, which by the way I I agree with, but I want to play devil's advocate. Um you know, the very thing that Meta said they wouldn't do, they're now doing. They said that they would only launch a cloud business if they had quote overbuilt. These were Mark Zuckerberg's words just a few months ago. The the plan was let's build out all of these data centers, build out all of this compute capacity because we within the meta organization need it so desperately because we're going to build all of these internal AI products and we're going to, you know, AI turbocharge our business. and then they turn around and say, "Actually, we don't have the demand internally that we thought we did, and so we're going to sell it to someone else, and we're going to let someone else figure out how to sell an AI product and how to make that a profitable business, which seems quite bearish uh from a bubble perspective because it basically says, I mean, who else but Meta would be the one to build out their own suite of AI products? If Meta can't crack it, if OpenAI is struggling to crack it, TBD on Anthropic, then who's going to crack this? Who's going to make this not just an interesting technology, but an interesting technology that makes money? And we've seen the same with SpaceX of course that they also decided to sell their compute capacity to uh Anthropic and others because XAI their own model hasn't really got much of a uptake and so they don't need all the compute they've built for their own model. Therefore they're selling it to others. So you could view this as a bare signal as you've just described or you could view it as a bull signal because it means that you've got some consolidation going on in the frontier model space and less competition means better margins for the remaining participants. It means that maybe there will be more pricing power for the ones that are left standing.

So I find I don't agree that that's I think that's the proper reading. The proper reading is we have a rationalization of the market. If you looked at the whole sort of US ecosystem um you know three four months ago you had XAI trying to compete Meta trying to compete and then on top of that you had the big three uh Google Deep Mind Open AI and Anthropic so that's five and that's before you count Mistra in France Coher in Canada and all the Chinese models right that's a lot of competition and I don't think that this thing is going to consolidate down to a winner takes all sort of you know 19 sorry 2010's um social media platform or something. But I think some consolidation is in order such that it looks like cloud computing where there's kind of three or four big providers. So now we've got you know three leaders who are still standing within the US plus the foreign ones. That feels good to me in terms of the future business stability of the sector.

>> If open AI runs out of money as you say per your prediction what do you think the outcome would be? I mean, one of the things that you wrote is that maybe it would be absorbed by another company. Um, I mean, how does that play out if indeed what you're saying might happen does happen?

So look, I think you know, we've seen lots of examples of either acquisitions or more recently Aqua Hires where you know, you have um a smallish AI company like Inflection which Mustafa Sleman was running and then it got sort of sucked into Microsoft or like Character AI which got sucked back into Google. Um so there's a playbook here. Now OpenAI is a lot bigger than either of those two. Um so it would be a more complex playbook but basically it seems to me that um you know the demand for AI talent and for AI products and therefore the compute infrastructure that under that serves that demand. I don't think that's going away because fundamentally I think this is useful stuff that people are going to figure out how to use productively. And so I don't know whether the whole of Open AAI gets bought by Amazon or Microsoft or some other acquirer or alternatively there's some kind of fancy aqua hire deal where part of Open AI is sucked into a big company or alternatively that like you know there's a bit of a splintering and the staff the technical staff at OpenAI get individually hired into other labs. Who knows right? Uh what I'm saying is that um there's a fundamental problem with uh the way they're they're going about their business model. Um I think they understand that which is why you know they pulled out of data center building and various other things in the last 6 months but they've got some way to go to to fix things and patch it up. And you know that one of the lessons about how you do startups um you know coming out of my previous book the power law um is that when you have a very high valuation a down round is super painful right you know you they're valued in the last round at 852 million billion dollars uh post money um and in the secondary market they're trading for a lot less than that and if they were to sort of just say okay we accept we're really worth 600 100 billion um you know the hit to everybody's equity options inside open AI would be horrible and they would lose people and the hit to investors who had believed in open AI would be bad and they would get pissed off and the whole momentum machine that Samman has built would really go through a convulsion. Now, it might be what you have to do to make this thing sustainable because but but my point is once you ratchet all the way up to this very high valuation, it's difficult to climb down. Um and and and that is why I think he says why doesn't the government have 5%. Because a strategy to get out of this box that he's in is for Altman to give 5% to the government and then the government will say right you know open AI is too important to fail now because we own 5% or 10% or something and they'll do what they did with Intel which they took a 10% stake in last year and next thing you know the commerce secretary Lutnik is like calling other tech companies in the valley saying you're going to do a deal with Intel you're going to bring Intel in as a partner on your next project blah blah blah and so you You got the US government, a Trumpy US government, strongarmming other companies into giving business once once they're in your corner. So that I think that is what Sam Wman's strategy is here to kind of recruit the you know the investment banker to whom you can't say no the US government which seems like he's basically just trying to take some sort of workaround shortcut around capitalism. And it seems like we are increasingly seeing that like if you can't figure it out in the free market then oh let's just go over to Washington walk into the White House kiss the president's feet and then hopefully he'll save us. And we are increasingly seeing that that is what is actually happening. We're seeing the the government taking up stakes in multiple companies. We're seeing the odds that the government will take stakes in even more companies. Those are going up. uh they may indeed take a stake in open AI. Last I checked on the prediction markets the odds of that happening were more than a third. Um it's possible that they would do the same with Anthropic, with Palunteer, with Andrew. It makes me very upset because I think of it as cheating. I think that you're kind of cheating the game of capitalism. I'd be curious to get your views there. And then following up on that, if that actually happens, say OpenAI is running out of money and then Trump just bails them out in whatever way we use taxpayer dollars to just continue to subsidize the business. What comes after that? Does that mean that OpenAI is fine? Does that mean that the rest of the AI industry is on shaky ground? I'm not even I'm not quite sure how to even model out that potential scenario.

>> First of all, I think your formulation that they're cheating capitalism and you know they're going to the government and doing an end round capitalism. I mean I think that's a good uh perceptive and quite amusing insight. So so thank you for that. Um uh I also though would say that you know this is like just the way the world is going. I mean or at least the US is going. So if you look at the number of American companies in which the US government has announced either done a deal or has announced the deal and it's yet to be consummated. You know, a colleague of mine called Jonathan Hillman at the Council of Foreign Relations did did a formal count which just went up on the Council of Foreign Relations website and the answer is there are 30 of them 30 such companies since um the Trump team came into power in January 2025 where where there's an equity stake from the US government in a private company. So this is where the world is going and I think this trend has been very much encouraged by the deceptive example of Intel. Right? So in the case of Intel, if you look at what their performance has been since the government took a state last August, it's been fantastic. I mean, it's been way better than the Philadelphia semiconductor index, which is the normal index you would look at as a kind of comparable for how Intel has done. Intel, I think, is up like almost 400%. The uh socks or the semiconductor index in Philadelphia, that that's up like 150%. So these you Intel has done incredibly well since the government came in. And I think people just lose sight of the fact that, you know, yeah, it did well because you've got, you know, the commerce department calling up other companies and ordering them to do business with Intel. So Intel gets a whole bunch of contracts and is like turning its game around um because you've got the government behind, you know, picking a winner. Now, it's one thing to say the government might have a justification picking a winner when we have a problem with, you know, all of the cutting edge semiconductors being made in Taiwan. We don't want to be reliant on an island that could be invaded by China. And so, we want domestic US semiconductor manufacturing. I get that argument, right? I don't believe in extending the same argument to open AI which is just one of multiple American foundation model builders. We don't need open AI for any strategic reason. Right? So there would be no justification for picking a winner uh around open AI. So I think it's uh I I I think that the you know capitalism is sometimes justifiably twisted because you have a national security reason to do so. Backing open AI would not be a justifiable instance.

>> Well, I could imagine that the justification that would be floated is open AI isn't systemic to the real economy, but they'd maybe try to say that, but it's systemic to the stock market because, you know, Microsoft's future revenues depend so heavily on open AI. Uh so do I mean I mean Google, Amazon, X I mean all of basically all the hyperscalers, Oracle, a lot of these companies are very very important to portfolios. Uh they are what make wealthy people wealthy in a lot of cases and maybe the argument for Trump would be oh well we we need to keep this thing afloat otherwise people's stocks are going to go down. What would you make of that argument?

I'd say welcome to China. I mean that's what the kind of thing the Chinese government would do is prop up the stock market with government intervention of that sort. I mean look in the United States when the Federal Reserve um you know operates a policy that looks like it might be about stabilizing the stock market people freak out and say well that's a Fed put and you know that creates bubbles more bubbles in the future and you know capitalism doesn't work unless there's real risk involved and that's the Fed. If you have like bunch of political types in Washington, you know, the commerce department and so forth picking winners and distorting outcomes in the market, you don't have a market anymore. It's not a free market. Your point about this is an end run around, you know, capitalism or to say the same point differently, you know, this is an end run against the notion of a fair level playing field on which different companies compete fairly and then the most efficient ones win. That's what we're supposed to believe in as the wellspring of efficiency in American capitalism. Well, if you start deleing the playing field by picking open AI as a winner, you've just trashed that.

We'll be right back after the break. And for even more markets content, sign up for our newsletter at profarkets.com.

Support for the show comes from Delete Me. The internet makes it easy for scammers, stalkers, and hackers to do their thing. That's just a fact. But it's important to remember that there are ways to protect yourself. That starts with protecting your data. And Delete Me can help you do just that. Delete Me does all the hard work to wipe your personal information from data broker websites. It was named the top pick for data removal services by Wire Cutter. And our producer, Jennifer Sanchez, has tried Delete Me. She was surprised to find out how much personal information was actually already out there, but Delete Me made it very easy for her to clean it up. Take control of your data and keep your private life private by signing up for Delete Me now. at a special discount for our listeners. Get 20% off your delete me plan when you go to joindeleteme.com/profg and use promo code profg at checkout. The only way to get 20% off is to go to joindeme.com/profg and enter code profg at checkout. That's jointdeme.com/profg code profg.

We're back with Profy Markets. This is a good segue uh into China, which is a topic that you also wrote about recently. Uh the title of your piece was quote, "I went to China to see its progress on AI. We can't beat it." I was in the New York Times recently, your op-ed. Uh what did you learn about Chinese progress on AI and why do you think that we can't beat it?

We haven't mentioned this yet, but I'm going to mention it now because you've given the excuse. So, I I published a book uh this year um called The Infinity Machine about Demisabis. Um Deep line. There you are. >> I was going to get to it. >> Go read The Infinity Machine, folks. But no, seriously, the thing about China is it does everything faster. And so, uh they published although they got my manuscript last and then they had to translate it into Chinese and then they wanted photographs and other embellishments. they produce in a way a more complex product but they actually published it before uh Penguin Press in the United States or any of the other deals I had in other countries. So I go to China right at the beginning of my book tour. I spent eight days um you know going to four different cities Hangja, Shenzhen you know Shanghai, Beijing talking to both computer scientists at the private labs like you know Huawei and Ant Group and so forth and then also talking to academic um computer scientists uh from universities and what struck me about these guys is that first of all they talk about safety they just bring it up the notion which I've heard from friends in Washington that the Chinese don't give a damn about AI safety is just not true. They do talk about safety. Now, I'm not claiming that the government's policy is to pursue safety or that the majority view in China is that they want safety. China is like the US. China has some accelerationists and some people who want to go slower because they're worried about the safety issue. That's the same as the US. So, neither side is going to deescalate and start going slower unless the other one does as well. But what I'm saying is it's to to caricature China as like only acceleration is 100% that's just wrong. And so there is scope to talk to them about safety and maybe we'll come on to that. But the other thing which I observed is that China is very good and very focused on applications. And so if you go to a company like you know Hike Vision which is under US sanctions and it's kind of a outof body sort of double take experience when you go there because on the one hand it feels like an American tech company. I love tech companies. They're kind of all about building cool things and making the world better. I kind of buy that. I drink that Kool-Aid. I kind of believe in it. I like technology. Right. So I see these people trying to build cool technology and they show me stuff like for example there is an AI kind of scanning camera thing and you point it at some water and you get a reading on the pollution count in the water and because they've created that guess what there is an internal market in water pollution reduction between different Chinese cities. So if you're the downstream city, you will pay the upstream city to reduce the pollution in the water that's going to come downstream to you. So you can do pollution reduction when you can measure the pollution. And this is what they're doing at this company. That's what they're building. But they're also under sanctions, these guys, by the US because the US says, and you know, historically this was actually true that they're building other kinds of cameras which are good for surveillance of civilians and so forth in Shinjzang and whatever. So, so they're both bad guys and they're cool guys. It's a difficult thing to figure out, but whatever they think, whether they are bad or cool, they ain't going away. These guys are for real. They are building cool technology. You go to Huawei, they've got application after application. You know, here is our special, you know, AI to service the bullet train between Shanghai and Beijing every evening. We used to have human technicians, mechanics, who would go under the train and make sure it's all fine. Now we just have AI cameras and a couple of robots and they fix the train for you. They are doing this. We're not stopping them. We have imposed chip export controls on China to try the to hold them back. It hasn't worked. These guys are are moving ahead. And the latest thing was you probably saw is this model from a group called JIPU uh in China which isn't quite as good as mythos from anthropic but it's pretty close. So we are kidding ourselves if we kind of assume away the reality of China being a technology superpower and we need to on the contrary get our heads out of the ostrich position in the sand and start talking to China about what happens when they have a mythos level model which could hack every single bank uh in the global financial system and wreak havoc. We need to persuade them not to release it on an open source openweight basis because then any criminal can do it and there won't be an off switch.

>> What is your view then on on AI policy with China? The obviously the big debate is should we have these export controls? Should we sell chips to China? Are we selling weapons to our enemy or do we need to sell dumber chips basically dumber weapons to the enemy or should we not have these export controls at all? I mean, do you think that we should have a policy or is the path forward more of a method of diplomacy?

I believe in American power first of all. I work at the Council on Foreign Relations in New York and you know, we do geopolitics all day long and I believe that US power is generally a force for good. So, I would rather that the Chinese were behind on AI. Okay? And so, to the extent that a chip export ban helps us to be ahead, I support it. And indeed, when it was first announced in 2022, I wrote a massive long essay in the Washington Post about why this was a good idea. But the reason I've had my doubts recently is that I look at the results and I'm not seeing that Chinese models are that far behind. And in the meantime, because they're not far behind, I think we have to reckon with the reality that they are building models which are going to destabilize the global cyber system. And unless we persuade them not to release them on an open weight basis, which is what they're doing at the moment, we have serious trouble on our hands. Like everything in cyerspace will be destabilized. And we need a policy to deal with this proliferation risk. And I would be willing I'm in favor of the chip export ban if we could have it for free and there'd be no downside. But if the effect of having chip export controls is that we can't talk to them about an agreement on not doing open weight mythos then I'm willing to trade a bit on the chip export band.

Just looking at some of the how these models have affected the ecosystem and something we were saying earlier uh in the US you've got anthropic you've got open AAI you've got Gemini those are kind of the heavy weights in the US right now but it does seem that as pricing becomes more of an issue companies are more interested in cheaper models which usually means Chinese models and indeed that is exactly what we're seeing when we look at uh open router which is basically tracks developer marketplace for AI models. Chinese models went from less than a third of developer traffic in late 2025 to 60% by mid 2026. There are some companies that American companies that have started using Chinese models. Cursor, Airbnb, uh Shopify, Uber, Microsoft is currently testing Deep Seek. Um, what do you make of this transition over to the Chinese models, specifically the cheap Chinese models, and what role does that play in potentially a policy discussion?

Well, I mean, it shows you that they make good models that serious American companies are thinking of using. And so, that's another argument for why you can't just pretend that China can be beaten and that's the end of it. And these guys are for real and we have to work with them, not just against them. Um now I I I think it's useful to just for a moment think through the lens of the cold war. Um where when there were nuclear weapons in the cold war there were two kinds of big risk right one was a nuclear conflration between the Soviet Union and the United States and the way we prevented that was through mutually assured destruction basically close to par in the power of the two arsenals and therefore deterrence. On the other hand, there was a different category of risk from nuclear weapons which was the proliferation of these systems to rogue states or terrorists and so forth. And we dealt with that with a separate mechanism which was the non-prololiferation regime. Now the point is we were both competing with Russia, having an arms race with Russia, having a Cuban missile crisis with Russia, being told by the Russians at the United Nations, "We will bury you." As Chrisv said when he banged his shoe on the table. So there was deadly serious competition between the two superpowers but at the same time there was cooperation on a non-prololiferation agreement and the way I see the future with AI is that we'll do the same. We will have inevitable you know competition between China and the US but we'll also I hope have collaboration because the proliferation risk is too awful to contemplate unless you have some collaboration.

It seems though that what they're doing is basically stealing what we have and people are calling it distillation. Um, and and you wrote about this and your definition of distillation, quote, "Every time a US lab produces a cutting edge model, Chinese rivals quickly reverse engineer its capabilities and build a copycat version, the follower has the advantage." And when I look at how I mean companies are switching to Chinese models because Chinese models are cheaper and as you say maybe they'll use the more advanced cutting edge models in America that are more expensive for certain tasks, the Chinese models for other tasks which essentially means that we are kind of maybe we're collaborating but also you could say that we're sort of seeding advantage to the enemy to the Chinese players uh in the AI ecosystem. And it seems that the reason that those models are good, cheap, is because of distillation, i.e. theft. Um, I don't know if I'm being crude by calling it theft. I don't think I am. And I think the Chinese have shown a pretty strong track record of stealing intellectual property from the US and then going out and monetizing it on their own terms. I mean what do we know about this process of distillation and what do we know about why and how the Chinese models have be gotten so cheap and therefore so successful on a global scale.

So distillation is a process which involves um asking a very strong model like America a new American model comes out. A Chinese um copycat would ask a ton of questions to that model and get the answers and the answers amount to training data such that you know you can train the Chinese model like if the question is like this the answer should be like that. uh and when a frontier like a first mover an American lab has to train the model in some specific very complicated frontier expertise like let's say you know quantum phys physics they expensively hire a bunch of quantum physicists and engage them in creating problem sets and you know model questions and answers and generating that training data for the AI is a super expensive time consuming painful process But if you've once you've created the AI that can replicate all those quantum physicists, the Chinese can come along and not hire the human quantum physicist but just query the machine equivalent and and that's what distillation is. Now um when these Chinese companies do this, it is not illegal but it is uh against contract. In other words, you when you when you sign up to use an American model, you sign you check some boxes and you sign an agreement saying, you know, I'm not going to like query you gazillion times and then train my own model by copying what you've done. And so they are violating contract, but not sort of federal law. That's my understanding of it. Now, whatever the legal nicities, the question is, can you stop it? I mean, I'm all in favor of stopping that if we can. And it seems to me that Anthropic and you know Google and and OpenAI have all of the commercial incentives in the world to put in anti-distillation safeguards if they can come up with some. So I think this is a like a selfolving problem in so far as it has a solution. And by the way, I should add, you know, Elon Musk the other day or a few months ago casually admitted that his company XAI had distilled from one of the US frontier competitors. But look, it this is this is the rough and tumble of of the marketplace. It's not nice. But the practical question is, you know, let's stop it if we can. But in so far as we can't, we have to live with a reality on the ground which is that the Chinese models are good.

Something I can't figure out is I mean if these AI labs are as capable as they say they are, can they not figure out some cyber security method to stop the distillation from happening? If Mythos is the most powerful cyber security technology and software that the world has ever seen, but we can't figure out how to get these Chinese developers to stop querying and replicating the same software. I'm sort of like surely you guys can can figure it out. I guess my followup would be say they do figure it out. Say we do put an end to Chinese distillation of US AI. Would that not one solve America's problems in one fell swoop and two kind of put an end to Chinese AI or at least the progress that they have been making? I mean, is that not kind of a poison pill for for China?

I'm not sure um is the answer whether if you could stop distillation and by the way I think the latest anthropic models do have some anti-distillation um technology built into them. So we'll see how effective that turns out to be. It's going to be obviously a you know cat and mouse both sides trying to get smarter um on this one. But but to to answer your question let's posit that um US labs figure out a way to stop distillation. Uh, would the Chinese fall behind like a lot or just a bit? I'm not sure anybody really knows the answer. Um, I kind of suspect that, you know, if they needed to generate their own data, they would and they would pay more money and it would be more expensive and it would take them a bit more time, but they would get there cuz they've got plenty of extremely smart Chinese scientists that they could engage in generating training data.

We'll be right back. And for even more markets content, sign up for our newsletter at profarkets.com.

We're back with Profy Markets. Okay, let's turn to the book for a moment. Uh your most recent book was The Infinity Machine, Demis Hassaba, Steepmind Mind, and the Quest for Super Intelligence. Um there there was one uh quote from the book that really stood out. You said, quote, "If you couldn't negotiate safety mechanisms inside one company, what chance would there be to negotiate common safeguards among multiple labs in multiple countries?" Which really relates to kind of what we're discussing here in terms of AI safety and AI policy, but also there's an important implication of that, which is that safety mechanisms were not able to be negotiated within a company. What did you learn about the inner workings of these AI labs and why can't they figure that stuff out?

Embedded in the story of Google deep mind and demiss is this sort of morality tale about somebody who really wanted to make AI safe and that was his sort of driving passion from the time he founded deep mind in 2010. um you know he bonded with his co-founder Shane Le at a safety lecture in which they discussed you know the potential for AI to attack humanity by the year 2030 which turns out to be perhaps a precient projection um uh at least the capability is going to be there whether the AI attacks is a different question but I mean anyway the point is Deis Habis was thinking about safety since the beginning and so when he sold his company to Google in 2014 14 a condition of the sale was you got to give me safety and ethics oversight board I can't let AI be rolled out into the world just on the say so of the Google corporate board there has to be these independent people from outside they mentioned Barack Obama as an example when Barack Obama

Was leaving the presidency, you know. Could we have somebody of that stature who would be on a board saying when it's safe to roll it out? Right? That was Demis' vision. And it was coupled with another hope, which was that all of the major scientists would come together in one single effort to roll AI out into the world, so there would be no competitive pressure to go unsafely and too quickly. Right?

And it turns out, and this kind of transpires through the story that I tell, that all of Demis' optimistic stories about how he was going to make AI safe, they all crashed and burned. The idea of just one lab building AI turned out to be a pipe dream. It turns out that humanity is too tribal and competitive and disputatious. There will be multiple labs. When you are kind of confronted with the prospect of being able to build a god machine, there'll be plenty of different sects of worshippers trying to do that, right?

Um, and the idea of oversight within Google. Ultimately, the Google board would not agree to giving some outside grandees a veto over how they used this technology that they were spending billions of dollars on developing. They weren't going to do that on a fiduciary basis obligation to their shoulders. They couldn't. They felt right. So the point being that, you know, the experiment that Demis ran at DeepMind, and I was, I discovered all these internal documents which had the back and forth between the red lines from one team of lawyers to the other team about the exact safety mechanisms that they might use, and all the sort of secret strategizing that Demis did to threaten to spin out of Google if he didn't get the safety oversight he wanted. And then the, like, the Google Deep Mind general counsel threatened me and said I wasn't allowed to publish any of this, and I said, "The heck with you, I'm publishing it anyway." So it was all quite dramatic.

But the bottom line of the story is, you know, it turns out to be impossible to impose safety restraints, uh, within one AI lab when that lab is in a competitive posture with respect to others. And we saw the same thing play out, of course, at OpenAI, but more in public when the safety board temporarily fired Sam Altman for like, five days. So, so, you know, what this shows us is that if you want to stop a race which has multiple players, you need the government to enforce restraint on all of the players at once.

And if there are players in China, you need the Chinese government to buy in and also agree to put restraints on their guys when the US puts restraints on labs within the US. France, Canada, that's fine. Basically, the US can compel compliance in those places because Canada or Mistral in France depend on American technology and the American market to function. Um, but with China, you can't compel them. So there needs to be two countries, two governments that do a deal where everybody agrees to put some caution and like checking of models before they're released.

It was the policy of the US government that they were going to do none of that. And they said, I mean, they they even issued an executive order, um, banning states from trying to regulate AI in their own way. Um, but then it seems like they've kind of turned on this. Last month, Trump signed a new executive order where he basically asks companies to hand over their models to the government, let the government check them and then kind of greenlight them. But I mean, on the one hand, it's progress in the direction that you think is the, it's the right direction, but also it's not very harsh or strict or strong. It's basically just like, "Hey, could you please send your model over? We'd appreciate that." What do you make of of Trump's uh AI policy at this point in terms of government oversight over these AI models and their safety?

Given my perspective that government needs to get involved, I've been very much cheered up by what's happened since April when Mythos first came on the scene and galvanized the US government into paying attention and restricting the release. Because although you could argue, you know, correctly, that on paper the executive order, um, is kind of a voluntary collaboration system with Frontier Labs, blah, blah, blah, blah, the reality is it's not voluntary in the least. Right? I mean, Commerce recently called up Sam Altman at OpenAI and ordered him to seek government permission before he gave his latest model to any customer. Right? Government has to sign off on each customer, customer by customer. This is extremely heavy-handed, right? So, so I I I think they're in it for real. Um, the government, they have realized that they can't let this stuff disseminate around the world without being controlled by government, and, um, and so we're going to get pretty tough controls.

I think the gap in the system is that they're not talking about doing this in coordination with China because the US policy world has two kinds of China expert. That you've got the kind of people who are always hawkish on China, and then the people who used to be a bit hopeful about collaborating with China, but then Xi Jinping rose to power and seemed to kind of frustrate all those hopes of collaboration, and so that group of former doves flipped and became uber hawks on China. So you've basically got the the traditional hawks and the new hawks, but they're both hawkish, and nobody wants to say they want to talk to China. This is the problem. This is the huge gap in the posture because the US government has done a 180 on domestic regulation of domestic models, and I welcome that. The next thing that's going to come, just because it's necessary and they're not going to have a choice, is they're going to have to get over their inhibition about talking to China.

So is that sort of the solution? Is get in a room with Xi Jinping and become partners in tackling this together? I mean, it sounds like kind of simplistic, but maybe that actually is the way to do it. I, the alternative would be, you know, force their hand in some way, create some sort of policy where you say, "No, you're not going to get any chips, or you're not going to get this, or you're not going to get that." Your view is we just need to talk with them and have more of a relationship.

>> It's a bit more complicated than that. I mean, I think you can talk and also put pressure on them at the same time. I mean, going back to that Cold War analogy, there was a vicious competition between the Soviet Union and the United States at the same time as there was collaboration over proliferation. And so, I think there will be competition. And by the way, you know, there are ideas, um, around strengthening the chip export controls. And I'm not against that. Like, there is one theory of the case. You know, economists sometimes talk about corner solutions. You can either have a fully pegged currency or a fully floating one. But if you go for some mushy middle ground where it's kind of semi-pegged, then hedge fund speculators are going to see that you're not really determined to defend that, and they're going to eat you for lunch, breakfast, and dinner. So, it's the same thing with this AI policy. There are corner solutions. You could either like give up the export controls or be willing to give them up and go talk to them and say, "Okay, we know you didn't like that. As a show of our sincerity in wanting to work with you, you know, we're going to offer to loosen those controls, but in return, we want you to collaborate on fixing this non-proliferation risk." Right? That would be one corner solution. Or the other corner solution is you don't say that. To the contrary, you tighten up the chip export controls. There's this massive loophole right now whereby if you're a Chinese model builder, get this. You can train on Nvidia chips, the most advanced versions, all day long because the cloud compute that you access is in Malaysia or some other offshore place, which is fully allowed to import Nvidia chips, the most recent sort. Right? This is a crazy loophole. You're telling the Chinese they can't use Nvidia chips, but then you're letting them just like use a data center kind of across the border. It's nuts that that loophole exists, right? So the other corner solution is get serious about the policy that you've enunciated and cut off the loophole and cut off the distillation and put China in a position where it's so weak, it's kind of begging for collaboration. Now, I'm agnostic. I'm like, we need to collaborate. I'm I'm flexible on how we get there. I think there's different theories.

>> Just going back to to Trump's changing of his positioning. It used to be, "We're not going to regulate, we're not going to have any oversight because we believe that if we do that, then it stifles innovation, and we want, you know, markets to do their thing and AI labs to sort of run free uninhibited, etc." Then Mythos happens, Anthropic's model, that was a real concern for cybersecurity. Um, and then Trump changes his tune and issues this executive order, uh, which you believe in this, I think is fairly so, that that actually is like stringent. They they do take it seriously. Why do you think that happened? What was it about Mythos? Was it maybe something to do with China? Like, why did they do this thing that ultimately did amount to a 180, uh, on AI policy?

>> Simply because Mythos was so powerful, it was very threatening. I mean, the prospect that you could take this model and find code vulnerabilities in every single entity on the internet and then hack it, like that's curtains for the financial system. So that's why they took it seriously.

>> Yeah. Fair enough. I mean, I think throughout this topic, throughout this topic, the logic of the technology is going to force governments to do things which six months earlier they said they would never ever do. And that's happened with domestic regulation already in the US. I believe it's going to happen with international collaboration. I've already started to see pushback from, I mean, Silicon Valley spent a long time not being friends with Washington, and then in the last couple of years, they became very close friends with with people in Washington. Um, and I would assume that this is going to be, I don't know, this is going to cause a rift again because a lot of the technologists said that what we want is government to have no involvement in this, in in AI, in in artificial intelligence capabilities. And Trump said, "Sounds good. I'm with you." And now he's not. Um, I'm not really sure what that means for the relationship between Silicon Valley and Washington, but I assume, I don't know if you have any insight into this. I assume it's not going to be great.

Well, look, I mean, you've got this sort of Putin and the oligarchs sort of story. You've got, you know, endless examples of authoritarian governments, um, with, you know, big business titans, and, you know, where does the power lie and how how stable is that relationship? And the answer is it tends not to be stable, point one, and point two, the government wins because they have the monopoly on coercion. And so I think Silicon Valley, you know, is figuring that out and they realize that the government is too powerful to ignore. I mean, you know, Darday tried to say to the government, "You shouldn't use these tools for certain things like mass domestic surveillance," and the government said, "Get lost. You are going to call you a supply chain risk, and we're not going to be dictated to you." I mean, who won that fight? Clearly the government won.

>> It has been fascinating watching Trump use the full power of that coercion, even this week when he decided to step into the proceedings of the World Cup, and he got exactly what he wanted, and the US.

>> Absolutely.

>> Got their player back. Um, just as we start to wrap up here, you have studied a lot of the characters in AI. You wrote your book about Demis Hassabis, founder of Google DeepMind, kind of the OpenAI before OpenAI. You studied a lot of these characters. From your research, from writing that book, what did you learn about the people in AI? And what has that kind of told you about what might ultimately happen next and who might ultimately win the AI race?

You've got Sam Altman, who is essentially a commercial opportunist who wants to win commercially, or at least survive commercially. And, you know, his drive is to to be a big shot. And he thought of running for governor of California at one point and being a political big shot, but then he decided that building AI was like a bigger big shot, and he wants to just like put his imprint on it. He's not obviously a PhD scientist. He doesn't have even a first degree because he dropped out of Stanford to do other stuff. Is not to say he isn't anything other than massively smart, um, but he isn't a deep scientist.

Uh, then you've got people like Darday and Demis Hassabis, who are PhD scientists who come at this from that perspective, who want to use AI to advance deep science. That's their deepest motivation, and I believe it's very deep with both of them, and I believe that's the reason why they are number one and number two in this race. Um, it's good for recruiting the best scientists. It's also good for holding together and leading a fundamentally scientific enterprise like building artificial general intelligence. And the point where where this came home to me is, you know, when I was talking to Dennis one day about his motivation for building AI, and he started to say, "Listen, when I'm reading scientific papers at 2:00 in the morning, Sebastian, I see reality staring at me in the face, calling at me, saying, 'I'm here to be discovered.' And if I had artificial general intelligence, I could discover the fundamental rules that explain the fabric of reality. It would be like understanding all of nature, which presumably may have been created by some kind of divine intelligence. And so, in this sense, my quest for AGI is kind of like my way of getting closer to what I might call God."

Sebastian Mallaby is the Paul A. Volcker, Senior Fellow for International Economics, the Council on Foreign Relations. A two-time Pulitzer Prize finalist. He is the author of six books, including "More Money Than God" and "The Power Law," which have become investment classics. His latest book is "The Infinity Machine: Demis Hassabis, DeepMind, and the Quest for Superintelligence." He also co-hosts a weekly CFR podcast, "The Spillover," which examines the ripple effects of global events across policy, geopolitics, economics, technology, and financial markets. Sebastian, thank you so much for your time.

>> Thank you so much. Nice to talk to you. Thank you for listening to Prof Markets from Prof Media. If you liked what you heard, give us a follow and join us for a fresh take on markets on Monday.