Transcription
Deep Seek R1 was released just a few days ago, and it has sent shockwaves through the AI industry. R1 is an AI model that has the ability to think just like OpenAI's cutting-edge state-of-the-art 01 and 03 models. But here's the thing: it's completely open source and open weights. Deep Seek, a small Chinese company, gave all of it away for free, and they even detailed how to reproduce it.
But that's not even the craziest part. It was trained for just $5 million, as compared to the tens and hundreds of millions of dollars that most people in the AI industry thought was required to train a model of this caliber. This has sent everyone in the AI industry scrambling to understand the ramifications. Deep Seek has been called everything from the downfall of major US tech companies like OpenAI and Meta to the greatest gift to humanity to a Chinese scoop meant to shake the US to its core.
This story is wild, so buckle up. Just about a week ago, President Trump, Sam Altman, the founder and CEO of OpenAI, the founder of Oracle, and many others got together to make the announcement about Project Stargate. That is a $500 billion investment in AI infrastructure built in the US, on top of the billions and potentially trillions that have already been spent on GPUs, mostly coming from Nvidia.
Then, right after that, Mark Zuckerberg doubled down on how much his company Meta is going to spend on AI infrastructure, also stating that they are going to continue to spend many billions of dollars building out energy infrastructure and AI infrastructure. So the theme among the biggest tech companies in the world is to spend as much as we can to win at AI.
Then something happened on January 20th, 2025. A small Chinese research firm called Deep Seek released Deep Seek R1, a completely open-source, open-weights AI model that has the ability to think, also known as test time compute. This model is directly competitive, if not slightly better, than the 01 model by OpenAI, which cost hundreds of millions of dollars to train.
Just like that, the AI world was flipped upside down. All of a sudden, we had this completely open-source version of a state-of-the-art model that we didn't think we were going to have so soon, let alone to be absolutely open source and essentially free. The initial reaction was extremely strong. I've made multiple videos about it; I'll drop them down in the description below. People looked at this and were stunned.
The biggest names in the AI industry realized we now had a completely open-source state-of-the-art model. As everybody was taking this in and super excited that we could play around with it and reproduce it, suddenly the tone shifted. In the technical paper that was released alongside Deep Seek, it was noted that the model was trained for just $5 million. That is a fraction of the cost of what every other state-of-the-art model cost to train.
Now think about what this means. Meta, Microsoft, OpenAI, and all of the Magnificent Seven—basically the biggest seven tech companies in the world—have been investing trillions of dollars building out AI infrastructure. Then, all of a sudden, this little Chinese company comes along, open sources a model that's comparable to the best models out there, and not only did they make it completely free, but they said it only cost $5 million.
Suddenly, a lot of analysts are looking at these big companies spending billions of dollars per year and thinking, "Do we really need that?" A lot of people are pointing at these big companies, saying, "You guys are about to lose. You've invested so much money, and it wasn't even necessary."
Now, I will tell you I do not agree with that whatsoever, but that is a theme going on right now in the AI industry. Then, somebody on Twitter asked how Deep Seek is going to make money because they're giving it away for free. How are they actually going to make money? The API endpoint to actually run the model is really, really cheap, and you don't even need it; you can run it on your own hardware.
Then this tweet went viral. Deep Seek's holding, and this is the Chinese company's name, is a quant company, meaning they are mathematicians tasked with building trading algorithms simply to make money. That's it. Many years already, super smart guys with top math backgrounds happen to own a lot of GPUs for trading and mining purposes, and Deep Seek is their side project for squeezing those GPUs. Essentially, this is not even the main function of the company; this was a side project.
So a handful of smart people got together, figured out how to make a state-of-the-art model incredibly cheap, and upended the entire AI industry, and it was their side project. That's insane to think about, and this went viral. The memes were strong.
Let me show you a few of the reactions from people in the industry. Here's one from Simp for Satoshi: "Sam spent more on this," referencing this incredible automobile, which I know was multiple millions of dollars, and that's Sam Altman driving it, "than Deep Seek did to train the model that killed OpenAI."
Now again, I don't really believe this. I will explain what I think is going on in a little bit. Here we have Neil Koos, son of Vinod Khosla, saying, "Deep Seek is a CCP state scoop plus economic warfare to make American AI unprofitable. They are faking the cost was low to justify setting price low and hoping everyone switches to it to damage AI competitiveness in the US. Don't take the bait."
Now, there was a community note saying there's zero evidence of this, and that wasn't even the craziest take. In Davos, Alexander Wang, the CEO of Scale AI, basically called out Deep Seek, saying, "No, they actually have many more GPUs than they're telling us," simply because there is an export ban on China from the US. We cannot export our cutting-edge chips to them at scale.
So in the research paper, if they admitted that they had a bunch of GPUs, obviously the US would be pretty pissed. In this clip, Alexander Wang talks about how Deep Seek probably has 50,000 H100s, which are Nvidia's top-of-the-line GPUs, and the fact that they can't talk about it because it goes against the export controls that the US has in place.
Maybe that's true, although again, remember everything is open-sourced, and they really went into deep detail—Deep Seek did—into how they actually produced this model for so cheap. The company Hugging Face is reproducing it right now.
Now let me show you some posts from Emad, who is the founder of Stability AI, who basically ran the numbers and figured out, "Yeah, it's actually legit what they're saying. Deep Seek are not faking the cost of run; it's pretty much in line with what you'd expect given the data structure, active parameters, and other elements and other models trained by other people. You can run it independently at the same cost; it's a good lab working hard."
Now, it wasn't enough; he didn't put any numbers, but of course, he followed up and did check this out. He basically says, "For those who want the numbers, here it is: Optimize H100 could do it in less than $2.5 million," and he actually used ChatGPT 01 to figure it out.
Now, I'm not going to go through this; it's a bit technical for this video. Again, now all of the focus is back to the major tech companies: Anthropic, Meta, OpenAI, Microsoft, who have raised and spent billions and billions of dollars to build out AI infrastructure, only to have the rug pulled out from under them from this tiny Chinese company.
Listen to this: Deep Seek goes mega viral, and they can handle the demand on their two Chromebooks they have to use for inference. Meanwhile, Anthropic cannot handle the load of their paying customers with billions in funding. Do I get this right? That seems to be the sentiment across the board.
Here's another one: "I have made over 200,000 requests to the Deep Seek API in the last few hours, zero rate limiting, and the whole thing cost me like 50 cents. Bless the CCP; OpenAI could never."
Now here's the thing: we've been talking on this channel a lot about test time compute. A lot of the scaling that's happening in AI right now is not at pre-training—not that $5 million that it cost to actually build out the model—but since these models can now think, the more thinking they do, the better the results. That thinking is actually just compute; it's using compute.
So what's interesting about this is that even at test time, they're hitting the API 200,000 times, zero rate limiting, and extremely inexpensive. Unless they are just losing tons of money and have a bunch of GPUs that we don't know about, they've figured out something about efficiency that the US companies have not.
Alexander Wang follows up with a post: "Deep Seek is a wake-up call for America, but it doesn't change the strategy. The USA must out-innovate and race faster, as we have done in the entire history of AI, and tighten export controls on chips so that we can maintain future leads. Every major breakthrough in AI has been American, and continuing China's Deep Seek could represent the biggest threat to US equity markets, as the company seems to have built a groundbreaking AI model at an extremely low price and without having access to cutting-edge chips, calling into question the utility of the hundreds of billions of dollars worth of capex being poured into the industry."
So that's a huge, huge claim here. Now, it's one thing to be able to train the model originally at a very cheap and efficient price, but it's another thing to actually be able to run the inference at an extremely cheap and efficient price.
Now, I said earlier I don't believe it, and let me tell you why. There are two possibilities. Let's just assume they were able to figure out how to make this model extremely cheaply. We're going to be able to replicate that. Awesome, right? Everybody wins. That's the power of open source.
Now, at inference time, even if—let's go down the two paths—even if this model is able to run inference extremely cheaply, then we are getting to Javon's Paradox. As the cost per unit of any technology decreases, the total usage and the spend actually increases. We've talked about that on this channel.
That is because as the unit cost of any tech decreases, the amount of use cases that it can apply to in a positive ROI way increases dramatically. That's what we've seen with every tech throughout history.
Then let's think about the other path: they actually do have a bunch of GPUs powering it, and they're simply faking how efficient it is. Well, first of all, we're going to figure that out because we have AI companies throughout the world replicating Deep Seek R1 right now.
But let's just assume they're doing that. Then that's fine; all of this investment is still very valid. Even if it is really efficient, all of this huge investment by these AI companies in AI infrastructure is still valid because at the end of the day, whoever has the most compute will have the smartest model.
It doesn't matter if it costs $100 per token or a fraction of a penny per token. The more compute, the better. Whoever has the smartest AI will win.
And here's Gary Tan, the president of Y Combinator, basically saying the same thing. This is in reference to the chart that we just talked about, where it is a big threat to US equity markets. "Do people really believe this? If training models get cheaper, faster, and easier, the demand for inference, actual real-world use of AI, will grow and accelerate even faster, which assures the supply of compute will be used."
Yes, that is the way to think about it. I agree wholeheartedly, but not everybody agrees. Chamath Palihapitiya, billionaire investor, former early Facebook employee, and All-In podcast bestie, has the exact opposite to say, and he actually broke it down pretty well.
In his first point, he's saying in the 1% probability that the CCP has all of these chips that they shouldn't, we need to go investigate that. So that's point one. Next, he talks about training versus inference.
Now we are in the era of inference right now. We always knew this day would come, but it probably surprised many that it would be this weekend with a model this cheap. Many new products and experiences can now emerge, trying to win the hearts and minds of the global populace. Team USA needs to win here.
To that point, we may still want to export control AI training chips. We should probably view inference chips differently. We should want everyone around the world using our solutions over others.
Now I'm going to jump down to point four now because this is interesting, and the part that I really disagree with: there will be volatility in the stock market as capital markets absorb all of this information and reprice the values of the Magnificent Seven—that's the Magnificent Seven companies like Tesla and Meta and Microsoft.
So keep that in mind. Tesla is the least exposed; the rest are exposed as a direct function of the amount of capex they have publicly announced. Translating that, it basically means the company's stock might go down because of how much they have invested into AI infrastructure.
Because if everything's cheaper now, why did they spend so much? I do not agree with that at all. Again, let's look at Javon's Paradox: the cheaper the tech, the more it's going to be used. The more inference needs to be used, thus all of that supply of GPU is going to be used.
Nvidia is the most at risk for obvious reasons. That said, markets will love it if Meta, Microsoft, Google, etc., can win without having to spend $50 to $80 billion per year. The markets might love that, but that is not going to be the case.
Again, whoever has the smartest AI will win. Eventually, when we reach artificial superintelligence, it is literally a battle of who has the smartest AI. And what does that take? The most amount of inference or the most compute in general.
And what does that take? The most chips, the most spend into chips. If we find really efficient ways to use these chips, great, everybody wins. But ultimately, the cumulative number of chips is really what's going to matter, or compute.
He goes on to criticize the US, saying that we've been asleep. I'll just read this because it's an interesting take: "The innovation from China speaks to how asleep we've been for the past 15 years. We've been running towards the big money shiny object, spending programs, and have thrown hundreds of billions of dollars at a problem versus thinking through the problem more cleverly and using resource constraints as an enabler."
Now, a key concept to know is that if people are faced with bigger restrictions and bigger constraints, they tend to get more creative. They tend to be able to extract more efficiency out of less, and that's what he's really referring to here. I think the quote is, "Constraint is the mother of innovation," something like that.
But not everybody thinks it's just conspiracy theories. At the end of US tech companies, Yan Laon, the head of Meta's AI division, who is a big proponent of open source, has this to say: "To people who see the performance of Deep Seek and think China is surpassing the US in AI, you are reading this wrong. The correct reading is open source models are surpassing proprietary ones. Deep Seek has profited from open research and open source, e.g., PyTorch and LLaMA from Meta. They came up with new ideas and built them on top of other people's work because their work is published in open source. Everyone can profit from it. That is the power of open research and open source."
I could not agree more. This is a huge win for open source. This is going to allow many companies to start competing with the closed frontier models by having open-source state-of-the-art models.
This story is still unfolding. It has been crazy to watch the AI industry react to the news that essentially everything that they thought might actually be changing right now.
So what do you think? Do you think they have more GPUs than they're leading on? Do you think that they were able to basically come up with this amazing efficiency with just a handful of people as a side project? Did China just jump into the lead of AI, or is this just a great gift to the world because it is open-sourced?
I'm going to continue following up on this story. I am enthralled with it. I am absolutely fascinated by what's happening right now in the world, and I hope I broke it down for you well. If you enjoyed this video, please consider giving a like and subscribe, and I'll see you in the next one.