Transcription
There's a new AI model in town. Chinese AI company Deep Seek recently made waves when it announced R1, an open-source reasoning model that it claimed achieves comparable performance to OpenAI's GPT-4 at a fraction of the cost.
The announcement unleashed a wave of social media panic and stock market chaos, with Nvidia losing nearly $600 billion in market cap today alone. But for those following AI developments closely, Deep Seek's R1 didn't come out of nowhere. The company has been publishing its research and releasing its model weights for months, following a path similar to Meta's LLaMA model.
This is in contrast to other major AI labs like OpenAI, Google DeepMind, and Anthropic, which have closed weights and publish more limited technical reports. What's changed is that now the broader public is actually paying attention.
So let's decode what the real deal elements here are, where they come from, and why they matter. First of all, it is important to distinguish between two relevant models here: Deep Seek R1 and Deep Seek V3.
Deep Seek V3, which was actually released this past December, is a general-purpose base model that achieves comparable performance to other base models like OpenAI's GPT-4, Anthropic's Claude 3.5, and Google's Gemini 1.5. Deep Seek R1, released at the end of January, is a reasoning model built on top of Deep Seek V3.
In other words, Deep Seek V3 applied various algorithmic improvements to optimize its reasoning ability, resulting in R1, a model that has achieved comparable performance to OpenAI's GPT-4 and Google’s Flan 2.0 on certain complex reasoning benchmarks.
However, many of the algorithmic innovations responsible for R1's remarkable performance were actually discussed in the past December V3 paper or even before that in Deep Seek's V2 paper, which was published in May 2024, or the Deep Seek Math paper, which came out in February 2024.
V3 stitches together many of these innovations, which were designed primarily with compute and training efficiency in mind. One way Deep Seek optimized for efficiency and got more floating-point operations per second, or FLOPS, from the GPUs was by training V3 natively in 8-bit floating-point format rather than the usual 16-bit or 32-bit format.
This is not a new idea; many other labs are doing it too, but it was key for getting such massive memory savings without sacrificing performance. A crucial enhancement is their FP8 accumulation fix, which periodically merges calculations back into a higher precision FP32 accumulator to prevent small numerical errors from compounding the result.
This allows for far more efficient training across thousands of GPUs, cutting costs while maintaining model quality. But why does this efficiency matter? Given its hardware constraints and U.S. export controls on the sale of GPUs to China, Deep Seek needed to find a way to get more training and more bandwidth from their existing cluster of GPUs.
You see, at AI labs, these GPUs, which do number crunching and matrix multiplication to train these models, are actually sitting idle most of the time. At FP8, it is typical to only see around 35% model FLOPS utilization, or MFU, meaning GPUs are only being utilized at peak potential about a third of the time.
The rest of the time, these GPUs are waiting for data to be moved either between caches or other GPUs. This is Nvidia's key advantage. It is not just about GPUs; it is about an integrated solution they've been building for over a decade that includes the networking with InfiniBand, software with CUDA, and developer experience.
Essentially, Nvidia provides a deeply integrated system that lets AI researchers program GPU clusters as a distributed system, and closer to what Jensen describes as one giant GPU.
Another clever way Deep Seek makes the most out of their hardware is their particular implementation of a mixture of experts architecture. Deep Seek V3 has 671 billion model parameters, but only 37 billion are activated for a given token prediction.
By contrast, the largest and most capable LLaMA 3 model doesn't use a mixture of expert architecture, so it activates its full 405 billion for each token prediction. In other words, V3 activates 11 times fewer parameters for each forward pass, saving tons of computation.
Mixture of experts isn't a new concept, but it's been challenging to train models with this architecture efficiently. Deep Seek introduced novel techniques that stabilize performance and increase GPU utilization.
Additionally, to overcome key performance bottlenecks, V3 makes use of multi-head latent attention, or MLA, which Deep Seek first revealed with its V2 paper published in May 2024. MLA is a solution designed to tackle KV cache storage limitations, one of the biggest sources of RAM overhead in large models.
Instead of storing full key and value matrices, MLA manages to compress them down into a latent representation, reconstructing them only when needed. This helped the V2 model reduce its KV cache size by 93.3% and boosted its maximum generation throughput to 5.76 times.
Finally, unlike traditional models that predict only the next token, V3 makes use of multi-token prediction, or MTP. MTP enables V3 to anticipate multiple future tokens at each step. This densifies training signals, providing more feedback per step for better data efficiency and faster learning.
It also improves representation planning, allowing the model to pre-plan sequences for smoother, more coherent outputs during inference. MTP modules can be repurposed for speculative decoding, reducing sequential processing steps and significantly speeding up generation.
Taken all together, this makes V3 one of the most impressive base models on the market, and it's been out for some time now. However, the recent release of Deep Seek's R1 reasoning model is what really made waves.
Most LLMs can be improved by being prompted to think step by step, but what sets reasoning models apart is that they are specifically trained to break down hard problems and think about them for paragraphs at a time.
In September, OpenAI showed the power of this new approach with GPT-4, which achieved state-of-the-art results in math, coding, and science benchmarks. With R1, Deep Seek took a similar approach and published the secret sauce.
OpenAI and Deep Seek achieve their impressive results through reinforcement learning, a technique to shape an LLM's behavior based on feedback and reward signals. Modern LLMs use some variation of reinforcement learning with human feedback, aka RLHF, or reinforcement learning from AI feedback, aka RLAIF, to improve their models' usefulness and alignment.
But reasoning models apply RL specifically towards the task of thinking step by step through complex problems. So how did Deep Seek apply RL to get a reasoning model?
At a high level, they assembled a bunch of problems with verifiable outputs, especially in math and coding problems, and then designed a training pipeline to get the model to think for a bit and output the correct answers.
But they don't give the model any external examples of how to think, whether from humans or AI. The grading process was extremely simple. Rather than using a complex AI to give the model fine-grain feedback, Deep Seek uses simple rules to evaluate the model's final output on accuracy and formatting.
They use these output scores to update their model through a novel technique they published in February 2024 called Group Relative Policy Optimization, or GRPO. Remarkably, with this process alone, Deep Seek saw reasoning emerge over thousands of RL steps.
The model learned skills like extended chain of thought and even experienced an "aha" moment where it recognized its own mistakes and backtracked to correct its reasoning. This model, R1, is one of the first large models to achieve top-tier results purely through reinforcement learning.
Pure RL has long been a subject of investigation in Western research labs, such as DeepMind's AlphaGo, which simulated thousands of random games of self-play to beat Lee Sedol, the world's top Go player, in 2016.
In 2019, OpenAI achieved notable success using reinforcement learning to train a robotic hand to solve a Rubik's Cube and beat a top human team in competitive Dota 2. But unconstrained by human examples, R1's thinking steps suffer from poor readability, switching between English and Chinese at random.
So Deep Seek introduced a cold start phase, fine-tuning on structured reasoning examples before RL to get R1. This eliminated the language mixing issues and made outputs far more comprehensible.
The results are impressive. R1 achieves comparable performance to GPT-4 on certain math and coding benchmarks. But the pace of innovation is speeding up. Just two weeks after R1 was released, OpenAI released GPT-4.5, which outperforms R1 and GPT-4 on key benchmarks.
So if R1 didn't actually come out of nowhere, what explains the hype cycle? One explanation is the sheer accessibility of Deep Seek's model. R1 is freely accessible through their website and app, and it is free to download, run locally, and customize.
Also, because of all the efficiency improvements, it offers near state-of-the-art performance at a fraction of the price of other reasoning models. Another explanation is that a lot of the hype cycle didn't actually have to do with the specific algorithmic improvements that we described, but with misconceptions around V3's alleged $5.5 million in training cost.
There's some important fine print here. The $5.5 million figure refers only to the cost of the final training run for V3. It doesn't include any of the training costs of R1 or the associated R&D or hardware operating expenses, which are presumably in the hundreds of millions.
Given the extreme algorithmic optimizations, that $5.5 million training run number actually seems perfectly possible. It is worth noting that this work is reproducible. A UC Berkeley lab recently applied R1's key techniques to produce complex reasoning in a smaller model for just $30.
What Deep Seek really proves is that there is still room for new players on the frontier. In particular, there's room for rebuilding the stack for optimizing GPU workloads, improving software at the inference layer, tooling, and developing AI-generated kernels.
Ultimately, this is fantastic news for AI applications in consumer or B2B since it means the cost of intelligence keeps going down.
So the big takeaway here: this is the best possible time to be building a startup. The deadline to apply for the first YC Spring batch is February 11th. If you're accepted, you'll receive $500,000 in investment plus access to the best startup community in the world.
So apply now and come build the future with us.