Transcription
Occasionally, some people around the world publish groundbreaking research papers—the kind of papers that dictate the entire direction the AI field will head towards.
With this Chinese AI company called Deep Seek, not only did they release a pivotal research paper that will probably change the course of AI development, but they also published their models free for everyone to download and use commercially.
With the models now being arguably the best in the world, you, me, and your friend Steve are all able to have a copy of this AI called Deep Seek R1, which literally goes head-to-head with OpenAI's R1, which they charge 200 bucks a month.
This completely shatters the price of intelligence. It's like having your own PhD student on your computer working for free instead of in your basement.
It's just that you probably need 16 RTX 390s to technically run the best model for free that matches OpenAI's performance. But don't worry, there are still smaller ones available for you that can fit into your computer.
A good rule of thumb to know which model size you can run is by taking the model size and multiplying it by 1.2. Then, if your GPU VRAM or your unified RAM is bigger than this number, it usually means you can run the entire model.
All you have to do to run it is to download AMA, open up a command prompt, and type in "ama run Deep Seek R1."
Wait a bit, and then, wow, it works in your terminal now! Congratulations, now you have a thinking AI model running for free on your computer.
But why is this thinking process so much better than anything that anyone has published and is now making so much buzz?
Well, for the entire time, OpenAI has had the secret sauce on the techniques of thinking models, aka test time compute, aka the Robberry. Ever since they announced O1, the performance increase has been incredible.
So the whole research world has been guessing and testing, and now with Deep Seek coming out on top, this is by no fluke either.
Deep Seek, which is founded by Lang Wong, in an interview back in 2023, shared some very interesting philosophies he has for running the company.
First, they look internally in China for young talents and young talents only. Because without experience, it also means that they don't have a preconceived bias, so everyone is willing to try a lot of new things.
When some very smart and adventurous researchers come together, brand new techniques are implemented and proposed, all of which are now implemented into Deep Seek R1.
Second, they are completely locked in on doing research, unlike a lot of companies that are now focusing on making one banger model and then making services to sell it.
Like now, Anthropic might just be stuck giving compute to Cursor. They are focused on not getting distracted by side missions; it's just a straight line for them towards HGI.
On top of that, they talk about how open sourcing for them is like giving back, which carries its own unique honor, and they believe they lose nothing by giving out weights and papers.
What's even cooler is that their CEO states they will not close source like OpenAI, as their CEO's main company is a quant firm that already provides enough funds and GPUs for Deep Seek.
So Deep Seek's way of business is definitely going to disrupt the current commercial AI landscape.
Just look at their API pricing: their latest model, Deep Seek R1, is sitting at 671 billion parameters, with 37 billion active parameters. It has literally the same performance as OpenAI's O1 but is 27 times cheaper.
On top of that, the R1 model is open-sourced under the MIT license, which means anyone can use it commercially or distribute it and do whatever they want with it.
This is a jailbreak prompt—don't quote me on it—but how did they achieve this?
Their focus on innovative techniques in their older research, like Deep Seek V2's MLA, Deep Seek ML E, and Deep Seek Math GRPO, has paid off with the release of Deep Seek V3 one month ago, which combines the three and is now capable of hitting state-of-the-art in the non-reasoning category, of course.
If you watch my last video, Deep Seek spent the same amount that OpenAI spent on running benchmarks for O3 to get Deep Seek V3, all while Deep Seek had no fancy hardware and only a tiny budget, which is absolutely crazy.
The reasoning model, Deep Seek R1, is then built on top of V3, with the main takeaway that shook the whole world being you just need to scale a reinforcement learning algorithm for test time compute.
For the rest of the complex reasoning behaviors, it will just all emerge itself if you scale the process large enough.
So you can just toss the complex search methods or process rewarding out the window. Coincidentally, the day R1 was released, Trump was inaugurated, with some of the biggest tech CEOs being there.
In the meantime, Deep Seek's CEO was at a high-profile meeting with the number two person in China. At the same time, a research called Kimi K1.5 was also published, which completely backs up the idea proposed in the R1 paper, with them also being the sponsor for today.
This is a great alternative if you like to have an AI to natively reason with images completely for free, which none others offer.
Unlike OpenAI O1, Kimi's K1.5 long thinking is capable of incorporating web search while reasoning with up to 1,000 sources and can process up to 50 documents at once.
This is thanks to their super large 400k context window and their method of training RL natively at 128k tokens, so it can do some very serious long thinking that no other models can do.
With K1.5 Vision that can reason natively, this means that you can attach images, like math questions, to help you explain the problems.
One of my favorite ways to test this out is to convert pictures of handwritten math formulas into LaTeX code.
So if you're obsessed with nicely formatted equations, you can handwrite it, upload the image, and it'll generate the math codes for you.
Thanks to the reasoning process, it even simplifies the answer even more here from the image.
With their approach of RL being very ambitious, just like Deep Seek, I think their paper is also worth checking out, as K1.5 is better than the old O1 and comparable to the new O1.
Definitely check out their web app with the link down in the description, as it is completely free, unlike a very overpriced alternative.
Thank you, Kimi, for sponsoring this video.
Anyways, in Deep Seek R1's initial experiment, their RL algorithm GRPO only has two main goals: be accurate and follow the thinking TX structure.
Contrary to standard practices, this RL is done before supervised fine-tuning, which is the training process to make LLMs generate in specific formats for applications like chatbots.
So they are doing RL on top of the base model that only knows how to do text completion.
But this simple setup actually helped to bring RL to its fullest potential because, throughout its training, the model with RL realizes that the longer the response it generates, the higher the accuracy is.
So it slowly reinforces itself to generate a longer response. During this process, there are spontaneous emergences of sophisticated behaviors like self-reflection, where the model re-evaluates its previous steps, and exploration, where the model would find alternative approaches to problem-solving.
As a result, all these behaviors emerged by having the RL simply interact with the environment freely.
I don't know if you remember, but it's like that one OpenAI research where AI game agents found re-bending exploits in the game to win, all through simple RL.
With only this method, they were already able to get performance near O1 level and even reached what they called an "aha moment," where the model suddenly learns to allocate more thinking time to a problem by re-evaluating its initial approach.
A lot of people are doubting this result, but I guess time will tell. Some researchers have already replicated its results, and as beautiful as it is, this RL freedom actually still has a few downsides.
It has poor markdown formatting since only the thinking tag was reinforced during training, and it has another problem of language mixing during its thinking process because only the accuracy of the end result was rewarded.
This makes it harder for people to understand the process, but the model slowly drifting to thinking in bilingual is indeed a really interesting observation.
So this initial model is called Deep Seek Zero, and to address these two problems, they had it generate a bunch of reasoning tokens, manually modifying them, and built this new Codar Chain of Thought data.
What I think is really clever about this Codar data is that it's based on a naturally emerged reasoning process, which should be optimal.
So by modifying on top of this, it ensures that the reasoning data is staying as optimal as it can get.
When a fresh instance of the Deep Seek V3-based model is fine-tuned with this Codar data, then during RL, it knows how to format the desired reasoning process that the researchers want.
Understand it now? Along with a new reward if the model uses consistent language, after the RL, the model would undergo some pretty standard supervised fine-tuning, which is just to add capabilities like writing, role-playing, and other general-purpose tasks.
And this is how Deep Seek R1 is born.
Research-wise, none of these steps are super surprising; it's just that they have blended everything together perfectly to mix such a good model that includes starting off from a very, very capable model.
Documenting all this is what makes this paper incredibly valuable.
So there is only one Deep Seek R1 model. Then what about the other smaller models we saw earlier on AMA?
Those are distilled models, distilled meaning that they had Deep Seek R1 generate samples that contain its reasoning process, then fine-tune smaller models like Llama 38 and Quin 2.53 2B on the generated samples to replicate R1's reasoning process without needing to undergo RL themselves.
What's surprising is that the reasoning process actually gave the models an insane boost.
It basically juiced out their performance using test time compute, with the distilled Quin 32B winning O1 mini across the board, other than coding.
And that's really big news because with Quo and this distilled version, both have the same base model.
We can judge it fairly on the reasoning quality. The reasoning process that is distilled from Deep Seek compared to what Quill has differs by 22.6% accuracy, which just shows how good the reasoning process Deep Seek has and how effective distilling the reasoning process is from a larger model.
This is why OpenAI was doing everything they could to hide their Chain of Thought, because from the look of this, there is no moat in this paradigm at all.
So if you want to run it locally, run either 14B or 32B, since there is a very high chance of their first reply being very accurate.
As for the official benchmark for the R1 model, it is basically on par with O1.
As for the third-party benchmarks, a lot of them are still not allowed for Life Bench, which has the most diverse benchmarks.
It sits at number two for AER Benchmark, which focuses on writing and editing codes. It sits at number two for artificial analysis, which measures model quality.
It sits at number two too, and for a model that's 27 times cheaper than the number one model, O1, I'm telling you, the cost-effectiveness is just mental.
On top of that, you can use your website or app completely for free, which is already catching up to ChatGPT on the App Store ranking.
Just got to watch out for the doom loops.
If you don't want to talk to a ChatGPT that's hosted in China, then there are alternatives like OpenRouter, Hyperbolic, or Gro that will host their own copy of the model that is not in China and doesn't store your data.
But of course, their API calls are not free.
Anyways, it's funny how America, the land of the free, doesn't have the best free-to-use model in the world.
Maybe OpenAI will outcompete them by spending another few hundred billion and charging us 200 a month.
If you enjoy cutting-edge research like I do, definitely check out my newsletter, where I cover the latest in the juus research weekly.
Through there, you wouldn't have to wait for my videos because I am always late to the party on YouTube.
Even so, I don't really have the ability to cover every paper that I find interesting on YouTube, so definitely check out my newsletter if you don't want to miss out on the latest technical developments the AI field has to offer.
Thank you for watching.
A big shout out to Andreula, Chelas, Chris, Leo, Dean, Mulim, Robert, Zasa, LS, Muk, Ben ER, Marcelo Ferraria, and many others that support me through Patreon or YouTube.
Follow me on Twitter if you haven't, and I'll see you all in the next one.