Transcription
You want to hear something crazy? GPT-40 got some massive updates. In fact, it is not only the best at image generation now, but it's also the number one non-reasoning coder on the market. And this is verified by independent benchmarks. Look at this artificial analysis: today's GPT-40 update is actually big. It leapfrogs Claude 3.7 Sonnet non-reasoning and Gemini 2.0 Flash in our intelligence index and is now the leading non-reasoning model for coding. Here's the intelligence index—this is not the coding index, this is the intelligence. It goes from 41 back in November 2024—that version of the model—leapfrogs all the way up to 50 on their score, just behind DeepSeek V3, the recent model that just came out. It's kind of nuts.
Why are they putting so much time and effort into an older model like 40? Well, it turns out there's actually a very explicit reason. Anybody who thought that the DeepSeek R1 phenomenon meant that everybody was going to start spending less on compute, boy, you really should have listened to the experts. With Javon's paradox, as things get cheaper, we want more of it, and that's what we're seeing. They can't even get enough GPUs to fine-tune and improve 4.5. We're not talking about some small startup; this is OpenAI partnered with Microsoft, a multi-billion dollar company that has raised billions and billions of dollars. They can't even get enough of these chips.
But those aren't all the improvements. GPT-40 got better at following detailed instructions, especially prompts containing multiple requests; improved capability to tackle complex technical encoding problems; improved intuition and creativity; and fewer emojis. The updated GPT-40 is available now to all paid users; free users will see it over the next few weeks. Now, here's the thing: they're already having to put in place rate limits on the image generation capability of 40 because it far exceeded even their expectations, and I'm already seeing it's super slow. I started noticing this a few weeks ago. GPT-40 for normal queries is almost unusably slow at this point, and they really need to speed that up because one of the factors that I look for in my go-to model is speed.
Next, the big news this week was also Gemini 2.5 Pro coming out of the gates swinging. It is incredible at coding. This is a full thinking model and incredibly fast. I made a full video about it already, so I'm not going to dive too deep into it, but it is the best coding model I have ever used, and again, I want to re-emphasize: super fast. And I think people underappreciate how important speed is, especially with agentic use cases and just as importantly, coding use cases. And now we have the best new coding model in the world, and vibe coding is really all I'm doing in my free time right now, so of course I was excited to see our next story. Gemini 2.5 is now available in Windsurf, and not only that, it's also available in Cursor. So I'm going to be testing it thoroughly. The great thing about 2.5 Pro is that it has a million token context window, which is about 10 times what Claude 3.7 has, so I really want to see how that affects its ability to understand the codebase as a whole because I certainly have less than a million tokens of code on my projects that I'm working on right now, so I'm going to be testing it out. I will report back.
Next, this really was an incredible week for new models. There is a new version of DeepSeek V3; it came out just a few days ago. They really made no splash about the announcement. It is a new checkpoint in the V3 series, so it's not a completely new model, but it really excels at coding and math and logic, and it is fast and open source, and you can download it, although it is a massive model, so you may have trouble running it locally. So here are the scores: DeepSeek V3 in the striped to dark blue versus the previous DeepSeek V3, Quen Max, GPT 4.5, and Claude Sonnet 3.7. Now it would be interesting to see this versus 40 new, but look at that again: DeepSeek V3, a non-thinking model compared to all of these other frontier models, and GPT 4.5 and Claude 3.7 are both closed source, and it performs extremely well, especially at math. Look at that Amy 2024 score, absolutely dominating the rest of the models in this comparison, and it is open weights, and they also switched to an MIT license, which is very permissive. So go download it and have fun.
The ARC Prize company has now released ARC AGI 2. This is their new benchmark to test the AGIs of models, and so we can see the scores right here. 03 Low currently has the highest score, and on ARC AGI 1 it has a 75.7% score; for ARC AGI 2, only 4%. And look at this: actually $200 per task for 03 Low. And look at the number one: the human panel for ARC AGI 1, they got 98%, so not even perfect; and then ARC AGI 2, they got 100%. And I love this: the fact that humans can score a perfect score on this test, but the best of the best models score so low—that to me is the perfect benchmark for testing AGI. And the cost per task for humans: $17. Now look down here: 01 High, $445, and down from there. And if you're not familiar with the ARC Prize benchmarks, they are essentially tasks that require you to take an understanding of one thing and extrapolate it out to understand other things. So here's an example: we have an input here, and all we do is just look at it. We have a 30x30 grid; we obviously have kind of this delineator right here; we have a bunch of colors and squares here; and then we have a bunch of kind of gray blobs in the middle. And so the point is you're trying to look for patterns between the two example inputs and outputs and then recreate it over here. And so if you want to try it out, you can; I'll drop a link down below. And so the point is this is supposed to be easy for humans but really hard for AI, and once again, there's a million-dollar prize. So congratulations to ARC Prize for the new benchmark.
Next, Zapier announced their own MCP—that's basically like getting 10,000 tools all at the same time. For MCP, all you got to do is sign up, configure the apps that you want, and then they'll give you the MCP server URL. I'm personally a big fan of Zapier; I've been using them for years and years. It's great for automation, and the fact that now I can connect my agents and my AI directly to it—just fantastic. But that's not all. OpenAI just adopted MCP as well. It's quickly becoming clear that MCP is the standard, and now part of their agents API, you can now use MCP to give your agents tools. And that's not all; MCP has now been adopted by Microsoft, introducing Model Context Protocol in Copilot Studio. So it seems wherever your agents live, that's where you're going to be able to use MCP. And this is a big win for Anthropic, who set the standard, even though this is an industry-wide standard; whoever creates the standard definitely gets to put their thumb on the scale.
And this was definitely the week of text-to-image generation. Not only did we have 40 absolutely dominating the headlines with text-to-image, but Revai came out with their text-to-image, and it looks really good. Look at all of these: the text is all accurate; all of the different styles is great. Look at this realistic piece of steak on a salt block; nature; more artistic. And according to the artificial analysis rankings, Reev Image 1.0 ranks highly in quality based on 100,000 user votes, and half moon is Reeve. And like I said, this was the week of text-to-image. Aadio launched 3.0, and it is also looking phenomenal. Now, of course, Audiogram says they are scoring the highest in the ELO rating, but you could do things like remix, upscale, style preference, and so on, so highly controllable, which is great. And just look at these images: they're beautiful, they're hyperrealistic, and you get lots of control over them. So although 40 seemingly got a lot of the press love this week, there are multiple other fantastic image models that just got released this week, so check them out.
And now back to OpenAI. Turns out they're printing money now. Although they're still losing money, they are making a ton. According to CNBC, OpenAI expects revenue will triple to 12.7 billion this year; sources say. AI is not slowing down. Occasionally I get the question: do you think this is a fad? Do you think all of this spend is not going to result in value? Not a chance. I am obviously extremely biased; I'm dedicating my life to this, but for the amount of value—just personally, a data point of one—that I get out of all of these AI tools is tremendous. And my team uses these tools every day; my family uses these tools every day, all day. They've been exposed to them because I've told them about them; I've told them how to use them. I actually think the problem is an education problem; people just don't know what's possible, and that's the problem I'm trying to solve. I want everybody in the world to understand how to use these AI tools and how to get the most out of them. That is my mission.
So a few other notes about OpenAI. So earlier this week, OpenAI announced some key changes in the C-suite. Sam Altman will shift his focus away from day-to-day operations and focus more on research and product, which is very interesting, while operating chief Brad Lightcap's role will expand to oversee business and day-to-day operations. SoftBank just last month was set to invest $40 billion in OpenAI at a $260 billion valuation, making it one of the most valuable private companies ever. Next, Quen released QVQ Max Think with Evidence. This is a visual reasoning model, and it's open source. These Chinese companies open-sourcing everything are really making these US companies look bad, and you know what? I'm all for it. I really appreciate open-sourcing it because once you can get it yourself, you can make the AI any way you want. This model can not only understand the content and images and videos but also analyze and reason with this information to provide solutions from math problems to everyday questions, from programming code to artistic creation. QVQ Max has demonstrated impressive capabilities. So here's an example of what it can do: let's upload these two images, and the scenes depicted in these two pictures—what is the relationship? So it is a thinking model with vision capability, so you can see it's thinking right now, and there you go; it gives you an answer. Now the only problem is Quen is not open to US users; you have to have a Chinese-based phone number to use it. But that's okay; we're going to get this model on inference providers that allow for US users, and you can also download it yourself. Now this is also a relatively large model, so yeah, you're probably going to struggle running it. Maybe we're going to get quantized versions of it, but we want to run the full thing, and hopefully some of the awesome providers will do it soon. So that's all the news for this week. If you enjoyed this video, please consider giving a like and subscribe.