Transcription
So, Grock 4 is set to be one of the most powerful chatbots in the world, and I think that this is quite likely to be one of the most underestimated launches in terms of AI announcements this year. Elon Musk has already shown us that he's extremely capable when it comes to innovating and competing in very fierce technology markets. And it's no different when it comes to AI. And this company, X.AI, are about to unveil Grock 4 sometime this week. And I don't think you guys understand just how much is at stake for the company and why this is probably going to be one of the most incredible launches so far.
So, in this video, I'm going to dive into absolutely everything you need to know because there's actually a lot to unpack. So, one of the recent tweets that we got from Elon, this one where he says, "We are grinding on Grock all night with the XAI team. Good progress. It will be called Gro 4 and it will release just after July the 4th." Now, at the time of recording this video, it is currently July the 7th, and we haven't had major updates since around the time of this tweet, but there are a few key details that I think people need to understand. And he also adds right here, which is, I think, something that a lot of people did miss because although everyone was expecting the model to be released on July the 4th, he did clearly state that they do need one more big run for a specialized coding model. So, I'm not sure entirely if this is something that will take another 2 weeks or something that will just take a few days, but for another entire run, that usually isn't a small process.
Now, I don't want to say they've completely left us in the dark because there have been a few statements and even a few leaks that show us the Gro 4 is probably going to blow our socks off. Now, this picture actually went pretty viral on Twitter, and I think it's rather important to state that this picture isn't true. I know AI is here, and these pictures do look super, super realistic, but this is just to underscore the amount of dedication that the team has. They aren't actually camping at the offices, but I do know that the team is currently working, perhaps maybe 7 days a week and working really, really long hours to ensure that this entire thing does come together. So, when you look at this picture, I know it does seem a little crazy. That's because it's quite likely AI generated, but the message is still the same. The engineers are working relentlessly to ensure that the company catches up. And we have to understand that XAI, I think they started around, you know, maybe 8 to 12 months after the AI hype was there. And to be able to catch up and surpass and still gain a decent market share is honestly quite remarkable considering the significant speed of the AI industry.
And speaking of speed and efficiency in the AI world, it's not just about what these teams are building, but also about the tools that let us harness that same power. And if you want to bring that same power of AI-driven efficiency into your own work, especially when it comes to communicating your ideas, this is where today's sponsor comes in. That's why I need to show you guys Gamma AI. Think of Gamma AI as essentially ChatGPT, Notion, and PowerPoint teamed up all in one incredible AI tool. It's super smart. You simply type in your ideas or key points, and Gamma instantly transforms them into sleek, visually stunning presentations. You know, you don't have to worry about tweaking design, struggling with all that nonsense. It's literally AI-powered slides that look amazing every single time. If you're sharing AI insights, pitching your next big idea, or presenting complex data, Gamma just streamlines the entire process so you can focus on the content that matters. And what's better is that you can start using Gamma AI for free today. And if you love it as much as I do, upgrading is seamless. Check out the link below and give your next presentation an AI-powered upgrade.
Now, one of the things I really wanted to talk about is, of course, not just entire speculation. So, when we look at things here, there is currently this floating around. And this one is 50/50 on whether or not it is real or not. But the reason I'm leading towards more so this leaked benchmark, which I'll explain in just a second, being a little bit more real is the fact that this benchmark isn't impossible to, you know, get better at. But it wouldn't be the first time that XAI has managed to shatter benchmarks when releasing a recent model. If you remember the Grock 3 release, that one really did take the internet by storm. So, with Grock 4 0629 here, what we can truly see is that we can see some key, key figures that I'm about to point out to you. And so I will actually highlight this figure right here, which is arguably the most important one. So, the HLE benchmark is probably going to shock people because this one is one of the hardest benchmarks. And I want you guys to pay attention to the value here. You can see it says HLE 45 and HLE 33. So, this is currently referring to the Humanities Last Exam benchmark. And essentially what that means is that this is a benchmark that is one that you really can't study for. And we can see that the Gro, the standard edition, appears to achieve a 35 without any test time reasoning. But when we have test time compute, with, you know, additional time through think, we can see that it achieves around a score of 45. Now, I want to show you guys just how crazy that is in terms of a jump. Remember that a standard model for Grock 4, potentially in terms of, you know, being released, achieves a 35, and with test time compute achieves a 45. Now, look at this. Okay. If we go over to the models currently, okay, the current state-of-the-art systems do not achieve anywhere near this kind of performance. So, we can see the 03 achieves 20% accuracy. Gemini 2.5 Pro achieves 21.6. But if those leaked figures are true, which I think is particularly plausible considering XAI has some incredibly talented engineers, we could be looking at a major leap on this benchmark.
So, this ultra-challenging benchmark, I think, is super important because it really is different because it spans over 100 subjects including maths, natural sciences, humanities, law, and medicine. And it's literally expert-level questions, and it's crowdsourced from nearly a thousand subject experts across 500 institutions from 50 countries. So, it's also multimodal, including text-based and image-based questions, and it just covers an incredibly broad range of knowledge from philosophy to advanced scientific reasoning. So, I think this kind of result is, you know, one of the benchmarks that people are looking at because most benchmarks currently like the GPQA and the MMLU math benchmarks, they've, you know, kind of been saturated due to the amount of data that's been scraped for these LLMs. So, it's pretty crazy because I think that if Grock is able to make such a leap, it shows that they're not just following in the footsteps of everyone in the AI industry, but it shows that they're actually innovating at a level that we haven't seen. So, right here we can see that, you know, on the Humanities Last Exam webpage, they do state that given the rapid pace of AI development, it is plausible that models could exceed 50% accuracy on the HLE by the end of 2025. And that high accuracy on the HLE would demonstrate expert-level performance on close-ended verifiable questions and cutting-edge scientific knowledge, but it wouldn't suggest autonomous research capabilities or AGI. So, whilst yes, it would, you know, suggest that those models are cutting-edge and incredible, it wouldn't suggest AGI just yet, but the benchmark is super, super incredible.
Now, another thing that I found about this entire Grock 4 debacle, which is why I'm so excited, is that apparently Grock 4 are releasing a dedicated coding model. So, once again, from this leaked screenshot, what we can see here is that the Grock 4 specialized coding model is apparently achieving a 72% on the S.Bench, and with test time compute, we can see that that goes up to 75%. So, once again, this is really, really impressive, provided it is true, because why don't we take a look at Anthropic's recent release, take a look at Claude 4, that benchmark when it was released, and Claude 4 Opus is the most powerful model yet and is the best coding model in the world, leading on the SW Bench at 72%, we can see that 72.5% is actually just shy of 75%, which is what Grock 4 is claiming, so this would mean that Grock 4 has basically caught up to state-of-the-art in terms of coding, not only in coding but also in other areas too. So, for me, that would be tremendously incredible considering that, you know, other companies are struggling to really, really perform at the level of Claude 4 Opus in terms of that richness, in terms of the coding ability. I mean, I know so many people that use that model for coding. Most people do actually use Gemini 2.5 Pro now, but it is super, super interesting to see if that benchmark ceiling has sort of been broken by Grock 4 and if people will actually start using the model on a day-to-day basis. I know most people that use Grock just kind of use it on Twitter, but I don't really see many use cases on a day-to-day basis of screenshots of people actually using the model.
Now, if we take a look at the other benchmarks, this is another screenshot floating around. It shows the Humanities Last Exam where these two candles, the yellow and orange ones. I do apologize for the color scheme here. It's probably not the best to highlight the differences, but we can see that this is the Grock area. Once again, on the GPQA, there's not a crazy, crazy difference. Like I said, the benchmarks have been largely saturated. The math benchmark here, once again, looking super, super great. And on the S. S. S. S. S. S. S. S. S. S. Bench. Once again, we can see that Grock 4 seems to be outperforming across the board. Now, maybe these benchmarks are completely false, but I do remember last time when Grock 3 was released, it did absolutely incredible. So, we have to ask ourselves, will Grock 4 once again present such a measurable jump due to XAI's continued innovations? I personally think that it's quite likely that they will. I do think that they are working incredibly hard on the model to make sure that it exceeds expectations because a lot of people, I don't want to say they've written them off, but they haven't particularly given them attention and, you know, put them in a position where they are poised to win the AGI race or the AI race, but it is kind of interesting to see where things are heading with regards to this kind of development. I do think that, of course, things could change over time, but so far it's super, super interesting.
So, I do want to add just this because this is a little bit of a caveat to Grock, and I think we have to understand that, you know, compared to other companies, I think Grock is a little bit more centralized in terms of how things are controlled. Elon Musk is, of course, the CEO of the company, and I think this issue needs to be solved if Elon Musk actually wants people to use Grock on a day-to-day basis. So, if you haven't heard about this before, I think you need to hear about this because the problem with Grock and the reason why, you know, people like myself and many others aren't using the model is that, you know, Elon Musk simply just forces the AI to regurgitate his own opinions. He sits on this high horse where he says that this model is going to be the only one that is true and will seek truth and unity and all of this other kind of, you know, righteous statements. But realistically, time and time again, we see that Grock often goes off track when discussing random subjects to sort of, you know, spout its own views on certain things. And it's clearly an issue of the system prompt where Elon Musk is instructing Grock to talk about, you know, whatever issues he finds to be wrong or correct in the world to either push them or to negate certain things. So, I mean, it's it's incredible that this is even allowed, that this is even happening. But it is also another reason that I do think that people have to pay attention with these chatbots that they're using because it's not just me saying this. Currently on Reddit, if you go on Reddit, there are like, you know, five to 10 posts showing that, you know, Grock is simply spouting so much information that literally just supports Elon Musk's opinion. So, I think it's a little bit dangerous that that is occurring, and it's not really something we would want, especially when a lot of people do trust these models as a second opinion. So, let me know what you guys think about this. Are you excited for Grock 4? Are you guys disappointed because it hasn't already been released? I would love to know your opinions.