Transcription
Okay, so Moonshot AI have dropped their latest flagship model, and this is the Kimmy K2.5 model. And in this video, I'm going to show you not only the model, but also some really cool stuff that they've actually got with Agent Swarm.
So, the Kimmy team reached out to me and gave me early access to actually try some of this out. And in this video, I want to show you what is actually cool with this model, way beyond just benchmarks and things like that.
Okay, so first off, if we look at this, we'll actually see that this is not just one model. They've got multiple models in here. They've got the K2.5 instant. It's like the flash model, I guess you would say. They've got a thinking model. They've got an agent model or their agentic model, which is working for things like slides, websites. And then they've got the brand new agent swarm model. And this is what I'm going to go into a deep dive with later on in the video because I think that big story here is not just another great model, but this whole idea of having up to a hundred self-directed sub-agents that can be working on your task at the same time.
Okay, so if we look at the general information about the release here on their blog, we can see that they've got now a native multimodal app. This has been trained on 15 trillion tokens of both sort of text, but also visual images and videos and things like that. And specifically, it seems that the way that the Moonshot team has actually focused on perhaps the RL element of training is to get this model to be very good at certain specific tasks. This is things like vision coding where it can basically do video-to-code generation, visual debugging, but also training the model to have the ability to realize that it can actually do different calls to itself or different instances of itself where those are going to be agentic.
Now if we look at some of the benchmarks here, we can see that agentic-wise, they seem to be doing better than both the OpenAI models, but also Claude and Gemini in here. Now, that doesn't seem to be the case for all of the benchmarks. If we look at some of the coding the three-bench verified here, we can see that OpenAI and Anthropic still seem to be edging them out, even though Kimmy K2.5 is actually doing better on things that are multilingual.
So they're promoting this model to be very good at a number of specific tasks. Now, the one for me that's by far the most interesting is the agent swarm, which I'll get to in a little bit. But the first one is coding with vision. So the idea here is that they claim that this is the strongest open-source model to date for coding, and in particular for doing things like front-end development. And they give a bunch of these examples in here of the model being able to do different kinds of tasks by both being able to generate code, but then also being able to call image gen models and stuff like that as well.
So I guess the key thing here is this whole idea of being able to reason over images and videos. So obviously, this is a thing that originally was introduced by Nano Banana. The whole idea of not just having image generation, but being able to look at the images and reason sort of what's going on. And you can see in the example that they've got here, they take a sort of pre-made website and then they get Kimmy to just watch a video of that and be able to reproduce a lot of what's actually going on in there.
On top of this, another product that they've actually gone and launched here is this whole sort of Kimmy Code. So obviously, this looks like it's basically the Kimmy CLI. It's their answer to something like Claude Code. But I get the feeling that even the things in the model here are going to be useful for things like Open Code. So if you saw the previous video I did about trying to use Claude Code with open models, one of the big feedbacks that I got there was just how much better Open Code is for working with open models. And I think we may actually see some really interesting takes on this kind of thing as sort of open harnesses or open sort of coding tools like Open Code, like RUE, like Client to actually start to take advantage of the coding abilities of this model.
And that brings me to what I think is like the main interesting thing here, and this is the whole idea of Kimmy 2.5 Agent Swarm. So they mentioned many times the whole idea of not just trying to scale up to make a bigger model, but scaling out and using lots of sub-agents that have multiple instances of this model to be able to self-direct, coordinate with each other, and execute different parts of this.
Now, I'm going to go through a live demo of that in a second, but just to sort of look at this, this is where they're basically training on what they're calling Parallel Agent RL, or PAL. This basically allows Kimmy to self-direct an agent swarm of up to 100 sub-agents at a time, executing parallel workflows where they can work across up to 500 coordinated steps. And each of those agents can have their own set of tools to be able to run things in parallel.
So we'll see in the UI in a bit, but the idea here is that you've got this sort of master trainable orchestrator agent, and its goal is then to farm things out where different tasks get decomposed into these parallelizable sub-tasks, and each of these can be run on an instantiated agent. So they've built a whole sort of reward system of how they actually train this orchestrator agent and how it sort of monitors what's going on with the different agents, and I guess is able to guide them going through this.
But here you can see a diagram of how this actually works. Conceptually, this is not that different than things we've seen before with like agentic one, like deep agents, like things like that. The key thing here is actually just the scale of being able to spin up so many agents at a shot if it deems that these are needed. So each of these agents basically gets their own set of tools, and that could be a whole bunch of different things like search, like Python, like web browsers and stuff like that. It also seems like these sub-agents have a whole set of either sort of system prompts or customized instructions to actually be able to make them work better as they go through it.
And we can see looking at this that obviously this orchestrator agent gets better the longer that they train it. But we can also see as it spins up more agents, it's going to basically have a lot more steps for those agents to actually do. Now, in my testing, it does seem that it kind of decides itself how many agents you actually need for the task that you need and what those agents should be.
When we look at their benchmarks basically of taking the agent swarm idea here and comparing it to the Kimmy K2.5 model alone or the Claude Opus 4.5, we can see this agent swarm idea is getting a lot of traction and is being able to scale up to do a lot more steps as it goes through this. Obviously, the other good thing, and I've noticed this in my testing, is that when you've got so many sort of sub-agents working on something at once, it's much faster than things like the normal deep research from something like OpenAI or Gemini, etc.
So, I think let's just jump into me playing with the agent swarm, and you can actually get a sense of what it can do.
All right. So, if we come into the Kimmy interface in here, you'll see that this is very similar to chat. If you haven't come in here, how to play around with this, you can get a free account and try things out. What I'm going to be showing you today, I think, is going to be behind a paid account when they release it. But you can see in here that we've got the different 2.5 models. We've got the sort of instant one. Again, that's like the flash. It's much faster. It's just really no thinking on there. We've got a thinking model. We've got an agent model. And then we've got this agent swarm. Right? So this is for long writing, batch tasks, a whole bunch of different stuff. And then of course, they've got their old models in there. And if you're doing something like a deep research, you can just come in here, click that, kick it off. If you want to do something with sheets or slides, it's actually quite good at making slides.
Next in here is this agent swarm. So I'm going to give it a task. So this was one of the ideas that was proposed in the Let's Verify Step-by-Step paper a long time ago, which led to the 01 model. Most of the main authors of that paper then left OpenAI, and some went to Anthropic. Some were at Thinking Machines. Idiot Saskcover has gone on to start his own startup, and that paper was interesting not just about what they proposed in there, but also what was left out, and this is all to do with the verification systems. So recently at a number of conferences, people have started proposing a bunch of these verification systems. How do you know that each section of chain of thought is on track for a good answer? And this has been one of the holy grails in this kind of thing.
So I'm going to basically run this out. So you'll see that I will kick it off, and very quickly it's going to go into this orchestrator mode where we can see that, okay, it's doing the orchestration. It's basically thinking to itself what it's going to do. And when it's doing that, it's not only thinking about that, it's also working out, okay, how many sub-agents does it need?
Now you'll see that there's some great graphics that sort of pop down here as it goes through different sub-agents. It looks like in this one, it's decided it just needs four sub-agents. Now, I could have told it I want a lot more. I have tried telling it I want 100 agents, though, and it doesn't seem to give me 100, but it certainly gets this going. And this sort of shows you, you can see where it's generating now in parallel. So we can see here that if I click on these, I can see a little bit about the different agents. In this case, we've got a number of different agents going on. I can see that this one is basically looking for the paper and looking for things related to citation. We can see ones that are final papers by John Schulman and his team's paper. We've got uh GPO stuff searches going on there. And then we've got fine-grain verification papers in there.
And you can see agent one has gone off and found a bit of stuff. But let's say we want to drop in on agent three here, where I'm clicking it. We can see what agent three is doing because this is going on in parallel. Now, if you see down the bottom here, this is earlier on, you'll actually see them decompose the task down to different parts. So while the orchestrator agent will do the sort of big thing, it will then pass it off to each one, and each of them will decompose it and start to go through different things. So they have a whole bunch of different modes from things like explore, generate, decompose, a whole bunch of these things, reflect mode. And you'll see that they're going through and getting different things. So this could be, in this case, it looks like they're looking at Google Scholar to find this agent. Our a third agent is looking for papers on chain of thought rollouts by John Schulman and his team. So John Schulman was one of the seminal researchers. He's one of the guys that created PPO for reinforcement learning quite a while back. He was at OpenAI. He left after the sort of fallout. He went to Anthropic for a while, and now he's at Thinking Machines.
And you can see how things's going through, and we've got it going through each of these. And I can change. If I click on one of these, I will actually get to see the thinking patterns of each of these. I will get to see what's going on with each of them. So, it looks like they're actually starting to get papers. They're actually starting to do, you know, a whole bunch of different tasks and stuff going through this.
Now, if we come back over here, we can actually see how far along it's actually going. Now, these tasks can run for quite a while. So, in a moment, I'll pause this, and we'll look in at where it comes in. But, let me show you some of the other ones that I've done while we're waiting on this one.
Okay, so this is one that's finished where I basically just asked it, "Can you prepare me a report about top AI ML startup ideas for making reinforcement learning environments or RL environments?" You can see this one kicked off a number of sub-agents to do stuff. It researched different things around the reinforcement learning stuff and seeing, okay, what were people actually making environments for? It then generated. It then had one of the agents that is an idea generator. You can see here, this Zach agent is an idea generator. There's one in there that looks very much like Sam Altman, which is the PDF generator, which I thought was quite funny. But we'll see in here that we can see like different takes on the same sort of task where it decomposes this down, and then it will start to put these together before it comes up with a final response out, and then will actually give us the markdown files for each of these.
In this case, another one that I ran, another one that I ran, just showing you here, is where I asked it to spin up at least 20 agents to look into giving me a 50k word report on AGI timelines. And you can see it didn't get to 20 agents, but it got to 12 agents in there and then reused different agents as it went along, and then spun up some more later on. Interestingly, you see that what it does is it uses agents to do different tasks. Then it will get to a point where it returns that back to the orchestrator agent. And then that can then decide, "Oh, okay, we need to now spin up some agents to put this report together." You can see this report is too large for one agent. Let me break it into parts and use multiple agents. So in this thing, we probably did end up with close on 20 agents being used and reused. And we get a pretty extensive report back, and going through it, it breaks down, you know, looking at a bunch of researchers, what chance by 2027, it seems to say that we've got a 15% chance, 35 chance between 28 and 32. And then, you know, goes on.
Okay. So, you can see here we've got one one agent that's complete, right? Where it's, and this is the first one to find and analyze that. Let's Verify Step-by-Step paper. It's found that paper. No problems. It's gone through. Interestingly, it's very interesting just to see what they name each of the sections. So, one of them is "weigh the evidence," and then "trace things" for looking for citations, "notice things," "reflect on things," "infer things." I'm not sure. I'm sure someone will actually get the system prompt for this, and it'll be super interesting to re-look at how they're actually doing it. We can see as it's going along for doing this. And then here we're now on the "scrutinize" part, where it's starting to look at different kinds of verification papers and a number of different things. So "synthesize" is one of the ones where it'll start to put things together, and then we'll see things like, in this case, we're back to "deconstruct" and shows what it's doing as it's going through this. And sure enough, looking at some of the things that it's actually printing out here, these are some of the papers that actually relate to this and different approaches that people have tried for this. So we could come back once we've got this final report and get it to go into more depth of what approaches show the most promise, etc., in here. So it really is pretty amazing that it can go off and do all of these things in parallel. You can see we've got, in this case, looks like we've got five agents now going on in parallel, and then it will get to the end where it will just put them all together and start to show us like how that actually comes together.
Okay, I paused it for a while there. You can see now it's starting to put together the final sort of markdown document here, and we can see that, okay, it's got some of the key things, and it talks about different techniques, and it will then group some of these things into different techniques. So this one is step-level verification. Then next one we've got like token-level verification, and it's gone and broken down the papers and their sort of technical approach. So the fact that it's been able to do all of this in a reasonably not too long time, it's really amazing for this kind of task. This is definitely better quality that we're getting out of this compared to the OpenAI deep research, the Gemini deep research, just because it's so much more thorough in the way that it spins up lots of agents to do this. And I'm sure we will see a lot of other companies end up doing the same sort of thing. And yes, you certainly could have tried to create your own deep agent that that does the same sort of thing as this.
Overall though, this has put together a report, and we can see like it's through its planning and stuff what it's actually done, and we can see some of them say that they're completed as we go through right through to this sort of final agent we've got that's still perhaps checking some of the other agents' works as it compiles the final result. So I do think the idea of these swarms of agents is a big game-changer. We've seen people try to do it with their own sort of thing, but generally we haven't seen something that's on this level of where it can just do it in parallel. It's quite good. Unfortunately, I don't think I can see how many tokens I've actually burnt in all of this. My guess is that we have burnt quite a number of tokens. We can see now it's basically building, I think, the final section out. Obviously, it has a bunch of tools that it has access to in there. We don't know, for example, what all those tools are. Again, like I talked about earlier, it will be very interesting to see the actual system prompt for this.
So, for me, this is definitely the most interesting thing about this model. So, I'm going to test out the Kimmy CLI and see how that goes. And I've already seen some amazing demonstrations in here where you can actually go from images or videos to actually getting it to code something up. So when you actually make a video, it can actually see how the interaction of the website is going to work, not just still images, and create that.
So overall, I would say this model is definitely an impressive model. What they've actually built around this model, I think, is really amazing as well. And don't forget, this is an open model. You can actually download the weights for this, and you can actually see how it actually works. It's a Llama 3. It's got a trillion parameters with 32B active. We can see all the details in here. If you wanted to serve it, you're probably going to need quite a number of GPUs to serve it at anywhere near to the speed that I've been using it on their side. But this is certainly something that if you are an enterprise customer or something and you want to run this privately on your own sort of private cloud, etc., you're totally allowed to do it as long as you have less than 100 million users, etc.
And finally, just to finish up, if you want to try it out as an API, obviously you can use Kimmy directly, but you can also use Open Router. Kimmy has a whole bunch of things of where they verify different providers for that. So, that's something that you could look into. But if you want to just come and test it for running your agents or for using with something like Open Code, you can certainly use Open Router for that.
Anyway, if you've had a chance to play with this, let me know in the comments what you think. For me going forward, it's certainly going to be a model that I think is going to be interesting to see where it lands in regards to the whole sort of AI coding thing and for general agentic applications, etc.
Anyway, as always, if you found the video useful and you would like to see more videos like this, please click like and subscribe, and I will talk to you in the next video. Bye for now.