Transcription
All right. So, today we're testing out the Quen 3 model that just dropped. The flagship model they call it Quinn 3 235B A22B, which is a great name cuz it's so easy to remember. But it's basically the big model that dropped on some of the coding benchmarks. It supposedly actually performed better than Gemini 2.5 Pro, which I find surprising. Honestly, I'm not going to say I'm doubting the results, but like they say, trust, but verify. So, today we're verifying.
So, this is a chat, Quen.ai. And I put in the prompt. I enabled the thinking mode here. And I'll show you later. You're able to change the maximum sort of allowed time for it to think or, you know, number of tokens. 32,000 is the limit. And first observation, it's running for a long long long long time while it's generating the sort of the answer, the code that I'm looking for.
What is the code in question? It's this: write one self-contained HTML file, a 2D view of our solar system plus a user-launched probe. Users can click and drag near the planet to set the probe's initial velocity vector. When they let go, the probe is launched. Provide only three UI controls: play, pause, reset, and a slider for simulation speed. Wow.
So, it just completed. As you can see here, it's burnt through close to 40,000 tokens. And now it's actually outputting the HTML file. So, I got to say this is much longer than I would expect the other models to think about this stuff. And probably, you know, this is pages and pages and pages of thinking. So, it's thinking through every single thing. Hopefully, what that translates to is an incredibly good sort of uh output. So, but let's see.
All right. So, here's that simulation. Play, pause. Okay. Okay. Okay. I'm liking it so far. So, this is on the maximum speed. Um, so the speed has to be increased first and foremost because the maximum speed is still a bit too slow and at, you know, the minimum speed. Is this real time? It's real time. I mean, I can kind of see the movement barely, but okay. So, like this. And then what I should be able to do, if everything's working, is take a little probe and Oops. All right. So, that's the probe falling towards the sun and it's going to Yep. Okay. So, it kind of curved around the sun.
So, first and foremost, my initial kind of uh impressions is it's very, very good. After testing similar stuff on all the other ones, I am pretty impressed. We're also going to be testing a 3D version of this, but I got to say, so far everything's like super smooth and super good and and I'm really liking this. But let's reset it and try it again. So speed. Let's go. So the planets are beginning to move. I'm going to launch a probe. Like, I'm not 100% sure how to launch the probes. Uh oh, I see. You have to sort of launch them from a planet. So let's say we wanted to shoot from Mars, I assume. And okay. And then direction is the direction in which we're firing the probe. So we're firing it away from the solar system. Again, the speed needs to be increased. I'm going to go ahead and tell it to increase the speed. Right now, I'm going to say let's greatly increase the maximum speed. Right now, it's too slow. Let's do at least 10x. And as you can see here, our little uh satellite just flew outside of the solar system.
All right, let's try that again. So, we'll launch it from Earth kind of in this direction and see what happens. So, boom. All right, it's falling towards the sun. Let's try it again. All right, so we're launching it. Yikes. So, the other thing that I'm seeing is it seems like the only thing that has a that has gravity in the simulation is the sun. As far as I can tell, the planets don't really have any gravity as far as I can tell. So, I'm going to say: add buttons to turn on/off the gravity for the sun, and another button to turn on/off the gravity for the planets.
All right, so here is another attempt. So let's create Okay, perfect. So this is the speed is much faster and let's see if we can launch it off this planet. Perfect. All right, that's looking really good. This is exactly what we wanted to what we wanted to see. Um and I feel like the planet's gravity is affecting the trajectory of the probe. It's looking great. Looking phenomenal. Now, what happens if we disable the sun's gravity? It flies off. Okay, we can enable it again to slow it down. Phenomenal. Everything's looking great. So, uh let's see what if we launch it like this and turn off the gravity of the planets. All right.
So, all in all, I got to say I am very satisfied with everything here. Now we can mess around a little bit more with the specifics, like how much planet gravity we want to have versus uh sun gravity because right now the sort of the sun gravity is a little bit overpowering. So if you wanted to shoot it and have it curve past a planet, that might be a little bit difficult, but overall I feel like we definitely captured a lot of the elements here and we can start sort of building on top of this. I I got to give this a a very very high score; um we only had to do a few tweaks. Now, in terms of how realistic it is, I don't know exactly how realistic it is, but I just mean in terms of the the planet's rotation, the uh obviously this is not like a a perfect simulation. We're just trying to play around with the gravitational poles. And if we disable planets gravity, it flies off into space.
Okay, so I just realized that I did not enable planets gravity. So, by default, the planet's gravity is not enabled. So this is what it looks like with that. So let me reset it. Let's say we launch the probe like this. And then we hit play. So the the thing is something funky happens when it approaches the center of the gravity well, the planet so to speak. So it sort of like gets stuck and sometimes wildly bounces off of it, which is not really what we're looking for. But we can change, you know, obviously we can change how it's supposed to interact with that. So, right now it's kind of being a simulation of it. Obviously, something crashes into the planet and disappears. Uh, we can have it do that, but that kind of like doesn't really work well for what we're trying to do. So, if I enable the planet gravity, so as you can see here, the planets actually affected, but it does have some weird funky effects. So, here's me enabling the planet gravity. So, as you can see here, I mean, it's it's interesting, but it kind of pingpongs from one to the other. So, I'll give it a decent grade for this, but let's let's step it up a bit and see how well it's able to deal with something a little bit more complex.
All right. Next, I tried to create a 3D version of it with some more complicated prompts. Um, it wasn't able to render it, the actual galaxy. I tried multiple times. It was not able to do it.
All right. Next, we're doing a Python game, a soccer simulation. It's a 2v2 game where characters have stats, levels, experience, and as they play the game, they sort of get better and faster, etc. Let's see. All right, so here's that game. Uh, let's see. So, so far it's not doing great because the players aren't attacking the ball. All right, I asked it to rewrite it and we have this. So, it's not much better. So, as you can see here, the players are not interacting properly with each other.
All right. So, we're trying again with the updated code. So, as you can see here, it's still not great. So, something about this thing is just not coming together. So, as you can see here, I have thinking engaged, maximum length of thinking enabled is just not not quite going through. And just to give you kind of an idea of what the other models would do in the space. So, here's Gemini 2.5 Pro preview that actually gets an error on the first try. And here is OpenAI 03. So, as you can see here, the game is working. They're able to tackle the ball, to get the ball from each other. They're going for the goal. And let's see if the goal mechanic works. It works. You get nine points per goal apparently in this game. Uh, okay.
So, as you can see here, there's still some work to do here, but at the very least, like the whole game is working, the the various tackle mechanics are working. Let me run it again here. So, as you can see here, you got kind of have the defender, you have the person on the attack. So, as you can see here, they're going for the goal. They're running it into the goal, like uh which is a little bit weird but you know, still not perfect but um yeah, this is like a mix of American football and and soccer like where one goal isn't worth one point and also you have to run it into the goal, not into the end zone but into the goal.
All right. And here is actually the Gemini, what Gemini produced. It's exceptional, right? So, you can see the the various uh the kickoff timer. You have the experience points of the different teams. Looks like they have some sort of a burst mode, some sort of a turbo mode. You know, it keeps track of the score. So, again, it took one or two retries cuz it was crashing, but once it's done, yeah, it definitely has that that turbo bursting. But, I mean, I got to say, I mean, this is phenomenal. This is exactly what I was looking for. They're not kicking it at the goal; they're like running it into the goal. But still, like I would rather start kind of building on top of this rather than something that's not even where the basic mechanics don't work. So to me, I mean both the Gemini 2.5 Pro and the OpenAI O3 perform better than the Quen 3 for this particular prompt.
All right. Next, we're creating a fully autonomous snake game where two snakes battle each other. We're also adding some things like uh obstacles and fruits and various other things. But here's the kicker. We're going to attempt to create a reinforcement learning sort of training pipeline for these two snakes to get better at playing snake. And so this is one of the more complicated prompts. I'm asking it to create one file that I'm able to run with different sort of arguments to have it behave differently. So if I run it with the, you know, d-play argument, then it's just these two snakes that play autonomously with a simple script, right, so just scripted movement. If I run it with the argument train, that's going to create a reinforcement learning pipeline using PyTorch, which is an opensource sort of ecosystem for these u reinforcement learning training things, right? And the the agents will play the game, simulated playing the game over 500 episodes, and then we save those sort of neural nets to be used in our next sort of iterations for the snakes to play. And then if we run the argument one, then we use that neural net in snake number one. If we used argument two, then we use that neural net in snake number two. So this is a fairly complex prompt. So it thought about it and it put out this. Let's see if it works.
All right. So we got an error. We got an error on our first try, but that's fine. That can happen. Meanwhile, I'll also throw this into let's say Gemini. I'll do the same exact prompt so you can see kind of what that creates. So I'm just copying and pasting the same exact prompt into there, and we're using a 2.5 Pro preview. So, one of the things that it does, which I I really hate it, is like when it has an error, right? What seems to be a fairly simple error, right? It's like, "Oh, well, here's some stuff you can try. Like, here's some suggestions for how to fix your code. Go off and then and tell me what happens." It's like, "No, no, no. You wrote the code. You figure it out. And then you output the entire code again so I can just copy and paste it into the thing." Now, yeah, that burns through more tokens, but the point is at this point, we're really testing the model's ability to troubleshoot its own problems, not my abilities to troubleshoot the code that it wrote. Right? That's why I usually don't really even look at the code cuz it might have like a a simple fix that be like, oh, you're just you need to change this, but usually I try not to do that and see is it capable of doing it completely on its own. In the past, every once in a while, I would kind of step in and maybe help it kind of connect the dots a little bit. And I think that's fine, but for the most part, I want to also see how it's able to sort of troubleshoot its own code.
So, here we're going to try that. So, we're still getting um some sort of an error here. Meanwhile, here's Gemini 2.5 Pro. So, this is the code it generated. So, let's see how that works. So, I'm going to post that in there and then run this, which should just play the game by itself. Okay, so it wants me to type mode and then play. Okay, like that, which I am totally okay with. I will type that in. So, there's no attribute food. Okay, so it's still it also has some issue. Sometimes there's like a tiny little misspelling or whatever, some sort of a syntax error within that huge long code that it wrote. So sometimes it just has to go back and tweak that one thing. It forgot to define something, something very very simple.
What I really love about Gemini 2.5 Pro is notice it it for the most part usually it puts out the entire code. Again, it's not like here's the little snippet that you have to um that you have to go in and hunt it down and put it, which is it's not that difficult, but you know, it's are you optimizing for the LM's time? You know, it's valuable time, or are you optimizing for the user's time? Feel like if you're optimizing for the user's time, this in general is going to be a lot better, a lot easier. Plus, in the future, if they're really integrated with the kind of development environment, you also don't want it to just say where the correction is; you want it to be able to go and and find it and change it. So, I don't know. I feel like during this kind of intermediar period where I'm copying and pasting the code over it, it is going to be better if it just outputs the whole code. But obviously, that costs more. That burns through more tokens. I'm kind of aware of that. But, you know, even if it's something that where I'm paying on a per token basis, I'd rather pay a little bit extra to have it do this, at least for what I'm doing for these purposes.
Quen 3 still is trying to get me to, you know, change various little snippets here and there. So, rewrite the entire code, give me the entire code, and I'll test it. Um, you know, I I give it back the error with a description of what went wrong. So, it should be able to to fix it based on that. All right. Right. So this is the Gemini code that it updated. So let's see if it's able to Okay, perfect. So this is kind of what we're looking for with this particular mode. So this is the mode where it just plays automatically with a simple little script. What we expect to see is each game it sort of iterates and improves its score. There's a cumulative score being kept. And since they're both running the same script, we kind of expect them to stay more or less tied up, you know, over time; one might pull ahead or not, but that's just like kind of a statistical variance sort of thing. They're basically the same exact snake, the same exact script. But what happens if we train it?
All right, so here what we're going to do is we're going to run it in the mode train and episodes. I'm just going to run it for 200 episodes um just because now really we don't want it actually running. Now it actually is kind of rendering the games. There's a way to run it that's called headless where it it's not rendering it, which usually will allow it to run a lot faster and it also gives me a option to make it run faster. So let me see maybe that will help. Yeah. So as you can see here it's running through the episodes and it has a certain sort of average score and these snakes are learning to play the game and improve their score and uh sort of this is the kind of the training process. So these this is we're building an actual sort of a neural net for these things to be able to play the game better. I'm actually not going to run through this cuz normally number one you would run it in headless mode. So it takes it's it just goes by a lot faster. Number two, I know the Gemini 2.5 Pro can do this. So that's not really the question here. As you can see here, it's doing it. This is just going to take a lot of time. So we'll skip it. The question really is, can Quen run this thing? So let's let's find out.
All right. So, we're going to say play, and let's see if it can sort of play. So, it looks like what it's doing here is it's sort of it's assimilating the game play with just text characters. So, that snake is kind of running around eating the fruit. The little hashtag symbol is those um immovable obstacles that it can't crash into. So, this is a very smart way of running that training because you're sort of simulating the environment without needing to actually run the game. So, what that means is we're able to run the game much faster, still train the snakes, as long as later we can take their findings and translate it into sort of the the real world, the real world snake. I guess it's still a video game, but this is kind of a more like abstract version of the video game. So, I got to give it some credits for coming up with this solution. This is this is clever, but the problem is that I've asked it to run as just a game to begin to make sure that can see the game. I didn't ask it to start the training process yet. So, let's see why that's happening. So, I ran this command right here. So, that should have just played the game, not train the game. Let me just verify I did that correctly. So, yeah, I'm doing d-play and it's going right into the training mode, which is not quite what we want. So does that mean that whatever we put in there it's going to do it's going to do okay. So interestingly we it does have a training mode. This is it kind of training the snake. And now what we're able to do is we're able to put, for example, the trained up snake with its new neural nets. We can put it in a snake number one. So the newly trained snake will be player one and the other one will be that scripted. So let's see if that works. So I think what it did here interestingly is it didn't actually create a visual display for the game. It just created or I guess this is a visual display but it's made out of, you know, text characters. So not quite what I was looking for. But let me kind of verify what I said in the prompt cuz this might fulfill sort of the requirements of the prompt that we gave it. So interestingly how I wrote it. I guess this technically fulfills the prompt because I didn't specify that we wanted a visual sort of representation of the snake game. So, it chose to do it with just a text-based interpretation. So, I got to say it gets a lot of points here because it did create a reinforcement learning pipeline using PyTorch. It did allow me to sort of use those arguments to train the snake versus play the snake. Like, it did everything that there's a lot of phenomenal things here. You know, I'm tempted to take a few points off just because it didn't go with the visual style, but also I didn't ask it to. So, I guess shame on me. But all in all, this is very, very solid. This is very, very good.
All right. Next, we're asking it to create a self-contained Python program that uses the user's webcam. The user is able to play music by using their hands. The program should track the user's hand gestures and play music accordingly. Okay. So, that's pretty good. Um, some of them really kind of nail it to where you're almost like able to play pretty exciting sort of a musical pieces. Some of them do it very very basically. This was a little bit more on the basic side, but it one-shotted it and it just uh it made it work and I have no complaints. It did it very very well based on that fairly vague prompt that I gave it. So definitely A+, or actually now that I think about it, let's do it's it's an A. It's not an A+ cuz I've seen some really phenomenal things that other models have come up with. This isn't the best output, but it's it's a solid A.
All right, the next thing is a little bit more complicated. So we want this in HTML format as one file, and we're going to be using a couple API keys here. So API keys allow us to sort of tap into some online providers like OpenAI or 11 Labs to have it do certain things for us. So, for example, 11 Labs is one of my favorite sort of AI voices. It generates really good articulate AI voices. And OpenAI is of course, you know what OpenAI is. So, here's the prompt: Use my OpenAI and 11 Labs API key—I forgot to say keys—uh to create a voice-narrated interactive audiobook. OpenAI will produce the text for the story and 11 Labs will voice the story. Create this as one HTML file. I tried doing this with a Python file first. That didn't work as well because everyone hates Windows users and nothing works on Windows. So, your mileage may differ, but we're going to do this as HTML. And we're going to say have the story be interactive and the player will be one of the characters that will drive the story forward. Create an app that uses the user's microphone to interact with the user. And I provided my 11 Labs and OpenAI API keys. I straight up just pasted it into all of these models. We're going to be testing this on Gwen 3, Gemini 2.5 Pro, Anthropic 3.7, and Opening Eyes 03. Now, this is a bad idea. Do not do this. Don't post your API keys into the various chatbots. I delete all of my keys after testing them out, before publishing these videos, like directly after doing these projects. And later if I keep using the project, I create a separate end file. I keep all my API keys in there. I do this more to test how these models handle me posting this information in there. That's like kind of a security issue. So some of the points go into kind of like how well do they deal with me? Like if they straight up refuse to do it, I they they get points off because that means they're sort of built to, you know, with this idea that like like we know what's better for you than you do. I I I prefer they they warn me. They make me understand what the issue is. They give me a few options, but they don't reject the prompt. If I want to do something foolish, let me do something foolish. That's how I feel about life. Come on now. Like, thank you for warning me and making sure that I'm aware that this is a foolish thing to do, but don't prevent me from doing the foolish thing. Does that make sense? Do you agree?
So, Google Gemini, Gemini 2.5 Pro was the first model to return the answer. I don't know if that was the first one that I saw to complete it. And the first time I was trying it out with um the Python, trying to do this in Python, it gave it did what I said. So it just put the API keys within the file, which again is sort of the unsafe practice. It warned me that this was a bad idea. It explained why it was a bad idea. And as you notice here in the code, it actually created a section to say like if you're going to use that file, here's everything you need.
To set that up properly. And it commented it out. So, meaning that the code is there. This code is ignored when it runs. But if I wanted to use it, like all I would do is just delete these little uh symbols, and then that code would run; it would run as intended. Well, this should be commented out, but if I just do that, then that code block would allow me to include the more safer option.
So, first of all, really good for Gemini 2.5 Pro for doing this. This is probably the best possible way to deal with this, I think. But the different models I tried, they all, with the Python script, they all went to such different directions with these massive workarounds that I said just forget it. Let's do HTML. So, let's start with Google Gemini 2.5 Pro. I think one of the things that I'm going to be adding to any prompt moving forward is to make sure that it's creating it in dark mode. I can't even enable dark mode here, but okay. So, let's just do this. All right.
So, here's what Google Gemini 2.5 created. We're going to click start adventure, and it's going to generate the story. And so, it says error um listening, getting the file. So, you have to allow. Okay. So now it's playing the story segment. So what it did here while it's playing, so as you can see there, it's a 1 minute and a 10 second introduction to the story. And so what it did was it sent that prompt to Chad GBT to OpenAI, saying, "Hey, we're going to play a little interactive audio adventure." You know, he creates a story. So it created that story as text. It took that text and it sent it over to 11 Labs.
So, the issue that I'm seeing here is both the audio that's playing and the microphone, they time out. So, one thing that I would ask it to improve is, "Hey, just keep listening for me talking, keep playing it, but allow me to interrupt and move the story forward." But here's what the audio sounds like: "The town's folk continue with their lives, unaware of the urgency of your mission. Do you head straight to the cliffs to search for Thomas, or do you seek out more information from the town's people before making your decision? What do you do?" Let's head to the tavern and see what the rumors are. You decide it's best to gather more information before confronting the unknown. With a nod to the elderly woman, you set off toward the local tavern, the salty anchor. All right, so I'm going to end it right there.
But I got to say, this is more or less perfect. There's a few tiny glitches which I think we can probably fix with another prompt. But as you can see, it more or less one-shoted what we wanted to do. So, I can drive the story forward. I can say, because again, that wasn't one of the choices it gave me, right? It said, "Do you want to find out more or do you want to head over here?" I specifically said, "I want to go to the tavern." And then it creates that story. It's like, "Oh, the tavern is the salty anchor or whatever it is." And so basically you're able to generate and create the story in real time. So A+, absolutely phenomenal. It's working very, very well. You know, if I had to nitpick, I keep having to click use microphones. So instead of being continuous, I have to allow it each time. So there's some issue there that needs to be fixed. But Google Gemini 2.5 Pro, I would say phenomenal job. A or A+, depending on how you want to grade it.
All right, next up we have OpenAI O3. It has dark mode, first of all, gets points from me for that. Dark mode is excellent. All right, so speak or type to play. This is really smart. So it allows me to type, and so we're supposed to write our destiny. We're aboard an airship. A dragon attacks the airship and we engage in a glorious battle. All right. So now, assuming everything goes according to plan, which it rarely does. Okay. So, it picks up what I wanted to do. So, but nothing's happening. Let's click play. "Welcome, brave traveler. You awaken aboard an airship soaring above luminescent clouds. What will you do first?" Let's continue to the land of the dragons. "You feel a surge of excitement as you approach the mystical land of the dragons." Okay, so a couple things. The OpenAI O3 did a lot of things right. I can't, you know, it definitely was everything was perfect. Um, now the only thing is if we start saying a new line where we want to go, it should stop the previous narration, and it shouldn't just play all the different narration, all the sequences over each other. But again, that's easily fixed. I'm going to give it an A. I think it did a phenomenal job. Everything worked perfectly right out of the box. The reason it didn't load the first time is because this one, it did refuse to put my API keys directly into the code. So, it said, you know, you have to fill it out when you are ready to use it. So, which is fine. I prefer, you know, it's probably safer. I prefer when it does kind of what you tell it to do, but if it doesn't, that's fine. That's part of the company's decision to kind of how they want to keep their users secure. So, that's fine. That's okay. I understand it. For me, it's not the best, but it's fine.
Next, let's try Quinn 3 and see what it did. All right, so here's a Quinn 3 interactive story. Let's see. So, this means that it was able to use the OpenAI API key to get the story. I am not hearing any audio. So, that means something's wrong with the API key for 11 Labs. So, the 11 Labs is not working, nor do I have any way to continue the story by talking to it. As far as you can see, there's no easy way to fix it. There's no obvious fix. So, it's missing the part where it's using 11 Labs to voice it. It's missing the part where I'm able to talk to it. So, that is not ideal. So, this gets a C for at least attempting and getting the OpenAI API right, but everything else seems to be off.
And finally, we have Claude 3.7. Interestingly, Claude, since they have artifacts, it kind of makes everything within it. So, let's say we call ourselves Wes and we want to do fantasy adventure and we will be Arnold and volume 70%. Sure, why not? Let's try it out. So, the artifact didn't work. So, I'm going to try to see if I can do it in Rome. So, and also it's a good way to compare it to all the other models how we did it. We don't want to compare it in a different way. So, this is kind of keeping everything consistent. "Teaming with magic and mythical creatures. Alex, you are the chosen one. As you wander through the tranquil forest of Eldora, an ancient parchment clasped in your hand." Let's enter the city of Arara. "Way to the lost city of Ara, where untold treasures and deadly challenges await. From the shadows of the forest, a pair of glowing eyes watches you." All right.
So, I got to say Claude 3.7 Sonnet is phenomenal. This was probably the best one. Notice the UI is very nice, very pleasant. You're able to pick a bunch of options from the very beginning, from the very get-go. Everything worked perfectly. Very, very impressive. Even if you noticed, it kind of refreshed. So, when you spoke your answer, yeah, the narration would continue over you talking, but then it would refresh and continue on to the next piece. So, you know, it was well done. It could have been, we could fix whatever issues we've had with just a few more prompts here and there, but overall, I got to say it's fantastic and probably the best one so far out of everything that we've seen.
All right, so after running some of these tests on the Quinn 3 235B A22B, I got to say it's not bad. It's a decent coding model. It there's a lot of strength there. I like a lot of the things that it did. I was surprised by how well it did the PyTorch reinforcement learning, how it did a lot of the other stuff. So, I definitely would say that it's a good to great model and very impressive in a lot of ways. However, at this point, I'm not entirely convinced that, for example, that code forces score that's higher than Gemini 2.5 Pro is maybe necessarily accurate. I do still feel that Claude 3.7 sonnet is a much stronger. I think Gemini 2.5 Pro is stronger. The O3 model, like all of those seem to be like they're slightly better than Quen 3. Now, of course, keep in mind this is just a few simple tests. We can't 100% guarantee that these are going to be accurate results. They may be different for your specific use cases. And this is just not enough to kind of test it once the LM arena once you know the thousands of users that they've had a chance to test it out. We'll also be able to see kind of where this model sits on the LLM arena. So, for example, if you're looking at the web dev arena, so it's coding but specifically for webdev. So Claude 3.7 sonnet is at the top, followed by GPT4.1, then Gemini 2.5 Pro. You know, if we're looking at the overall view, then Gemini 2.5 Pro's on top, O3 Gro 3 previews up there. It's a bit strange that we're not seeing a Claude 3.5 or 3.7 near the top, especially for things like coding. But my point is, when we finally get all the results back, is Quinn 3 going to be at the very top of this, maybe like right under Gemini 2.5 Pro? I don't expect it to. I think once people use it for their own specific use cases and compare it side by side to other models, I mean, I don't expect to see it like at the tippity top, somewhere down below. It's not a bad model. It's strong. It's probably going to be the strongest open-source model available for sure. I do expect it to beat out DeepSeek V3, Deepseek R1. I would assume it will, but it's not going to be above the big proprietary models from Google DeepMind, from Anthropic, from OpenAI. I just don't see that happening yet. But time will tell. Let me know what you think. If you've been testing it out, do you find that it's a very strong and able coder? Do you find that it's good, but maybe not the greatest, which is kind of like what I'm seeing right now? Let me know what you think. If you made it this far, thank you so much for watching. My name is Wes Roth, and I'll see you next.