Transcription
So today, OpenAI released an official codec plugin for cloud code. You can see here it says if you already use cloud code, this codeex plugin gives you a simple way to pull codeex into that same workflow. Now this isn't like groundbreaking technology. People have already been kind of using a combination of codec and claude code together, but this plugin makes it a lot easier.
And so obviously I've been focusing a lot on cloud code, but I have played around with codecs a little bit and they felt very similar to me. But as of lately, I've been seeing more and more people talk about how they've been using codecs inside of their cloud code workflows and projects to help do code reviews and to help get a second pair of eyes on their code. So here you can see the head of developer experience at OpenAI said that they've been seeing cloud code users bring in codecs for code reviews and use GBT 5.4 for some more complex tasks. So that's kind of why they decided to do this.
And the cool thing is, which I didn't actually know, you can use codecs for free. So, you can just use your free Chatbt subscription, which means there's basically no reason for you to get this plugin, put it into some of your Cloud Code projects where you're building some software or game or whatever it is, and then just have it do some code reviews and see what you might have missed with Claude Code.
So, you might be wondering to yourself, is this just overkill? And is it actually needed? And so, I was thinking the exact same thing, and I decided to do a little bit of research to compare the benchmarks. So, starting off, Opus 4.6 versus GBT 5.4. Opus is the purple, GBT is the green, and we're looking at different coding benchmarks. So, the S. S. S. S. S. S. S. S. S. S. S. Opus has a slight edge. But in all of these other ones, you'll notice that GBT 5.4 is actually ahead of Opus 4.6. And if you actually break them down headto-head, you can see here's the S SWB verified where Opus leads by almost a point. But in all these other ones, 13 points, 10 points, 1, 2.5, 3.5, GBT 5.4 actually beats out Opus 4.6. And when you think about those benchmarks and then you realize that Opus 4.5 is also much more expensive than GPT 5x4, it definitely makes you think.
Now, all of these are the benchmarks, right? So, I think they're always important to look at and to sort of be aware of like what models are supposed to be good at certain things, but I always want to get in there, get my hands dirty, and see how I like them, see what they feel like, rather than just basing my assumptions off of benchmarks. So I did some research on X and Reddit and I scraped information and thoughts from people who have been using both of them a lot and I threw together some of the pitfalls of both of them. And it's interesting because when you hear the weaknesses of both, the strengths of both kind of complement each other really well.
So for cloud code, some of the things that people have been saying are weaknesses are that it overengineers, that it can be really token hungry, and that it can sort of get into long run drift. So it can miss edge cases and just build bugs. And the important part of that is when it goes and reviews its own code, sometimes it misses those because it just can't see them. And these things right here are some of the things that Codeex is known for being really good at. But then if we switch over to Codeex weaknesses, people complain that it's not super good at planning and it doesn't ask the right questions and sometimes you don't get a lot of good creative outputs from it and it's just a little bit more rigid in that way. And it's interesting because some of these things that people complain about codecs, cloud code or opus does really well. And so naturally, that's why you've seen a lot of people on X talking about, oh, I like to plan and maybe build the initial PC with cloud code, but then I'll bring in codeex to execute the rest of it, kind of push it into production and do all of the reviewing.
Now, what this really makes me think about is just the fact that you should be trying different tools and understanding like where to use them together. It's not always like, oh, I'm all in on this one and I'm only going to use this one. It's about understanding like, okay, for this specific use case, I might use 30% Claude and 70% OpenAI, and for this next use case, I'm probably going to use Claude Code pretty much all the way through until the very end.
So, I'll leave a link to this article down in the description of this video, but it's really, really simple to set up. All you have to do is run these three commands in order. So, this one basically installs the marketplace, this one installs the plugin, and then this one helps you get set up. And you can also go to the actual GitHub documentation, which has been growing in stars pretty quickly since the release. And you pretty much can read every single thing about this. You can see that you can also use your free Chatbt subscription once again, but you would just run these in order. And it also breaks down all of the different functions that are in here. So we have like a review, we have an adversarial review, we have rescue, we have all these different little functions that you can run, which almost you can think of them like skills now. And you can do some other things that are pretty cool by adding these flags that let them run in the background or wait for things. And you get a lot of extra functionality here.
So in a session, if you come over here and you do /plugins, you would basically have to, you know, try to install that marketplace. You can see that I have the OpenAI Codex one right here. And then you can see right here I've got the codeex plugin installed and enabled. So now if I went ahead to do a /codex, you can see all of these different things that I could actually call on. And all of these would be using GBT 5.4 instead of Opus.
So real quick example of what that may look like. Here's a project where I'm setting up just some sort of dashboard for an internal system. And keep in mind, a lot of this is mock data. This is something that I just recently spun up and right now I'm just working on sort of the flow and the feel rather than like having the data synced in. But anyways, I built this obviously using Opus. So now in this project, if I do /codeex, I can see all these different things to run. And right now I want to decide between a review or an adversarial review. So if I go over back to the GitHub, we can read the difference between the two, which is a review runs a normal codeex review on your current work, which is the same quality of code review as running a slash review inside of Codex directly. So you can use this for reviewing uncommitted changes or comparing branches. And this is a readonly type of skill.
Now the adversarial review is kind of just like a review on steroids. It's steerable and it questions the chosen implementation and design and it can be used to pressure test things, look at trade-offs, failure modes, and whether different approaches would be safer or more simple. This is also a readonly command that does not change code. Essentially, these are both just kind of giving you uh a nice audit. So, I'm going to go ahead and try the adversarial review right here. So, what you'll notice is right away it has to get familiarized and acclimated with the environment. So, it's going to look at the working tree size. It's going to check the differences between what's staged and what's unstaged. After that, it should come back and ask us how we want to run this review. So, it's asking me how we want to run it. I'm just going to go ahead and say in the background and shoot that off. You can see that it also said that this is a pretty large review. So, we'll see how long this takes.
So, by the way, I'm on Windows and when I was using this, I've gotten a bug a few times which basically said that there was something wrong with like path, but it was able to do research and fix that for me. And now you can see that that adversarial review has been complete. So, let's take a look and see what we got here. It gives us a target. It gives us a verdict. It tells things that we don't need to ship. And here are the things that it found. And it also looks like it has priority scores. So, we have some high priority things. We have a recommendation. Same thing right here. And then we also have a medium priority thing as well as some next steps. So from here it would be up to you to either continue on using codeex to build out these changes or switch back to cloud code or what I would probably do is I would split them off do one with opus one with GBT and then just see which one's better.
So that was a lot of the use cases that I've been seeing on X and just when I started looking into this, right? But then I of course wanted to try okay let's give both of them the exact same prompt and see what the outputs actually look like. So, I gave them both this prompt, which was basically just like to build me a game. I gave them some, you know, specifications and things like that. And it's detailed, but it's not like super super detailed. I didn't put them into plan mode. I just shot it off in bypass permissions to see what the outputs looked like. And so, here's codecs open up in VS Code as well with the extension. And you can see that I gave them the exact same prompt, and then I just shot them off to see what happened.
Now, before we open up the outputs, there's two things I wanted to tell you. The first thing is that Opus finished way quicker than Codeex did. So whether you take that as a good thing or a bad thing, just something I wanted to call out. Codex ran for a lot longer. And the second thing is Opus came back and basically said, "Hey, the server started, you can open it up and you can play the game." But when Codex came back, it basically said like, "Hey, this is only task one out of three, but the game's finished and you're ready to play." And then after that, I said, "Okay, well, um, is the game done?" And it said, "Not fully. It's playable and it's local, but there's still a lot of things that I need to do to meet the original spec."
Now, let's take a look at the games. This first version right here, this is the version that was built by Claude Code. So, we have dungeon crawler, a roglike adventure. We're going to go ahead and start up a new game. And you can see here we have a little navbar on the right with our floor. I think maybe our health, our stats, our XP, equipment, gold, combat log. We have a little mini map. And then we have our controls. So, if I start moving around here, you can see that I can kind of see what's going on. I can break through these little barriers. And here's like a goblin, apparently. I can move up here. There's a skeleton. Here's some gold. I don't know how to pick that up exactly. Um I don't know exactly what all this stuff is, but you can see that it is, you know, it's a 2D playable game. It's nothing crazy, but the mini map starts to unlock. And honestly, like for one prompt and maybe like about a 5minute workflow, this is not too bad.
But now, let's just go ahead and close out of that and let's go over to Codeex's version. This is what the initial UI looks like. And already it looks a little bit more polished and it feels more like an app. And I can go ahead and start this new game here. And let me just zoom in a little bit more now. This one already looks a lot less pixy. And I guess I died really quick. But this one looks a lot less pixy and it just feels a little bit more polished as you can see. Um I don't actually see on this right hand side though. I guess the mini map is down there. So there is still the mini map. But overall I would have definitely said that Codeex did a better job of this game on the first oneshot prompt than Claude Code did.
So now what I would do is I would come back into this game, the version that was built with cloud code, and I would just want to do an adversarial review. And then what I'm going to do after Codex comes back with this review, I'm just going to tell Opus, go implement all of these changes, and then we're going to open up the game again and see how much different it is. But that's why it's always interesting to test things out yourself because some people were saying that Codex does a worse job with like UI design. Although clearly from this example with the same prompt, Codex's version in my mind was a lot better. >> [snorts] >>
Now, as this is running, I did want to just address something real quick, which is basically like, okay, you know, if Codeex is just better than Cloud Code and it's cheaper, then why would I not just use Codex instead? And that's honestly a very fair question. And the only answer that I can give you is download it, try it, and just see which one you like better. From what I've been playing around with so far, I think there's definitely very clear strengths where Codeex is better than Cloud Code. But overall, I really like the way Cloud Code feels. I think that it's a lot more forgiving, especially for me because I don't come from a formal software engineering background. And that type of feel aligns with a lot of the stuff that I've been seeing other people say as well. But I'll tell you for sure that I'm definitely going to be using this codeex plugin to help review stuff and to help build on extra features. And maybe one day I'll get to the point where I decide that I want to use codeex 80% and cloud code 20 rather than the other way around.
So this is now running in the background and I wanted to show you guys what it looks like when you do the codec status. It basically just shows you the exact job that has, you know, been running and how long it's been running. And it looks like that's actually finished up because now it's giving us another output. And this is interesting because I just spun up this game as a demo. There was no actual branch to compare itself with or there was no commit. So what it decided to do is rerun it properly by creating a review branch so that codeex can actually diff the actual game code. So, this has come back and keep in mind that these changes that it's suggesting are probably not going to be anything to do with the UI. So, that won't really see any changes. It's about the actual game play and it's about the actual like functionality of the code and how this works.
So, we have just two high things. We have final floor stairs let the player soft lock the run permanently. But the ancient amulet is only ever spawned when floor equals 10. So, there's no matching upward travel mechanic and victory only triggers from using the amulet. So a player can therefore step on floor 10 stairs before collecting the amulet, get sent to floor 11, and then the run becomes unwinable. So that's definitely a flaw in the way that this was actually built from Opus. And then there's another issue over here, as you can see, which is a real data loss rollback bug for a game that exposes a continue entry point. So it gives us a recommendation for how to fix both of these. We can gate floor 10 stairs, and we can also persist after state changing player actions or debounce an auto save after each turn. and on major events like new game blah blah blah blah blah.
So I'm just going to go ahead on bypass permissions mode just say implements. But obviously all of this stuff is kind of more of a demo. I would be taking this feedback and I would be once again going into a plan mode with cloud code and then taking that plan and telling it to go ahead and make the changes. So those changes have been made as you can see. And if I go ahead and open up the game and I give it a quick refresh, like I said, nothing really about the UI of the game has changed, but apparently those changes have been fixed. And actually, it does already look a little bit better for some reason. I'm probably just making that up. But anyways, the point being that was a super strong combination. As you can see, we're using Cloud Code and then the adversarial review with Codeex.
So, anyways, that is going to do it for this one. I know that I haven't made a video on Codeex yet on this channel, but I've been playing around with all these different tools and focusing mainly on Cloud Code because I don't want to just confuse you guys with throwing a bunch of different tools at you. But, I do think that this one is definitely something that is worth checking out. So, if you enjoyed the video or you learned something new, please give it a like. It helps me out a ton. And as always, I appreciate you guys making it to the end of the video. I'll see you on the next one. Thanks, everyone.