Transcription
Okay, so we have a brand new planning mode inside of Cloud Code known as ultra plan, and I'll be going through that in this video as well as my own thoughts about this because, despite the name "ultra" being the name, you may actually be underwhelmed when using this new feature.
Now, essentially, if you go to Cloud Code and then you do slash ultra plan, then you can give it a prompt. So, for my application Hyperspace, I basically want to upgrade Quinn 3.5 to Gemini 4 instead for the local model. So, if I give it that prompt, "Can you upgrade Quinn 3.5 to Gemini 4 for the local model in the Mac OS application?" and then pressing enter, it will say that it will run ultra plan on the cloud. Press enter, and then it will give you a session URL so you can like watch it planning and then actually respond to the plan that it gave you.
So, clicking on this URL over here, it should then take you to the Cloud Code web, like, uh, version, and then you'll see that the ultra plan is in action. Now, I actually made 10 ultra plans yesterday, so I ran this prompt before, and I ran 10 different prompts for a local planning mode and 10 different prompts for ultra plan mode.
Now, when the ultra plan is presented to you, you will see that it looks kind of like this. So, they have a nice UI over here, and you can select things over here, like leave a thumbs up, like emoji, smiling face, and you can leave comments in different areas over here. So, like leaving a comment saying, "User recommended sampling parameters." And after reading through the plan, I could send these two comments back to Cloud so I can adjust the plan on the cloud version, and I can press "Approve Cloud's plan," "Start coding on the cloud version," or "Teleport back to the terminal" instead.
And you'll actually notice two different things here. Firstly, that ultra plan is actually really fast on the cloud. So, I noticed that it was up to two times faster than the local plan for the same prompt. And you'll also notice that sometimes you get diagrams in your ultra plan, kind of like this ASCII diagram or this mermaid diagram over here. But other times, you will not see any diagrams at all.
And then on your terminal, once the ultra plan is ready, you will see "Ultra plan ready" over here. Press down, press enter, and then you can stop the ultra plan, go back, or review in Cloud Code web. So, reviewing in the web version, I could then press "Approve plan" and "Teleport back to terminal."
Now, this will then send the plan back to the terminal, and then I can do "Implement here" like in this current session, or do "Start a brand new session" for this plan. So, starting a brand new session, it will then like move me over to that with the ultra plan passed back in. So, essentially, the difference from the normal planning mode is that you use a trigger word "ultra plan." It hands it off to a web version of Cloud Code that researches and drafts a plan. You can leave inline comments to the plan, and then once you've approved the plan to your liking, you can either run it in the cloud or teleport back to the local session instead.
And you will also notice that for other sessions, when you run the local plan, then you will see right over here, and there'll be another option that says "No, refine with ultra plan on Cloud Code on the web." So, pressing that, it will then transport your local plan to the web version, and then it will do a validation step on the cloud.
Okay, now the question that you probably have is that is this ultra plan actually better than the normal planning mode inside of Cloud Code? Now, yesterday I did like 10 different ultra plans and 10 different local plans and then compared them for the same prompt, and I did notice some differences, but like I couldn't reliably get those differences. So, some of the differences that I did notice is that for things that make small changes across the entire application, like updating a dependency, ultra plan seemed to do a better job at auditing the blast radius and then flagging any risks compared to the local plan instead. Whereas a local plan just kind of assumed that any build step or build command would catch any things that it missed when upgrading a dependency. So, in one particular case, I was migrating from TRPC version 10 to version 11, and the ultra plan caught more potential issues compared to the local plan. But in other cases, removing Quinn 3.5 and using Gemini 4 instead, it seemed that the ultra plan didn't really have an impact and make the plan any better. So, the only difference was that it basically presented a nicer UI for actually reviewing the plan and leaving comments at specific points.
So, as a short aside, I do also have a free Cloud Code newsletter. So, if you're interested in getting the latest from someone who's using Cloud Code like 10 hours every single day, then I do share this in my newsletter. There will be a link down below if you're interested, and signing up will give you access to a bunch of free videos from my masterclass.
So, then I was wondering, like, okay, why is ultra plan really good in some cases and like kind of mid or basically the same as a local plan in other cases? So, I got Cloud Code to do some digging through the binary readable strings over here, and essentially it discovered that there are three different planning modes inside of ultra plan. So, there is a simple plan, the visual plan, and then three sub-agents with a critique plan. And this is the system prompts for each of the planning modes right over here.
So, the visual plan is basically the same as a normal plan, but it just generates some ASCII diagrams and like mermaid diagrams as well, which is why sometimes I notice the ultra plan mode like generating mermaid diagrams kind of like this, and other times not.
And then this is the most interesting planning mode. So, they call this the deep plan, and it's basically a multi-agent exploration with a critique pass. So, they have one agent that understands relevant code and the existing architecture, another one to find all files that need modification, another one to identify potential risks, edge cases, and dependencies, and then another one to review the plan for missing steps, risks, and mitigations over here.
And essentially, what Anthropic are doing behind the scenes is that they're using a remote config to decide, and this is entirely server-controlled. The user never actually picks this. So, I noticed that sometimes I was getting a really good plan because they assigned me variant three, which had the three different sub-agents with a critique, and other times I was getting a pretty mid plan, which was the same as a local plan.
And ultimately, what I think is happening here is that Anthropic want to like measure how many people are accepting each variant and then deciding which like planning mode, planning prompt is better for users. So, essentially, this whole ultra planning thing is like an A/B test for us to figure out like which system prompt they should be using going forwards for planning. But I think that this gives them the infrastructure to do other testing as well. So, for example, for an upcoming model, they could use an upcoming unreleased model to generate the plan, and then they will measure how many people are accepting that plan to inform whether that prompt or that model is actually good for like planning. So, maybe with all this A/B testing that they're doing with ultra plan, they'll get like a better sense of how good planning works.
But ultimately, I want to know exactly which planning mode that I'm assigned to. So, I think that I will use the ultra plan deep plan prompt directly as some kind of skill instead. So, this will be down below for those of you who are interested in copying this because, just by reading through this prompt, it seems that this approach may actually be pretty good compared to the normal planning mode prompt.
Anyways, that said, you will find that the ultra plan is actually faster. In all 10 versions that I ran, ultra plan finished about two times faster compared to the local plan for the same prompt. I think that it is better for multitasking as well because you can spin up in many different ultra plans, uh, from like your terminal and then review them all in like a nicer UI as well. And it can be quite handy for quick prototyping and iteration because you can fire off a change to the cloud version of ultra plan and then see what that would look like. And then if the plan doesn't look good, you can just discard it quite easily without polluting the rest of the conversation.
Anyways, I think that I will stay away from ultra planning for now and just use like three sub-agents with a critique as a custom skill instead.
Anyways, if you're interested in diving much deeper into Cloud Code, then I do have an entire masterclass about this, which many people from some of the world's biggest companies have already taken. It is the first and the most comprehensive Cloud Code class that you'll find online, and there are many of the ideas that are covered here that you will not find anywhere else on YouTube. I do update it every single day whenever a new feature is released or there's a new technique that I discover.
If you do have any questions about this, then you can email me using the email you can see on the screen right now.