Transcription
The number one thing that everyone wants when working with AI is to get exactly what you're thinking and what you want out of the AI agent. But a lot of the times your experience might be something more like the agent forgetting something, missing context, skipping details, not following instructions exactly, or ultimately getting an output that isn't exactly what you want.
Your problem might not be bad prompting. It could just be that the agent doesn't have the exact information it needs to solve your task. I'm Kenny and I spent a lot of time building and optimizing AI agents. And recently that's been mostly optimizing cloud code for both work and my personal life. Now, in my opinion, context engineering is the most important concept and skill when it comes to getting the most out of AI agents, whether they're commercial agents like Cloud Code, or even ones you're building yourself. So, today I want to talk about what context engineering is, why it's so important, and by the end of this video, you should be in a much better place to get consistently better results from AI, no matter what the application is.
I want to start by sharing this simple but really fun example of efficient context engineering in Claude. So, what you're seeing is a live stream on Twitch of Claude playing Pokemon. And at the top, you can see it's been playing this game for a while. It's got four out of eight badges and it's trying to go through the entire game and beat it without any human intervention. It's 55,000 steps in this process of beating this game. And it's literally just been playing this game by itself without human intervention for weeks. So, you can come check out this fun little example. And the way it works is so simple. It's just writing down its moves and writing down little memories as it plays through the game to markdown files and then just continuously compacting its context window to free up space. So, by the end of this video, you're going to understand exactly how a system like this can work and how you could do something similar uh for your own use case.
Let's quickly define what context engineering is. And I like this definition from Langchain. Context engineering is building dynamic systems to provide the right information and tools in the right format such that the LLM can plausibly accomplish the task. So in other words, it's giving AI agents the right information at the right time so that they can solve the exact task that they're on at that time. Now, don't worry if that doesn't really click yet. I just wanted to throw the definition out there. We're going to get into the specifics in a bit, but let's talk about why we even need context engineering and why it's important in the first place.
The first reason we need context engineering and to really think about how we utilize a model's context is because it's a limited resource. Context windows have grown over time and you can fit way more context into a model now than you previously could. But at the same time, models are getting much better and more capable and so they consume a lot more context. So the requirements for their tasks have also grown over time, right? We want models to help us more, to do more, to work more autonomously and perform more longunning tasks. And to do that, we need to take up more of that context window. So how we use this limited resource is still really important.
The second big issue that context engineering helps solve is this problem of context rot. And the idea is really simple. The performance of LLMs and AI agents degrades as you increase the number of input tokens or the context that you give it. In this paper, Chroma demonstrates several different kinds of tasks that models show degraded performance as the input tokens increases. So, I know the font is really small in this one, but you can just look at the graphs where you have a needle in a haststack experiment where basically the LLM has to retrieve a very specific piece of information in a long text. And you can see as that text gets longer going from left to right on your screen, the performance degrades. Here's another interesting variation where again the model is asked to find a very specific piece of information in a large text, but this time distractors or very similar but irrelevant pieces of text are inserted throughout the document to throw off the model. And you can see going from blue to yellow to red as you add more distractors in the text, the harder it is for models to find the needle in the hay stack. And then I'll just cover one other experiment that I thought was interesting. This long mem eval here we have different cloud models on the bottom and then the yaxis is average score for answering questions. So a higher score is better here. And the blue bars are if you took the full conversation history and passed it to the model and asked it a specific question that was somewhere in that conversation history, you get the scores in blue. And then you have the red bars which are a more focused prompt where instead of the full conversation history, you do some light retrieval to only give the model the relevant parts of the conversation history. So removing tool calls and other irrelevant stuff. And you can see how much better the models are at retrieving the right information. So when you remove irrelevant information, when you reduce distractors in the text, models tend to perform a lot better. Not surprisingly.
And then that gets us to our final issue, attention scarcity. And this one's pretty intuitive to understand as well. Models simply have a limited amount of attention. So, just like us, if we were working on a task, let's say we're writing a report or performing some research, and you have people talking to you in the background, distracting you, or maybe you have a second monitor with a video playing, and you can hear the audio, those distractors would probably make you worse at completing your task. And it's the exact same analogy for models. If they're given too much information, too many things to read and think about, then their attention is spread across a lot of things rather than devoted and focused to what actually matters.
So hopefully now you have a better understanding of what the limitations and issues are with LLMs and AI agents when it comes to working with uh their context window. Now let's come back to context engineering and understand it a little better through this really nice diagram from anthropic. On the left here you have all of the possible context that we can provide to the model. So you might have a knowledge base with a ton of documents like you see up here. You could have local tools or tools provided by MCP servers. a memory file which could be like your cloud.md or your agents.md different sets of instructions so those could be slash commands or prepared prompts you have domain knowledge which could also be packaged as part of claude skills like we talked about in a previous video and then finally you also have your entire conversation history which itself might not be completely relevant to the next message or task that the agent is going to take. So from there, out of all the possibilities of this, what do we actually want to give the model? And again, the more relevant and concise this information is, the better the LLM is going to be at solving the task in this step, whether that's a question you answered or a task that you gave it, simple or complex. So in the context window, we will of course have our system prompt as well as the latest or current user message along with whatever other context we felt was relevant from all of these possibilities on the left. And that's what we're going to pass to the LLM. And at that point, the LLM or agent is going to provide a response, either a simple text response as a message or it could decide to call a tool. Whatever that response is, it comes back to our system over here on the left where again we're going to decide for this very next step what the exact relevant context is that we need to narrow down to stick in the context window. And what I want to emphasize here is that this is a continuous process of curating and refining what context you're putting into the context window. As this loop executes over and over again, as you're talking back and forth with your agent, as your agent is taking different actions like calling tools, it's not just about prompt engineering the system prompt and giving it a role and specific instructions about how it should work. Again, this is a continuous process where we want to have a high amount of control over what's going into the context window at each step that the agent takes.
So hopefully that's clear. Now let's talk about context engineering in practice and how we can use it to get the most out of cloud code. So first we have system prompts or instructions. And in this I'm including claude.m MD files which you might have in a lot of your projects as well as things like output styles. To demonstrate how these different concepts work in Cloud Code specifically, we're going to be using a tool called Claude Trace, which I covered in my last video. So go check that out if you haven't already. But this allows us to look at a full log of everything that's going on within Claude code behind the scenes, including all of the system prompts, the tool calls, the thinking tokens that Claude has. So, we're going to be using this to break down and take a behind-the-scenes look at how these different things work within Cloud Code.
You can see in this trace at the very top we have our system prompt followed by a list of tool definitions. So if we expand the system prompt, this is going to be Claude Code's official system prompt from Anthropic. You can read through this and see how detailed and comprehensive it is. Here you can see the task management system. This is where Claude has instructions to use to-dos to track its progress and all of the work that it needs to do. They provide examples as well as counter examples for when and when not to use this. So, this is of course one of the most important things that goes into the context window for any agent. This is going to have a lot of the role definition, the goals of the agent, as well as all of the tools it has access to, and details about the environment in which it's working. Down here, you can see you have several MCP tools. And here there's a snippet saying Cloud Code can use these tools without requiring user approvals. So when you're using cloud code and you say yes, don't ask me again to use this tool. This is basically where it gets added in the system prompt. Then you've got some environment information about the current directory as well as the system that we're working on. You'll see towards the end of the system prompt, you have our output style that gets inserted here. And output styles are just a way to remove the kind of role and identity of cloud code as a software developer and coding agent and allows you to customize the identity and role for cloud. Here the output style that I'm using is for my personal assistant. So it's just going to define the identity and goals that I want in my personal assistant. And I've done videos on using Claude as a personal assistant. I now made this into a plug-in. So this output style is actually being injected as part of a plug-in, which I'll show you more on in a bit. Now, as part of my output style, you can see I have some information on a context system that I've given my personal assistant. Then after my output style, you can see at the very bottom of the system prompt, there is this context about the git status at the start of this conversation. So this is injected in here by the cloud code harness because they're kind of assuming that you're working within a code repository and probably using cloud code for coding. And so this would be really helpful context if you are using it for that application. But generally for the system prompt, you're going to want to put things in here that Claude should always have top of mind as it works with you. You don't want to put things in here that are only going to be used sometimes because that's kind of a waste of some of your context window. So, an example I've talked about in other videos is if you connect a bunch of MCP tools that it might only use one out of 50 times, then you might want to consider moving those tools to something like a CLI tool where it's not going to be injected into the system prompt, but just giving Claude awareness that it has access to these other tools and it can go read about them if it wants to get more information. So that's the idea of progressive disclosure which we're going to talk about more later.
Following the system prompt, you have a long list of all the tools that Claude ships with. So all of these are default tools that come with Claude. And then you also, if you have connected any MCP servers, you're going to see your MCP tool definitions here as well. After the system prompt and the tool definitions, you'll see several system reminders that are passed as user messages, and they can be used strategically at specific times to give very targeted and focused information to Claude to help it do its task at that point. So if we look at the first system reminder, this is from the cloud code harness where there's a reminder about the to-do list that claude has and the reminder is telling cloud code to use this to-do tool to track its progress. Looking at this last system reminder, this is also where your cloud MD file gets injected. So the cloud MD is not appended to the system prompt. It's actually passed as a user message. So when Claude sees this, it's reading it as instructions that you, the user, have provided. So you can see at the top there are specific instructions that are written by the anthropic team where cloud code is instructed to use this context if it's relevant and telling it that these instructions override any default behavior and that it must follow them exactly as written as well as the source of where this file is which for me this is in my user directory. And then after those initial instructions, you get the actual cloud MD file down here.
So far, those system reminders come from cloud code itself. But you can also add these system reminders yourself using hooks. So here is an example where we have a user prompt submit hook. And this is going to run every single time a user submits a message. So, I'm using this hook to provide information about the context system that I built for my personal assistant. Here, I'm giving it very specific instructions for where it can go to read more about this context system, which also has a memories folder where Cloud is responsible for creating and maintaining its own memories over time. And we'll talk about memory more a little later, but this hook is basically what allows Claude to be aware of this memory system. And we're guaranteeing that Claude sees this at the beginning of every user message.
Next, I want to talk about hooks. And we just saw an example of how we can leverage that user prompt submit hook to provide additional context at the start of a conversation. And don't worry if you are interested in using this personal assistant plugin that I made. I will have my AI Launchpad marketplace in the description so you can go ahead and install it with your cloud code and see exactly how it works. But I want to show you the other place where we're using a hook with this plugin. I just scrolled down to the very bottom of this conversation. So we have this kind of final reply from Claude. So, it's about to wrap up its message and then you see this stop hook. So, a stop hook gets triggered every time Claude is ready to finish its response and turn it back to the user. And just before the conversation goes back to the user, this stop hook will inject this additional message. And Claude will have to either read it or respond and take action before actually completing its response and then turning it back to the user. So here I'm leveraging this stop hook to say, "Hey, before you finish your response, make sure to update the memory and context systems." And then I'm pointing it to the various files that explain the context and memory systems as well as how to update them. So once Claude sees this prompt that's triggered by the hook, you can see the thinking tokens here where it's going to make a plan to update the system. It's going to think about what it actually accomplished in this session that should be recorded. And then it's going to go ahead and update its memory and context system files. So you could easily put the context system and memory instructions in the system prompt. But you can imagine after a conversation gets really long, Claude might not always do it at the end of every conversation turn. Whereas if I want to guarantee that it performs this task of memory maintenance at the end of every conversation turn, then I can do that more reliably with a hook. And these hooks can be set up even more dynamically, making them super powerful.
So, I'll show you another example here where I had a fresh claude code with the personal assistant plugin installed, but I hadn't set up the context system yet. And if I quickly hop over to where I have this user prompt submit hook, we have a couple of conditions for this hook. So here we're getting the path to the cloud MD file in the context directory which is the context system. And if this directory and file exist then that's what we're simply going to return in the hook which is what you saw in the system reminder. But if this cloud MD file doesn't exist that means that we haven't yet set up the context system on the user's system. And so in that case we can pass this other prompt saying that the user has installed this personal assistant plugin but it has not been set up yet and then give it instructions to actually create the context directory and all of those files. So here you can see I have a context template directory. You have an identity file, a preferences file, rules that it should follow, different workflows. So this dynamic hook is basically passing in the directions to set this context system up on your system if it hasn't already been set up. So we can see that in this example where I was just testing it, my message was just high. Here you can see the exact same user prompt hook, but this time instead of telling Claude to read about the context system and memory system, we detect that it actually hasn't been set up yet. And so we're giving it these instructions to go ahead and run this slash command to set up the personal assistant. And that's going to create all of those files on the user's local system. After I just say hi, Claude sees that system reminder and then it it understands that it needs to set up the context system. So it's going to just ask me for permission and then as soon as it gets my permission here, it goes ahead and runs that slash command. So this is a really powerful use of hooks and you can have some pretty complex logic in there.
The next concept is progressive disclosure and I talked a lot about this in my video on cloud skills. So definitely check that out. We've also been seeing examples of that in these hooks where we're pointing to other files or directories on our system where the agent can go and read more. The other place where I think this has a lot of potential is in using CLI tools or scripts over MCP servers. The problem with MCP servers is that you need to have all of the MCP's tool definitions here in the system prompt. So these automatically get injected whenever you connect an MCP server into the system prompt. But instead of doing things this way, you can also create a CLI tool like I did for generating YouTube thumbnails. So if I go to my terminal, I can type in thumb and this can be accessed from anywhere on my system in any project, any terminal. So Claude will always have access to it. But if I run the command thumbkit, then you'll see I get this super compact message that this is a YouTube thumbnail generator CLI. It has a few functions to generate new images, edit existing thumbnails, and you can even look at the full documentation by just typing in docs. And so in the system prompt, I can literally tell Claude, you have this CLI tool called Thumbit to generate thumbnails, run thumb docs to read more about it. That's really powerful because Claude is only going to load the tool definitions when it actually wants to use this tool and when it needs to. if it was to run Thumpkit Docs. Now you can see the full documentation for this tool, including how it should be using it and how it should save files, etc. But this is only going to be disclosed when needed. Now, this is actually a really insane thing to think about because I want you to imagine a world where we simply tell Claude that it has access to many CLI tools via the terminal. And to explore them, it just needs to run this command UV tool list to list the tools. So if we come back over here, I can run UV or Claude could run UV tool list and it would see here I only have three different CLI tools. But you can imagine having tens or hundreds and each one of those CLI tools could have within it hundreds of different tools. So we're talking about thousands of tools. But because of progressive disclosure, it's not going to consume very much in the context window. And Claude can navigate its way through these menus to find the exact tool that it needs without consuming a ton of context. So that's pretty crazy to think about.
Next, we have sub agents. And what makes sub agents so powerful for context engineering is the fact that they have separate context windows from the main agent or from cloud code. So cloud code itself has a 200k context window and the sub aents that it spins up also have their own 200k context window. And sub aents are going to be really good for tasks where they just need to gather information and then synthesize it into a concise uh report or message. So here is an example of a researcher sub aent and Claude is going to call this researcher with a specific task to analyze comments on my different high-erforming cloud code videos. And this is everything that cla code the main agent sees. It calls this task tool to assign this task and then only gets the final output from the sub agent which is going to be its findings here in this report. So you can see it's really concise but if we look at the actual trace down here for the sub aent this is the full trace that the sub agent had. So of course you have the system prompt that it has here. all of the tools that the sub agent has access to, the task that was assigned to it from the main cloud code agent, and then you see all of the tools that it ends up calling. These, of course, are going to be returning very long text results. So, this is already a ton of context that is going into the sub agents window that's not affecting the main agents context. So you see all of the comments that it's pulling and then analyzing and then it's going to go ahead and write that final report and output it to the local file system and then the main agent can just read the final report which is going to have a much more concise high-level findings. So research is a perfect application of sub aents to protect your context window.
Next, we have memory. And we saw how we implemented a really simple context and memory system that's just file-based with our personal assistant plugin. So, going back to one of our traces, once we hit our stop hook, reminding our personal assistant to update its memories at the end of the conversation, it's going to update its current progress in this memories file. It's also going to add some old memories to the archive down here. And that's just going to keep a long running list of its memories. And then it's going to update another file that we have called project index. And this just keeps track of all the projects we're currently working on. So that the personal assistant knows across all your life all of the different things that it's working on. And it has a quick reference to understand where are all the files for this project? where does all the context live along with important information that it can track for each project. So whether you're using a really simple system like this where you just have cloud code write things to like a scratchpad or these memory MD files or you're using an MCP server for memory or a more sophisticated tool. The idea is the same. You want to persist really important information so that Claude can always reference it across sessions and across conversations. And that's just going to make your experience of working with Claude a lot more consistent and reliable and also build Claude's understanding of you and your work style, your preferences, etc. over time.
Finally, we have compaction, which in Cloud Code is going to be a slash command, just slash compact. And what this does is it essentially takes the entire conversation history and then passes it to an LLM with a prompt to just summarize all of the important parts of the conversation so that you can shrink the conversation history into a really short summary. This is really effective to keep conversations going for longer. Of course, the risk is that you lose some details, but what's really awesome is that if you are leveraging a memory system like I am, especially one that is storing the memories on each conversation turn, you can pretty much run compact at any point, and you'll already be persisting the most important details in those memory files.
Ultimately, context engineering is a super powerful concept and skill when it comes to getting the most out of AI agents. So, hopefully you have a much better understanding now of how context engineering works and how you can leverage it to get more out of agents like Cloud Code. And if you are interested in the plugins I talked about today, go check out the AI Launchpad Marketplace, which is where I'm going to be hosting all of the plugins I develop for free, so you guys can install them. There's very detailed instructions here, so you can follow along. Right now, we have my personal assistant plugin as well as a YouTube content strategist and a researcher. So, if you do use the marketplace and find it useful, definitely give it a star. That is going to do it for today's video. Thank you guys so much for watching. Let me know in the comments below what use case you're using Cloud Code for outside of programming or coding. And let me know if there's a plugin that you guys would really like to see as part of the AI Launchpad marketplace. I hope to see you in the next video. Thanks again for watching. Bye.
[music] >> [music] >> Hey.