Transcription
It happens that you are sitting with Cloud Code, trying to implement some feature in a project, you've almost finished, and suddenly you realize that the context window is running out. And there's auto-compact, which helps with this, but after it, the model becomes much dumber, and you want to implement everything within a single context window, but it's not always possible due to the large volume, for example, of the codebase. In this video, we will look at one of the ways to solve this problem using sub-agents. Let's go. First, let's take another look at how the context window is formed when working in Cloud Code. We have a general window of 200,000 tokens, which is divided into three parts: system prompts, tools, and other instruments, and the auto-compact buffer, which is at the end. And in the middle, we are left with approximately 99 to 135 thousand tokens for the window, within which we can work. And this window is usually filled with user requests to the model, the model's reasoning, and answers, reading some files, well, and other information that accumulates during the process of requests and answers. And part of this information is not always relevant for solving subsequent tasks. For example, we asked the model to implement something in the code. The model looked at the file, the content of this file got into the context itself. The model, based on this content, wrote new code, but the entire history remains in the context window and fills it, even if this information is no longer relevant. And the question arises: can we remove part of this information from there in order to free up space for the next tasks? And the answer to this question is not so obvious. There are two main ways: auto-compact or simply compact, and the use of sub-agents. Let's first look at compact. Suppose we have a filled context window of 140,000 tokens, and we have 14,000 tokens left until auto-compact. And we execute the compact command. This slash command sends a request to the model to archive effectively the entire context window with information loss. When this slash command is launched, a request is sent with the entire context from the previous meeting and the request: "Please summarize, blah-blah-blah," and an example. And the model, accordingly, responds, and this summarization then goes to the beginning of the context window when continuing to work with Cloud Code. Schematically, this can be shown as follows. For example, our context window is filling up. The user writes. The model looks at the entire context and compiles summarized information about this dialogue. After that, Cloud Code clears the entire context window and puts the summarized information about the past conversation at the beginning. And then the user can also continue the conversation, and the assistant can respond until the next iteration occurs, when the information currently in the context window is summarized. The problem with this approach is that when compressing information, obviously, part of it is lost. And if some information in the previous context window was quite important, and the model missed it, then this information may not be in the new context window. And in this case, we lose a lot in the efficiency of this model's work. One way to avoid compact is to keep the context window in the current iteration clean enough, leaving only the information that is important for solving the overall task without exceeding the allocated context window. Sub-agents can help us with this. What are sub-agents? They are essentially another instance of Cloud Code within our running Cloud Code. We can ask our model to allocate a part of the work and send it to a separate sub-agent with a separate context window. For example, to work with some large log files or a large codebase, we can ask the model to launch a separate sub-agent, create a description of working with this task, add all the necessary context only for this task, and transfer the work with the files to this model. After launching, the sub-agent will perform the work in its own context window, adding a chain of reasoning, which can be quite extensive. We can also respond to the sub-agent's requests in its context window and work with it like a regular context window, except that initially, the system prompt and the set of tools can be adjusted for each sub-agent individually. And on average, this reduces the occupied space by these overheads. Plus, the auto-compact buffer is also much smaller by default. Therefore, the main context window, within which reasoning can occur, expands. And after its work, the sub-agent returns the answer to the main context window, with which the user can then continue to work without cluttering the main window with unnecessary contextual information. It is also important to note that we ourselves, as users, can create our own sub-agents and form a system prompt for them so that they act within a specific context. We will create our own agents a little later and see how they can be configured and what tools are available for this. And Cloud Code has some set of sub-agents that we can use out of the box. One of them is Explore, which uses the HQ model and is often used to explore, for example, a codebase, read files, and summarize them in some way. There is the Plan agent, which creates plans for implementing, for example, new features or complex tasks, which it tries to break down into smaller ones for execution. There is also a non-specialized sub-agent, which does not have any specific context, and it is often used simply to isolate a task into a separate context window to preserve the main one. There are also other types of agents with their own context, with their own tool settings, but this is the main list that is often used. Let's now look at where these are described in Cloud Code. As we know, Cloud Code has a set of tools for reading files, executing bash commands, writing files, and so on. And the first in the list is what is called Task. So, here the principle of operation of all sub-agents is described. First, there is a description of the agents, what each of them is used for and what tools are available to them. Some agents, for example, Bash, have Bash available, and Explore, for example, has all tools allowed except Task. And here are listed all built-in agents plus those that we have created ourselves. Here I have created three custom agents that have their own description, what they should be used for, examples of their use, and what tools are available to them. This particular one has all tools available. Next is Project Manager, also with its description, with examples of use. And all tools are also available. Frontend Expert also has a description. It even has a description in Russian. And all tools are also available. After that, there is a description again for all agents in general. I have now deleted all agents that I had created locally. And let's see how we can create new ones. For this, we can use the slash command agents. And now we see that we have built-in agents, bash, tool setup, explore, plan, and so on. And we have the option to create a new agent. We can choose where we create it, locally in our project. This is convenient for, for example, committing it to Git and using it with the rest of the development team, or in our personal home directory, so as not to mix the project itself with Cloud Code settings. Now we will create it in our personal directory. And here we are offered two different options. Either with the help of Cloud Code, or manually. Let's create it with the help of Cloud Code for convenience. Here it asks to enter a prompt to fill the context, which will then go into the system prompt. Here we entered the following prompt for the example. We send it to the model, and inside it, it creates the main text that will go into the sub-agent itself. The model has finished its work. And the next request is to enable or disable access to tools. You can enable all tools, only those for editing, for execution, other tools. And you can see the specific tools that we have. Let's allow all tools and select continue. Here we have the option to choose the model itself. Three options: Sonnet, Opus, and Haiku. And the fourth option is to inherit from the main model that this sub-agent runs. Let's choose Opus here, as it's a developer, and the color that is displayed in the interface. And here we see the main information about the created agent: its name, Senior Frontend Architect, where it is located, the MD file, what tools, what model, description, and the system prompt that will be used when launching this agent. We save it. And here we see it. We can edit the tools, the model, what we filled in. And let's see how it looks in our file. Well, in the file, it looks like this. We have a Markdown file with frontmatter, where in a separate field, we have the name, description, which the model formulated. Here we have a mix of English and Russian because initially, in this skill, the model received a description in English, and part of it in Russian it received from us. The model is specified, the color is also specified. And here is the main body that goes into the system prompt when launching this agent. And about dynamically loadable tools, I talked about them in the previous video. MCP tools are available, but they are not loaded into the context window immediately. It is available for dynamic loading. And earlier, when launching sub-agents, all these tools were automatically connected. But in version 2.1.15, which is current now, this error has been fixed. And when launching sub-agents, here we see that the sub-agent is launching and what tools are available to it. Tool search, formerly called MCP search, and two additional tools. And here, the automatic tools for our GitHub are not loaded. They are available to it. It can still load them. Here, in the description, we can see them, that it can load them. But they are not loaded by default. In the previous version, this was not the case, and they were automatically loaded here. Also, speaking about agent settings, it is important to mention what other means we have. As mentioned, there is a setting for tool access. Here we have set read, write, and bash. If we restart Cloud Code and look at the agent, we will see execution tools and we will see that bash, git, grep, read are available. We can also allow one of the operating modes with permissions. Default allows editing, do not ask. Do not ask means the agent will not be able to ask additional questions to the user, where we can choose 1, 2, 3. Or everything is allowed, like Yolo mode or Plan, which is used for plans. Let's try to enable Yolo mode, restart, and see how it looks. Launch the sub-agent and update the architecture in the README. Well, here we see how Cloud Code invents writing to files, and it has bash available, but not write, so it creates a file in TMP, writes text to it, and then moves this file. During this process, it did not ask us a single question. Accordingly, all these tools work. You can also specify in the frontmatter which skills should be loaded into the context window of this sub-agent's work. This is useful if, for example, the agent needs to work with a task tracker. You can immediately load the rules for formatting tasks, rules for formatting commits, if our agent is about working with Git, for example, and others, so that it doesn't go around searching if we know for sure that these skills will be useful to it. Here we can list them for it. I plan to shoot a separate video about skills, I will explain more there. Another possibility that sub-agents allow is to run them in parallel in two, three, or more threads. For example, we can ask Cloud Code to explore our codebase from different perspectives. Use sub-agents, explore the code, and give me a list of dependencies separately for the frontend and for the proxy. Here we see that it launched two Explore sub-agents, one for UI dependencies and one for proxy dependencies. This can speed up operations like exploring a new codebase or, for example, code review from different perspectives. Well, let's break down the pros and cons of working with agents. The first obvious advantage is the preservation of the context window. We can move routine work on a task to a separate window and keep only what is important for the global task in the main window. The downside is the complication of the flow with the need to decide what exactly to move out and what to leave in the main context window. As a second advantage, I would highlight permission management. We can immediately grant access to certain tools in the sub-agent without asking, and it will automatically work with them. The third advantage in context management is analogous to the second; we can clearly define the context required for working with a specific task for this sub-agent. As a second disadvantage, I would mention that when launching an agent with a clean context window, it often lacks some information that it has to go and collect separately from files or request from the user, which takes additional time and tokens. The fourth advantage is the ability to run multiple sub-agents in parallel on one task from different angles. This is the main information about sub-agents that I use daily. In fact, sub-agents are a rather broad topic. If you have any remaining questions, write them in the comments. I will shoot a separate video. Subscribe to the channel, there will be videos on other topics here too. Goodbye everyone. M.