Agent memory resolved?

So, one of the key limitation factors for cloud code or any coding agent is context management. If you use coding agents enough, you will often notice that agents just seem to become dumber as they work longer and more complex tasks. You'll make the same mistakes and also forget about some stuff that it tried before. This is fundamentally limited by the context window of the learned model. Even though modern models can take around 1 million tokens, the actual effective context window is somewhere between 120 to 200k. That's where all coding agents today are designing this conversation compacting mechanism around those optimal context window limitations. But ideally, you want an agent to be able to remember every single trace it did before.

And even though Cloud Code introduced this memory feature recently, where an agent can log information into its shared memory file of the project, fundamentally the way it works is still somewhat limited. It is basically keeping a separate memory MD file for every single pull. So if your project is actually pretty complicated, this document can blow up very quickly as well. And also, this is very specific to Cloud Code. In an ideal world, you want those knowledge and memory shared across any possible agent sessions, no matter which agent it is accessed to. Only by doing that your team's knowledge is actually accumulating and agents are getting better the more you use it. But this is a hard problem. Many teams attempt to solve this but haven't got an ideal solution yet.

I found this project called One Context that was released this week, where they are utilizing this method called Get Context Controller that is defining a specific memory framework that is extremely simple and basic. So the agent is managing the context just like Git, and this memory persists across any session and any coding agent. It can even be shared with your Open Cloud as a memory file, where you can chat to in Slack or Telegram. And with this method, they found Cloud Code is performing 13% better on software engineer tasks and even enables some cheaper and smaller models like GPT 4.5 Air to perform at a similar level as frontier models. And the most interesting part is that this setup is extremely simple and basic, something you can literally set up now. And this is what I want to share today. I'll take you through how One Context actually works, the idea methodology behind it, and how you can start using it.

But before I dive into that, I know many of you are first-time vibe coders, and when you're just getting started, I know things can be pretty hard. And the best way to learn is by doing and practicing real projects. That's why I want to introduce you to this awesome free resource from The Hustle, where they did a whole bunch of research and came up with 20 AI app ideas you can start building and practicing. They did a whole bunch of research on Reddit and Twitter to find real pain points that people have been asking for. And specifically, those ideas are easy to start and execute, something you can just spend a weekend to build, deploy, and launch to the market. And for each idea, it also gives you a step-by-step breakdown and prompts for generating prompt specs to get Cloud Code or Cursor to start implementing. This should give you a good sense about how to write good prompts and specs for coding agents. And they even give you some ideas about what you should do to start generating your first revenue. And those app ideas span both consumer and business-facing. They even link to a big database of all side hustles that you can start practicing. So if you're looking to actually ship something real and hone your AI coding capabilities with Cloud Code or Cursor, I highly recommend you go check out this resource. I put a link in the description below for you to download for free, and thanks HubSpot for sponsoring this content.

Now, let's get back to how One Context works. So, at its core, the Get Context Controller is basically defined rules or skills that an agent can follow to maintain a list of memory files that are saving all the actions and learnings the agent had at different levels. It has these four main files. One is main.md, which is storing the global context of the project that you are executing. Then they also introduce this branch idea to log different approach paths and tasks that the agent has ever worked on. And for each task and branch, it has these three files. One is commit.md. There are high-level milestones the agent has hit. Like after every time it finishes a subtask or delivers a meaningful piece of work, it will log a commit.md file, just like how we normally do a Git commit. And then log.md is where it will store the full raw conversation history. This is what they refer to as OSAA: Observation, Sort, Actions. And if you're using Cloud Code, it's basically the raw conversation message as well as metadata. Metadata is storing the metadata of the project. Then again, it gives high-level information for the agent to find specific pieces of knowledge.

So, this structure provides the foundation of how memory is stored for particularly long-running tasks, and they have four different actions that an agent can take. One is branch, they should be triggered when an agent decides to explore an alternative strategy. Second is commit. This is what an agent does when they hit certain milestones or complete certain subtasks. And merge. This will be triggered when an agent completes a certain task or exploration and wants to merge this history of exploration by a certain branch. And this is quite important because for complex, long-running tasks, this branch, commit, merge method allows an agent to fork conversations very easily without losing the overall context. And this might be abstract, let's take you through an actual example. So you might have this task for an agent to build a web scraper for LinkedIn. And with these methods, the agent will firstly try to set up a main.md that is logging the global roadmap. It can be just a folder in your repo. Then it will start deciding a method or approach. Like it might decide to build a Playwright script first as a direction. This is where it will create a branch, Playwright. And what does this branch action actually mean? It's just creating a folder called Playwright under the branch folder and starting to initialize commit.md file, log.md file, which should automatically retrieve and save all the conversation history, and then the metadata. So it sets up the foundation about how do we track memory and knowledge of this specific Playwright script-based approach. And as the agent works along this approach, it might hit some milestones like got a V1 script finished and tested. And that's where it will commit a memory. And what this commit action does is basically instruct the agent to update the commit.md file to log a summary of what it did, and also has the option to revise the main.md file. And the benefit of this is that you basically start having this log and source of truth of actions that the agent is taking. And after finishing, it can run this merge command. And what this merge command does is that it will try to merge the commit.md file into the main one and combine the log.md file as well as update the main.md file to provide a high-level summary.

So, this is like a basic flow of how an agent should maintain memory in this structure, and it can be much more complicated and flexible. For example, an agent might find this approach of a Playwright script is not correct. Instead, it should use API instead. Then it can create a branch for API exploration and maintain all the learnings around that approach in that specific branch folder. And the interesting part is that once we set up the file structure like that, the agent is also able to retrieve information progressively. If later we ask it, "Hey, help me understand where we are at in terms of this LinkedIn scraper project," it can firstly look at the main.md file to understand the overall project scope and what kind of exploration has been done. And if it decides to dive deeper, it can fetch specific branch information, looking at the commit file to see for the Playwright approach, what are all the different commits and milestones we ever hit. And if there's anything interesting, it can also dive into the raw conversation history of a specific commit by looking at the log.md file.

So, this is the whole setup. It's basically these four commands plus a file structure that is maintaining the agent's learning and allows Cloud Code to achieve 14% more performance by just logging and feeding relevant actions that the agent is taking along the way. And just later this week, they also introduced this Context Repositories feature, which is a very similar concept to what we just talked about here, except it is maintaining a different structure of memory. But the overall approach is very similar, using the file system to create multiple different levels of information. So any coding agent can access memory and information progressively. And the most interesting thing is that it works across any sessions and any coding agent. That means for any project, we can spin up multiple different agents working on different things, and they all share the same context and memory in real-time about what the other is doing so that they can make decisions better.

And the author of Git Context Controller basically introduced this tool called One Context that encapsulates the overall methods that he just showed in the paper into a command-line tool that you can run here and share context across different agent sessions. And this is what I want to show you how it works. And meanwhile, if you're interested in agent memory and context management, we have a few workshops in AI Builder Club where we share practical tips and learnings of how to best set up your coding agent memory MD files, as well as tricks for building production agents, alongside step-by-step tutorials for how to set things up. So, if you're interested, you can click on the link below to join.

Now, let's set up One Context together. So, firstly, you do `npm i -g one-context-ai`. This will install One Context on your computer. Then you simply run `one-context`. This will open up a view like this, which will give you a split view. On the left is the agent session, and on the right side is the actual agent. So, the first thing you will do is that you will want to add context. So, for context, you can consider it as a kind of group of information that you can share with others later. So I can click "Add Context," and here I can give a name, and it could be anything, to be honest. Each context is considered as a group memory. Later, they will allow you to share this group of memory with others, but the easiest way could be for each project, you can set up one memory. So let's say I call it "demo-project," and then we can start adding a new session to this context. So I can select a coding agent, it can be Cloud or Codex, and then I can choose the workspace. Click "Create."

Now, here it opens a new session on the right side. So if I ask it information like, "What's my name?" it wouldn't know this information. But you can see here, after finishing, it is showing that it's running this stop hook. And on the left side, you will see this session already generated a quick summary called "Username Inquiry and Identification." And if I hover, this context already has some high-level summary. And now, let's test again. In here, I tell it, "My name is Jason." Okay. So, the name has been remembered. Even though here it ran memory, which is using Cloud Code's own memory, and we don't want that. So I'm going to tell it, "Remove that from memory. We don't need it." Okay. So now we're making sure it is not part of memory. And just to double confirm that, I'm going to open the `docloud` folder, and I can see in that specific project folder, the memory is empty.

And now I can create a new session. And this time, I will even try something new. I'm going to add a new folder called `new-one-context-demo`. And I will ask it, "What is my name using One Context?" And you can see it is starting to run command-line alignment search. Align is a company behind One Context. And it is basically doing the search in memory and returning back information. "My name is Jason." So you can see it is totally across different folders, but it is still able to share the same information. And what's really cool about this one, you can imagine, is you can use the same mechanism and get agents to explore totally different directions but still share the same memory about what's going on.

And to understand how this information is actually saved, if you're using One Context, you can do `code align`, which will open this folder. It has this `skill.yaml` file, which includes context for the agent about how to use it. And it has very specific context here that it will firstly do a broad search to search a specific query within that context folder. But then they can also narrow down the scope by passing `-s`, which is a specific session information, or a specific turn. And if they really want, they can also dive deeper to do a search on a specific turn to look into the actual conversation of a specific session. And all that knowledge is saved locally on your `db` folder here called `align.db`. But it's basically replicating your actual Cloud Code conversation history as well as summary information. And the way it works is that every time you add a new session here, this session will be logged and captured by a watcher service and automatically has a stop hook. And the stop hook will run to save that new conversation information into the DB here, as well as triggering a large model call, which by default is using GPT-4 Mini to generate a summary of the information.

And just to demonstrate a real-world use case, let me add a new context to be "memory-research." And here I can set up two sessions and open another session with Codex. And I'm going to ask it to research about different memory setups. So here I'll ask it, "Help me investigate and learn how does Open Cloud memory mechanism work." And while it is working for the Cloud Code session, I'm going to ask, "Help me research and investigate how does Let's Context Repositories work." And I might add another session here to say, "Help me research how does One Context memory mechanism works." Okay. So now each session finishes research: one for the One Context memory mechanism, one for Let's Context Repository, and another is for the Open Cloud memory setup.

Now, let me add a new session. And now I'm going to tell it, "Remind me how different agent memory setups work across Open Cloud, Let's, and One Context from our past exploration. Don't use the internet, just from our own memory, and give me a breakdown of pros and cons." Now you can see what happened is that it will firstly search for relevant sessions. Firstly, try to search the memory, which will return a list of relevant sessions within the project. And each session basically means the title and the description that we have here. Then it starts looking deeper into each session and each turn. In the end, it returns me each method and its pros and cons, and in the end, a summary comparison. Even though it does seem to make some mistakes, the Open Cloud somehow, it says it does `cloud/per/memory`, which is not exactly true. Let me check if it's because we researched the wrong thing. Oh, no. Okay. It looks like there's something wrong about retrieval, probably because Cloud Code also has its own memory. But I'm sure this can be improved.

What's really cool is that you can see this very effective way for an agent to retrieve that and look at what has been done collectively, not just one session, but across all the sessions. And if you have teams working together, this becomes a really good tool since all of you can contribute to the same knowledge graph. And they actually have this share button as well. So I can click on that. It will ask me to confirm, but once confirmed, it will return me a link, which I can copy the link. This basically creates this URL where people can talk to. This is like a simple chatbot, but loaded with the conversation context that we had here. So I can ask here, "How does Let's Context memory work?" It will do the same thing, search across different sessions and retrieve information, and then it will return back this summary learning, which looks correct. Obviously, it's like the whole interface is very bad, but it helps you understand how did this memory system work. But the core thing you just need to remember here is that fundamentally, the way it works is a file system to store and summarize abstract all the past actions, which you can easily set up and achieve your own skill for your Cloud Code and save their memory locally. And if you're interested, I can show you a setup I personally have next time. That's it. I hope you enjoyed this video. Thank you, and I'll see you next.