Transcription
Everybody talks about agent harnesses, but what exactly is a harness? Now, even people who are actively building agents can't always give you a clean answer. Uh, the word gets thrown around constantly, but nobody really agrees on what exactly it means. So, in this video, I want to do three things. First, define what a harness actually is and just as importantly, what it's not. Then we will walk through nine components that I think make a modern harness. And finally, we will build a tiny one in Python so you can see exactly what is going inside. This is going to be especially important for people who are thinking about building agents and harnesses.
In simple terms, a harness is a fixed architecture that turns a model into an agent. So if you think about modern LLMs or models, these are just one short text generators. You ask a question, it answers and stops. A harness is what gives it the ability to take action, see the consequences, and keep going until the problem is actually solved. So think of um a model as the engine and the harness around it as the car. That's what makes an agent. So a really good example of this is um agentic coding tools like Codex, Cursor, uh Codium, Windsurf. These are all harnesses. Each one started from a concrete problem making a model write and edit code across a real repository. And I think they have uh converged on remarkably similar architectures.
Now we're going to look at that architecture in a minute, but first I want to talk about something else that you probably have heard about, and this is frameworks. So think about things like Langchain, LangGraph, AutoGen, CrewAI. These are not harnesses, and I think this distinction is really worth making because right now people are using these terms interchangeably, and it's kind of getting confusing. So a framework gives you abstraction. Uh, think about state machines, chains, memory, agents, and retrievers. You, as a user, have to wire them together. The fundamental assumption is that you, the human architect, will configure these pieces together.
Now, a harness, on the other hand, uh, is from the opposite direction. There's no assembly step. It basically uh ships a working agent, and in simple terms, it's just a while loop with a tool registry and permission layer, and everything comes wired together. Now, another way to think about this is that a framework is built for a human to assemble an agent. A harness is built for the agent itself to do a task. And in the big picture, you just provide the goal. The harness will handle the rest. So in the rest of the video, we're going to primarily focus on harnesses and what makes them interesting.
Okay. So what exactly is inside a harness? I would say there are nine main components that you need to consider if you're building an agentic harness. Now, we're going to uh go through the list. This is mostly an opinionated uh architecture, but something that I have seen to work really nicely in practice. I'll try to tie it together to Codium because I think this is um an example of a really great harness put together.
Okay. So the first component is the while loop. Um, this is basically the foundation. It's the outer iteration loop. The harness is, at its core, a while loop. The model reads its system uh prompt, decides which tool to call, runs the tool, feeds the result back into the context, and loops again. And this process keeps going until the uh model produces a text-only response or it uh hits a maximum iteration cap. Now, we're talking about uh text-only models, but the same can apply to multimodal models as well. So think of this outer loop as the whole engine that runs everything.
Now, number two is context management. On every turn, the tree grows. Um, as you encounter more uh user messages, more tool calls, more, you're going to see that uh you hit the context limit of your large language model. So the harness has to decide what to keep verbatim, what to summarize, and what to throw away. In Codium's example, the budget used to be around 200,000 tokens. Now they have increased it to 1 million tokens in case of Opus. But let's say when you are reaching almost half of it um or maybe 80 to 90%, it triggers a compaction. Some of the most recent messages are going to stay in full. Everything older gets summarized. Now, this compaction is very important um and it can have some real bad consequences if not done um properly. So you need to be very careful about context management.
Now, the third component uh is skills and tools. So tools are the primitives that read a file, a data file, run bash, search code. Skills are a top a layer on the top. So they are how organizational knowledge gets encoded. Uh, usually you're going to see them in markdown files. Now, to think about u tools and skills, I would say tools are universal. Uh, skills are specific to your team, your workflow. And then there is the registry. Uh, so it tells what is available, what permission each uh thing needs, how the call gets dispatched.
Now, the fourth component is sub-agent management. Now, at some point um a task gets too big or too parallel for a single conversation thread. So the harness uh is going to create sub-agents that work in isolation. Each sub-agent gets its own session, its own restricted set of tools, and a focused system prompt that says, uh, you're working on this specific task. Now, the idea over there is to span, restrict, and collect uh the outputs. That's kind of the pattern you want to use.
If you want to learn about generative AI in a structured way, you will love today's sponsor, which is Coursera. They have a number of different courses on generative AI. Here are the four that I highly recommend for you as a beginner. Start with Google AI Essentials. It's about 10 hours, taught by Google's own AI team. Over 1.7 million people have already enrolled. Next, Wonderbuilding's Prompt Engineering for ChatGPT. This is where you learn the actual patterns: Chain of Thought, Few-Shot, Persona Prompting, etc. If you are a developer, jump to their Generative AI with Large Language Models built with AWS. It's a hands-on lab. You fine-tune, deploy, and ship real LLM applications. And if you want a full career ramp, IBM's AI Developer Professional Certificate is for you. You get to build chatbots, apps, RAG, and agents. Right now, get 40% off of three months of Coursera Plus through the link in the description.
Now, back to the video. Number five is built-in skills. So, we already talked about skills um that you provide as a user, but every harness ships with a baseline set of kits that are going to work out of the box. So, think about file operations: read, write, edit, search, or shell execution, um code navigation, things like that. Now, for modern harnesses, these are really non-negotiable. If your agent cannot read or edit files, it isn't a coding agent. So beyond the primitives, modern harnesses also ship with higher-level skills. Uh, for example, a harness can have a skill of how to make a Git commit, how to open a pull request, how to run test results. Now, some of these built-in skills are going to be uh specific to the vendor or creator of the harness.
Number six is session persistence or memory. So, a long agent session is stateful. If the process crashes, you lose everything unless the harness writes state to disk. And the most, the way modern harnesses do this is pretty elegant. So typically, uh, they're going to use append-only JSON files or maybe markdown files. So every message, every tool result, every compaction event gets one line. Now, the beauty of this is that you can resume exactly where you left off. U, I actually recently attended a talk from the Anthropic team where they were discussing about how they built managed agents. Uh, I'm going to cover that in another video, but they had the session uh management separate from the harness itself, which was, I think, a very interesting design.
Number seven is system prompt assembly. Now, this is the one that will surprise most people. The system prompt is not a static string. It's basically a pipeline that walks um ancestor directories looking for specific types of instructions. So if you have cloud.md or agents.md, it's going to inject those into the system prompt. Now, you also want to be a little careful here um because most these um third-party harnesses or even the first-party harnesses have really strict prompt caching. Now, if you dynamically introduce uh components to the system prompt, that is going to break the caching, right? So you need to be careful about that, but in certain situations, you want to uh assemble the system prompts.
Okay, number eight is going to be lifecycle hooks. So this is extensibility. Hooks let you inject custom logic before or after a tool runs without touching the harness itself. So a pre-tool hook fires before execution. It receives the tool name, the input, and can allow, deny, or modify the call. A post-tool hook uh runs after and can inspect the results. So the protocol is kind of, think about uh a JSON file with exit codes for allow or deny. Now, the beauty is that hooks also enable intercom communication between different harnesses, and hooks are how enterprises today adopt harnesses themselves.
Now, let's talk about number nine, permissions and safety. So this is the layer that makes the difference between a useful tool and a dangerous one. Modern harnesses define a hierarchy of permission modes. You can have read, workspace, write, um full access. Each tool declares the minimum permission it requires. Now, the job of the harness is to enforce that at dispatch time, before the tool even um ever runs. And for tools like bash, the harness even classifies the commands dynamically. So let's say, u, if you say list files, uh, it's going to be read-only. Uh, if you want to delete something, that will need full access, and the harness figures it out by passing the command string on top of the static permission. You get interactive approvals. The agent can pause, ask, "Should I run this?" right? So before executing anything dangerous, you want to have this safety layer built into the harness.
All right. So these are um the nine uh components that I think every harness needs to have: iteration loop, context management, skills and tools, sub-agents, built-in uh skills, session persistence or memory, system prompt assembly, lifecycle hooks, and permissions. Now, the easiest way to actually understand um a harness is to build one. So, let's write a minimal version in Python. Um, nothing fancy, just enough to see all these components together.
Okay, so in this part, we're going to quickly go over a reference implementation. Uh, so think of this as a structure or template that you want to use if you're building a harness. Okay, so the main engine is the while loop, uh, that basically controls everything. It assembles the system prompt and starts looping. Now, on every iteration, um, the context is going to be compacted if it grows too large. All of these things, plus the tool calls and uh calling to sub-agents, are going to be implemented within this while loop. You also want to cap how many iterations are going to be in this loop so that it never runs forever. This is really the entire engine. Uh, every other file in the project exists to support these few lines.
Now, uh, this code implements simple context management. Um, in in this very simple form, we are just doing compaction. So if the history uh grows beyond a certain point, we just summarize uh some of the older conversations and put them together. Now, there are more advanced uh compaction techniques, but this is a very simple reference implementation. Now, if you're making tool calls, you also need to decide whether you're going to bring in everything that is done within the tool call or only the input and outputs. So, those are the design decisions that you'll have to take as an architect.
Okay, this code uh implements a simple tool and skills registry. So every tool in the harness is described by a small um data class: a name, what type of permissions they're are going to have, and a handler function, and a one-line description. The registry is just a dictionary that maps the tool name to that record. Now, there are a few functions. Calling register adds a new tool. Calling get u retrieves one for dispatch. Calling descriptors returns a lightweight version of the list which is going to contain name, permissions, description that we are going to send to the model so it knows what is available. Skills are registered um the exact same way. They are just tools whose handler reads a markdown file at invocation time.
Next is sub-agents. Uh, so you can implement multiple different sub-agents. This code looks at three different things for exploration, general, and then verification. Now, each archetype has its own permission levels, its own restricted tool list, and its own focused system prompt.
Now, every uh harness also needs to have built-in primitives. These are the non-negotiable tools every coding harness must ship with. Example of this would be reading files, running bash commands. Now, this also depends on uh the type of work you want your agent to perform. Now, in this case, one thing to keep in mind: these primitives need to use pure standard libraries. You don't want to rely on framework dependencies, u which is going to be critical because that actually enables the model to take actions.
Okay. So here is a simple reference implementation for um session memory or uh context persistence. Now, every event the agent generates gets written to disk as one line of JSON. You can also use markdown, but JSON seems to be uh the better choice. The append method opens the file in uh append mode, writes the event, and immediately flushes it. That way, if the process crashes after the next line, this one is already safe on disk. The replay method reads the uh file back line by line and reconstructs the full session because the file is append-only. Two runs of the harness can share the same log without stepping on each other. If the harness dies, the file does not. This is the whole durability story. If you want to persist memory.
Okay. Uh, next one is system prompt assembly. Uh, so you don't have to have a fixed system prompt. You can actually dynamically uh load things into the system prompt. So, for example, uh you can load agents.md, cloud.md, or any other memory files that you have stored into the system prompt dynamically just by reading a directory and reading files from disk. One thing to uh be aware of: the order matters here. So keep the static part first, and then dynamically load content second, otherwise you're going to break the prefix caching.
Okay. So next one is hooks, which are used for extensibility. Now, there are two different types of hooks. One is pre-tool hook, and the other one is post-tool hook. And the idea is the pre-tool hook fires before any tool runs and can u either allow or deny the call. A post-tool hook fires after the tool runs and sees the output. U, it cannot block anything. It's there to audit, can be used for logging and observability.
Okay. The last component is permissions um and safety. So each tool declares the minimum permissions it needs. U, it can be read, workspace, or full. And now the harness needs to uh provide that extensibility and uh control the permissions of every tool. Now, there's one more thing that you need to be aware of. The same tool can be safe or dangerous depending on the command. So we uh classify it dynamically. Safe commands like list, concatenation, and grep uh stay at read-only. Dangerous commands like uh delete, sudo, or shutdown jump straight to the full access. So anything else gets workspace level. On top of uh these static rules, the agent can also pause and ask the user for explicit approval before running anything destructive. And this is the part that you need to implement within your harness.
Now, these are the components that I think every harness needs to have. Now, do let me know your thoughts. What type of of uh components your harness have? And if you're interested in technical content like this, make sure to subscribe to the channel. Anyways, thanks for watching, and I'll see you in the next one.