📱

Get Our Mobile App

Take your business learning on the go!

Download on the App StoreGet it on Google Play

New AI coding paradiagm - OpenAI Symphony

AI Jason12:34

Transcription

So, OpenAI just released this open-source repo called Symphony. On a surface level, it looks like an orchestrator that allows you to manage coding agents through ticket trackers like Linear, but it is a lot more than just connecting Linear. It's a totally different way of interacting with agents.

So, the way we use coding agents has shifted a lot for the past few months. From initially just auto-complete to primarily interactive sessions with coding agents, to now most of us having two or three different sessions in parallel, each working in an isolated work tree for different features or bug fixing. And then new tooling like Super set or Conductor that has been introduced to help you run and manage different interactive coding sessions easier.

The problem is that even with those tools, many people, including myself, will feel this burden when we are working on more than like three different sessions, 'cause we just can't context switch every minute. And I personally have had multiple times sending the wrong instruction to the wrong thread. So, the ceiling of how much we can get out from those coding agents is no longer the model capability, but our own attention and cognitive load.

And the reason the recent project Symphony is so interesting is that OpenAI's engineering team had this realization that the current experience has been orienting around coding sessions, merge PRs, but in reality, for the past decades, software workflows are largely organized around deliverables, things like issues, tasks, tickets, milestones. Engineer leaders have been managing a massive amount of tasks across thousands of workers, not by reviewing everyone's PR, but looking at final outcomes using tools like Linear and Atlassian.

And OpenAI's proposed solution is to move humans up a level. Instead of managing two or three interactive sessions, you manage tickets. The agent works at the ticket level, reports back through the ticket itself, and you stay in the loop without monitoring individual sessions. The ticket tracker becomes a state machine itself.

And the way Symphony makes this work is almost embarrassingly simple, but very effective. It's a background process. You run it once, point to a workflow file, which we'll talk a bit more, and then it runs forever. Every 30 seconds, this background process will glance through your Linear board. If it finds any ticket in the "to-do" slots, it will set up an isolated workspace and start an agent in that workspace.

And the whole system has three key components. One is the scheduler, the background process that is pulling ticket data and setting up workspaces, managing session lifecycle. And a workflow.md file that lives inside your repo. It contains configuration of the scheduler and detailed instructions for the coding agent to know how to work with those ticketing systems. And those external systems like Linear are a durable state machine for humans to interact with agents.

And this whole setup is actually very flexible. You don't have to use Linear. You don't have to use Code X. You can actually customize it to whatever you want. But the overall implementation concept is what's interesting.

And the most interesting part is this workflow.md file. It basically breaks down into two parts. The top part is the YAML front matter. It configures the scheduler directly, like which Linear project it is, what type of ticket it should pick up, where should the agent create an isolated workspace, and even programmatic hooks to run after it sets up the workspace. And this is very useful, so you no longer need to rely on the agent to set those things up. As well as how many agents can be run in parallel and specific agent settings.

And after that, the bottom half is a markdown file. This is the prompt agent every single turn details rendered in. It's a standard operating procedure for handling tickets in this repo. How should the agent plan tasks? How should the agent go validate its work? And what would be considered as done? And when should it outreach for human review?

And what I love about this design is that the same file just lives inside your repo, so it's version controlled and can be changed through normal pull requests. And the file itself contains some programmatic rules that control the scheduler and also what an agent does. There's no separate config service, no admin panel, no UI at all. And the team only codes in this workflow. So, when you onboard a new agent capability or add a new step in the process, you just very easily change this markdown file, and the rest will just follow.

And this whole system is designed very flexibly. You don't have to use Code X, and you don't have to use Linear. They have one example implementation in Elixir, which is a programming language. But they have this spec.md file that's detailing how this framework or system is designed. So, you can just drop this file to any coding agent and ask it to build and design a system in any programming language. There are already a lot of different community attempts. Like someone building a custom TUI based on the task data. And also another person already rebuilt it to support Cloud Code as agent harness.

And I'm going to show you step by step how you can set these things up. But orchestrating agents is only part of the work. As OpenAI mentioned, this whole thing only works if your coding agent's environment is set up properly in a way that it can complete tickets end-to-end atomically, which you can call it harness engineer, but fundamentally just whether your environment or codebase has been set up in the right way, so the agent has everything it needs to complete the task end-to-end.

And typical things like, is the system bootable, so the agent can just run a script to get everything set up without spending time to figure that part out. And does the system have a proper documentation structure for different things. And I think most people do have these two things properly set up in your code.md or agent.md file. But the part I think most teams didn't set up is those self-verifying tools. They allow the agent to do an end-to-end test after implementing something. And even submit a video recording to prove that it has tested and it's working in the ticket directly, just like in their demo.

But in the docs, they didn't really mention how they were handling this part. So, I did some research across many major skills. And the best one I found is this Playwright CRI tool. So, I believe many of us are pretty familiar with Playwright MCP, which allows the agent to use the browser and do a task, check the logs. But the problem before was that Playwright with MCP setup, it took a huge amount of tokens in the context window even when it's not needed. But they have released this Playwright CRI tool alongside agent skills that detailing every single command.

And the most interesting command is this video recording CRI. So, Playwright allows the agent to run commands like "video start" and "video stop" to capture browser sessions into an MP4 or WebM video. They even have some pretty advanced video rendering capabilities where they can add different chapters on the screen. Like here's one example video where it can record its own session and even add new HTML elements on top of the screen to annotate the actions the agent took. And then upload the session into Linear, so you can very easily verify if things actually work. And as far as I know, other tools like Chrome DevTools MCP or Agent Browser don't have this video capability out of the box. So, this is one very important skill that will make your whole experience complete.

And meanwhile, there are also other skills that you should add. And I just take one of the repos I have as an example. We have this Playwright CRI tool that has a skill as well as a list of references for the agent to know how to like record a video and trace the debug logs. And we also have a skill here to tell the agent how to start a server locally. And because ours is pretty straightforward, so it's just a skill file. But sometimes for more complicated things, you can create predefined scripts as well. So, the agent no longer spends cognitive power on those types of stuff.

And meanwhile, I also created this Linear skill that allows the agent to know how to operate Linear tickets by using the Linear API as well as things like uploading video evidence of the test. And we actually have more documentation about different parts of the system. And in the agent.md or cloud.md file, this is where we have a proper index of different documentation systems, so you can always go and find the relevant information. We also give more detailed debugging skills. For example, we use Grafana to track and store all the logging in production. And we add a relevant Grafana log skill in our repo, so the agent can fetch real production logs for bug fixing.

And all those things are trying to serve one purpose, which is setting up your codebase so that your agent can fix bugs, build new features, verify things are working fully atomically end-to-end. I put all skills inside AI Build Club, so you can copy-paste and ask your agent to customize for your own codebase. I put the link in the description below, so you can join and access. And once you set this up, even though you don't use Symphony, they're still going to be really useful.

But after that, this is where we can start setting up Symphony, connect to Linear, as well as this workflow.md file. So, once you clone the Symphony repo, you'll see folders like this. You'll have this folder of Elixir. So, this is one version implementing the Elixir programming language from OpenAI. And most of the time, you can just use this Elixir directly. But if you want to customize it to like connect not Linear, but connect to Trello or Jira, you can ask a coding agent to customize it or even build a different language by pointing to the spec.md file.

And here's basically what I did in the Python folder. I just pointed to the spec.md file and asked it to build a new version in Python. But most of the time, you actually don't need to do that. You can just reuse what OpenAI provided.

And firstly, you can confirm whether the script is So, you can run the script by doing this, which points to the Symphony program that has been built. And run help. So, this should show you the actual commands about how to run Symphony. You basically just do Symphony and point to a path to workflow.md file. And by default, you can't just run Symphony like this. You can run this to bind the Symphony command to the specific path. So, just run this. And then you can do Symphony, point to a specific workflow.md file. And by default, it will give you this warning. Then you can add this argument to the command, which will set our Symphony background process like this. It will track all the tasks, show you the project, and next refresh time. It will track a specific Linear project you set up every 30 seconds. If there are any tickets in "to-do," it will pick them up and show up in this list.

And all those configurations are actually defined in the workflow.md file. So, in the workflow.md file, at the front matter, there is a project slug. And the Symphony script will basically read that metadata, importing information from a specific project. Same thing for all the other configurations, like how frequently it should pull the ticket data, what are things it should do after setting up a new workspace, how many agents can be run at the same time, and the Code X configuration.

But once you set this up, it's basically monitoring the specific Symphony repo with the Elixir implementation. What we want to do is apply this to your own workspace. It's actually pretty straightforward. You can just open any coding agent like Code X or Cloud Code, point to the spec.md file and say, "I want to set up Symphony for my repo, and we will reuse the Elixir implementation here, and help me build the workflow.md file for my repo." With just one command, the coding agent is smart enough to look at your own repo and design a workflow.md file inside there. And this is the one it created for me, including the project slug and API key and all the other configurations.

But you do need to set up Linear first. If you haven't created a Linear account yet, just go create one and then add a new project. And in this project, click on the button here, you can just paste into your coding agent. This thing in the middle here is a project slug, or you can manually paste it into the workflow.md file as well. And meanwhile, you need to get a Linear API key, which you can get by clicking on settings, security and access, and add a new personal API key here. And once you did that, you should run this command, which will save the Linear API key globally on your computer. So, every time when the agent tries to use Linear, it can access any projects you have access to.

And there are some configurations you should do, which is status. So, Symphony out-of-the-box is designed for some special status control flow, like human review status and also merging status. Once you put a ticket into "to-do," Symphony will automatically pick it up and put it in progress and trigger an agent session. And once the agent finishes the work, it will change to "human review" status, so that you can review the work. And once finished, you can set the status to be "merging," which will trigger the agent to automatically raise a PR from this work.

And once you did all that, you can do run Symphony past through your workflow.md file, plus this "I understand that this will be running without the usual guardrail comment." And now Symphony will be working and picking up all the tickets in your project here. To make it easier, you can also create a new view, set up this board, so that you get this kind of Kanban experience.

But to just test, I can just create a ticket, change the landing page hero copy from "your company on autopilot" to "your AI growth team," and then set up the status to be "to-do." And this should trigger our agent here. If I go back here, you can see this time it picked up this ticket, and then you can see the agent session show up, and then the last agent message here. And depending on your settings, you can also go check this workspace. You can see inside this workspace, it has one workspace per ticket. So, each one is running an isolated environment.

And this example implementation also has a kind of web UI dashboard that you can visit, and this will list out similar information you will see in the terminal here. Not particularly useful, but I just thought I'd mention this. And you can see after a while, this agent changes the ticket to "in progress" status, which reflects in our Linear board as well. And if I click on that, the agent made a plan and logged all the steps it did. After a few minutes, the agent checked off every single item on the checklist, and uploaded a video recording to verify things are working. And as a human, I can just very easily see if things are working or not. And once I mark something as "merging," it will also create a PR for me. So, this is the whole end-to-end process and how you set things up. It definitely feels like the future.

If you hit any blockers, I have more detailed step-by-step breakdowns, as well as all skills posted in the AI Build Club. Every week, we have workshops to go through those latest learnings and answer any questions. So, if you're interested, you can click on the link below and join our next batch. But this is project Symphony, how it works, and what are the implications. If you found this video useful, please give me a subscribe and comment below. Thank you, and I'll see you next time.