We're good. >> There we go. Right. How's it going? >> Good. Good. Um, so this week, uh, this time, I guess I should say we wanted to talk. Okay. All right. Let's talk about your, uh, skills project. You've you've taken your entire workflow that you've used for a long time now, uh, that has allowed you to do things like build an entire assembly tool chain, which we'll talk about later, um, and do all your crazy projects, the things that you're building so effectively with Claude, uh, and Cloud Code, and now Codeex even. And you've taken all those skills and the whole workflow that you use, you put it into a set of like more generalized skills that aren't just specific to your machine. And you've put it into a GitHub repo. Can you tell me about that GitHub repo and selfishly tell me how to use it so that I can go and build really cool things, uh, on my own as well?

>> Yeah. Yeah. Um, so I mean, this kind of started back when Cloud Code came out, and I just spent months, every single day, trying to figure out how to get good results out of these things. And started building up when skills became a thing, started using that as my main vehicle for trying to get like a reusable workflow that I knew would be getting consistent results. And every time running through, um, building something, I would notice, okay, it's having a problem with, uh, X or with Y. And then I'd try to go back to the start of like, what's wrong with my skills or what's wrong with the, you know, initial prompts for these things? And how did I, how did I not set it up for success? What did I not give it in order to, to do the task properly? Because I was pretty confident even back then with Sonnet 4. I don't remember what the, what the big model was at the time that I started this with, but those models were still super capable. And I had this mentality of, I know that it can do the thing. I just need to figure out how to guide it the right way to get the results that I want. And so like, rather than, what was the metaphor I've seen before of like, if you're running a factory, you don't criticize the one guy in the assembly line that's not putting, you know, the the bolt in the right spot properly. You go look at the process that leads to him not doing that properly. And like, you take a step back and then you try to get your whole, you know, your whole line running smoother. And so that was my mentality on this of, I'm not going to really blame the model for this because I'm quite confident. I've seen it get good results. They're inconsistent, and I know that it's capable of getting the results I want. I need to figure out what is the process around this, um, to, to do that. So I started building skills and I would kind of tweak them over time. I started with really just one, um, last summer that was the core one that I called the autonomous development skill. And that was kind of the basis of the Ralph loop concept. Um, so you'd have a bash loop that's just running Claude code in headless mode. And the prompt for that, the entire prompt was, run your autonomous development skill. And it would go load the skill and it would take actions based on whatever was in that file at the time. And then I would just tweak that file continuously as I noticed that that loop was giving me what I want or didn't want. I would go change how that skill works.

>> So, so sorry if I can interrupt. That was basically your, you were turning the, the Ralph loop from a like set of like a text that you would put into the prompt to like the Ralph prompt was in that skill, for example. And you were just tweaking that over all your different projects. And which, to me, that makes sense because it's a central place that's version controlled, that you can manage, that you can share across your projects. And, um, and it also makes it easy for you to like write and clearly see the, whatever bash loop or whatever you decided to use to create that Ralph loop, like the harness for it. Um, it's short. It's just like, refer to that skill. So that's what you were doing for all your Ralph loops?

>> Um, yes. I mean, the, the very first Ralph loop was just the normal literal bash loop of, you have a prompt string here that says, I have a markdown spec file and I want you to go look at it and go look at the implementation code and find a gap and, um, make some changes to bring it closer into alignment with that spec. Like that was the core of the Ralph loop back when Jeff Huntley put that out. And, um, that was a great start, I found. But it wasn't, you know, there, there's a lot for it to track. Then if you have a 5,000 line markdown file that has all of the different behaviors and requirements that you want from your project, and then it has to go audit 10,000 lines of code in whatever your language is, and then compare the two. And that's a lot for it to go do. And it would burn a lot of tokens and it would get confused. It would lose track of what it's doing. And so I tried to give it more structure over time. And so I'd give it specific goals of, okay, here's the gigantic spec. We're not going to tackle all of this in one shot. Um, let's pick one thing. Go see if that thing has been implemented or not. And if it hasn't, then go implement it. And if it has, go look for another thing or like, and then kind of keep notes on where you've looked at and stuff like that. And like that was kind of doing some things too. It was, it was getting there a little bit, but, um, still wanted more, like more structure to it. Um, but not a lot of structure. There's a lot of teams and people that are putting a lot of work into harnesses and tools and memory systems and different things. And, um, I really want to keep it as simple as possible. Um, and just, you know, don't overthink everything, but have some kind of consistent process. You know, like as a human, when I go to sit down to work on a project, I kind of have a big backlog of features that need to be built or bugs that need to be fixed. And I'm going to pick one of those things and work through it and then call it a day. And that was so kind of the idea for the, the Ralph loop, but I'm gonna help it a little bit on the picking a thing because I already know what's being, what's been done or not.

>> Um, and so I would kind of act as the project manager a little bit there. And, um, okay.

>> Then so I implemented a product, uh, project planning skill. Then that would be kind of the first pass for this whole thing.

>> Wait, sorry. So you're talking about like this is how you formed the skill structure?

>> Yeah. This isn't what you do now. Got it.

>> How this stuff kind of came to be here. So like starting with just like the bash loop Ralph skill thing of just the Ralph prompt in a file. It would go run that, do the thing. It would get lost a lot. It was okay. It gave me some decent results, but that was like the core of the epitome of AI slop. It would just do stuff.

>> Yeah. And that's not what I want. So thought, okay, I need a planning stage in here of queuing up the work that Ralph is going to go do. Um, rather than just having Ralph pick stuff at random and and work through it. And at that point, maybe it's not really Ralph anymore because that's just Ralph just bubbles around as a as a character and things happen and that's that's great for him, but that's not great for software.

>> We're, we're, we're starting to diverge a little bit from the original metaphor, basically.

>> Yes. Yes. And so like, I don't really refer to it as a Ralph loop. From like, none of my scripts say Ralph anywhere. It's just as the autonomous dev loop, whatever. That's just what it runs. And, um, so then I, you know, worked into being a, um, an architect role or a PM role or whatever where I'm doing some up planning. So I have a skill for planning that says, I want to work on feature X. The skill walks through a process of researching current implementation, researching prior art in the industry, researching documentation around the libraries that are being used.

>> Pull all that together with the goal of whatever the new feature is. Interview me to understand if there's critical pieces that it hasn't understood properly. And then go write out a giant plan. And then it turns that plan then, um, into now tickets. Uh, when Beads came out back in October, that was a huge unlock because I could, um, prior to Beads, I could have Ralph just, you know, say, "Okay, the, the skill is now go look at the plan document and just work through the kind of the checklist, um, or whatever is in there."

>> And that kind of worked, but it would also still lose track of things. It would forget to update the file. It would forget to, it would do it inconsistently. It would leave too many notes. And now the file is 80,000 tokens and it can't read it properly. And, you know, confusing stuff. So Beads came out and I added an extra step then of, we have this now, this plan document of all the phases that we want to work through.

>> Convert that into Beads and then tell Ralph, pick the next ready bead and work on it and then exit. And that became, um, amazing loop actually at that point because, um, it could, it had a queue basically of here's the stuff that's ready. There's a dependency graph under the hood so that it's not going to work on something out of order. It just runs the bead ready command and says, okay, there's three items. I'm going to pick whatever the highest priority one is and I'm going to go do that. And then it would do it and it would exit. And that just was huge just on its own right there because it had a single focus. It didn't have the entire spec to distract it with all these different concerns and stuff. It just said, "Okay, I'm going to do this one thing." Um, and it could do it excellently. Um, that was that was really great. Um, but noticed that the plan document itself that was making those Beads issues, um, had problems. It would have gaps or different, different inconsistencies in it. And so then the Beads then would have inconsistencies in them where something following would reference something by a different name or something that was never even implemented because the step to implement it would got skipped somehow in the plan. So added another little mini skill in the middle of, uh, reviewing plans. And so that's a separate Claude session. So like, first Claude session, write a plan. Next Claude session, review the plan where go through with a ton of sub agents and look at all the different aspects of it. Figure out, are there gaps? Are there inconsistencies? Contradictions? Is there missing research? Is there missing anything else? And it go, you know, fix it all up. Um, and then from there, then you can turn it into, uh, tickets. I use tickets now rather than, than Beads. But, um, that became the, the process. And now this is really dialed in at this point where we have a kind of a structure that's agnostic to any project. It's, it's all just a workflow for it to go through of, plan out a feature, review the plan to make sure it actually makes sense, convert it into a dependency graph for implementation, and then now a skill to go just burn through all of that work and and build it out. Um, and that's the, the essence of what is in this, uh, repository here. There's a few other extra skills around that, but like, that's, that's the crux of it. And that's the workflow that I go through now for everything. Is I'm involved in the very first step of the planning, just to make sure that everything is what I want it to be. And then the reviewing, uh, skill will stop if there's anything critical that it's found that's like, this is wildly inconsistent or this doesn't make any actual sense or or whatever. It will stop and say, what is it that you want here? Um, and then I can go answer that couple of questions and it can go carry on. But otherwise, it fully resolves everything else by itself.

>> So, it has one of the skills, the reviewer skill is like, it, it's set up in a way that whenever there's, uh, the model sees like has is uncertain about something about an action to take, it will prompt you.

>> Yes. Um, it used to do that a lot, actually. It would go through, it did a review and would come up with, here's 45 issues with this plan of,

>> Oh, so this was still in the planning process. This review part? Yeah, the review is part of the planning. So it's kind of just like the grooming phase in the normal SDLC of, you have, you know, your make up a PRD and then you go do some research and you have some spikes and stuff and you come back and say, well, okay, this doesn't make actual sense with how the world works. You go update your plans and stuff. And then once everything is kind of groomed out and you have your Jira tickets or whatever, then people can go start building it. And so there's a bunch of upfront planning, just kind of mimic that of,

>> Do a bunch of upfront planning, review the plan to see if it makes sense. My initial review thing had it basically check with me for every inconsistency that I found. And that was a lot. And I was just finding myself constantly saying, you know, it would, say, here's a recommended option. And I would say, um, you know, it give me a big list. It would say, you know, here's 15 inconsistencies I found. Here's A, B, C, and D for each one. And I would be like, A, B, B, C, D, A. I wouldn't even say any other words. I would just literally just type, you know, A space, B space, A space, whatever. Submit that. And it would go fix it all up. And almost all of the times those were the recommended options as well. And so I thought, why am I even putting in input here? Just if you have a clear recommendation based on your research, there's a very clear path forward,

>> Do that and just update the plan. And,

>> So now it only pauses to,

>> Ask me things if there's something that really sticks out as like, this doesn't make any sense at all, or this is not technically feasible for some reason, you know, it'll, it'll stop for that. But everything else, you know, it regularly will say, "Okay, I found 32, um, issues that have now been autonomously resolved and the plan is in a great state. Go ahead and build it." And like these are, that's at that point, it just kind of works. It also has a kind of a gate at the end of it that where the AI can pause and also say, "Well, I did find a lot of gaps here. I'm not totally sure that I addressed all of them. So I recommend you do another review cycle." And so you exit that Claude session, start up another one and say, also go, you know, run your plan review skill against those plans. And it will go through and do it again.

>> So you still need to go in the order. So you still need to kind of like guide it like you'll you'll go through into Cloud Code for example and you'll start with be like, run the, um, I know the planning skill and, yeah, so to help me put a plan for this project. And then, and then that planning skill will ask you questions and like, you kind of give it a, like where, kind of walk me through that that planning process and like what you would actually do to do a project.

>> Yeah, that that's exactly it. I I have an idea for something that I want to exist. So I start a new Cloud session. I say, "Run your project planning skill, >> right?" >> And then I'll usually just do a new line and do kind of a brain dump on what it is that I want to plan out.

>> Um,

>> And then it'll go run the skill and take whatever I gave it there and do some initial passes at research. Um, just to kind of understand current implementation and everything else around, you know, what might be involved there. And then they'll come back to me with questions and, um, have kind of this, you know, interview session where I, I answer whatever it might need to know from my end.

>> Um, have a bunch of strong wording in the skill now where if it's a open question that should have a clear answer by doing more research,

>> Just go do more research. So it'll go read documentation online. It'll go read more code.

>> It'll do that in waves until it really has fully understood things. And then it's just anything that's kind of outside of that where it's project, you know, vision, context, what we're trying to do, or, you know, if there's any outside technical constraints or something like that, it'll stop for those things. But if there's a clear obvious path forward that can be resolved just by doing research, it will go do that and figure it out.

>> Do you, do you heavily use sub agents like in those, in those skills? Do you tell it to use sub agents and in parallel?

>> Yes, everything is sub agents. Um, so like the main agent is just orchestrating the workflow and fanning out waves of sub agents to go do all of the actual heavy lifting. Um, so for the initial planning, um, skill, it starts with my brain dump and we'll go fan out a set of researchers, usually three to five sub agents to go out and kind of crawl the world to, you know, get the initial set of what's the scope here. And then if there's gaps, it'll do another wave after that as well and kind of get a really good sense of everything. Um, but none of that is supposed to feed back to the main agent. Those agents report back little summaries of what they found and report everything else into files in a temp directory. And,

>> Then once you know, once all of that content has kind of been brought together, then part of the skill then goes into like the next orchestration phase is a,

>> Go review all of that content with another set of sub agents to,

>> Oh, so they don't report it back, they just put it all in an temp file and then the next phase is going through and reviewing them. Is, is that part of still the same phase where the orchestrator has another set of sub agents go through? So like you don't have to go and launch another like review phase of that.

>> Um, yes. Yeah. There, there's, there's a lot of waves of sub agents going on in here,

>> From the same skill.

>> Yeah. From the same skill. All this, this project planning skill just goes top to bottom,

>> Waves of sub agents. Wave after wave after wave. Um, and it has, so initial researching ones and then like an aggregation one that might kick in. If all of the research totals, you know, 300,000 tokens of content or something like that, then you can't have one sub agent tackle that. So then, you know, it'll do some other sub agents to kind of comb that down into something that's like, what's the actual truly relevant bit of content that goes to this plan?

>> And then, uh, and then a round of sub agents for doing the actual planning based on all that research content and the goals that we have. Let's make up an actual plan. And then, um, you know, it and just kind of keeps carrying on with that with sub agents that have specific chunks of the planning process assigned to them. And they go and do that piece, report back that they completed it, if they ran into a problem or not. And there's more that needs to be done or something for their piece. Maybe that does happen. The orchestrator agent knows what to do with that because it's in skill. Um, so it just kind of, it just keeps working through with sub agents.

>> Got it.

>> Um, uses a lot of tokens. It takes quite a while sometimes. Uh, like if it's an advanced feature, it's that thing can just run by itself for up to an hour or so, just to, just doing all this.

>> A question I was going to follow up, uh, about this, but I think it's more is relevant right now is like, is this, you, you, you mentioned like feature multiple times. Is this something that you use for feature development? Do you use, is it better for greenfield projects? Like what's the, do you, do you go through this process every time you want to do like a new feature within something you're developing?

>> Um, yes. It, it depends on the scope. I have kind of a, just internal threshold for if it's something that I know Claude can handle just in one session as a one-off thing, I will just go open a Claude session and tell it what to do and it'll go do it. Um, but if I know that it's something it's not going to be able to handle in one session,

>> Uh, then that's where these skills come in and,

>> It, you know, it will funnel through the whole planning thing and reviewing, getting a bunch of tickets. Usually there'll be like 20 to 90 tickets that come out of this whole process. And then that's where that looping comes in to just work through all of those.

>> Okay, I see. So the result of this planning process is creating a bunch of tickets. Like, the, the outcome is a bunch of tickets, which you will then use the next skill, which I don't think you've talked about yet, but the next skill to go through those those tickets and just start working through them. And then by the end of that, once that's done, the tickets will all be cleared. Then at that point, you could go and create like, go through this planning process again for another huge feature, or even halfway through it, you could probably go through and, and create like, go through this planning process to create more tickets. Although I'd imagine that would could result in some conflicts when it comes to like ordering which ones to do and maybe like you don't want to build, try to have it work on too much at once, right?

>> Yeah, it, it can do it. Um, I try to keep things a little bit more focused. Um, just even for my own, I have have timeline expectations and things. And so I have done it a few times where I'll do,

>> A bunch of plans. They're all kind of touching different areas of the project. So I know that they can be worked on independently,

>> But those would usually better be done in separate work trees where each work tree can focus on that and then merge it all back together in the end. Whereas if I just flood the main, um, if I put it all in, you know, my main development branch, then now I've got 300 open tickets and I can't really run the loop,

>> You know, concurrently against that because they're going to step on each other's toes. And so then it's just one worker basically that's chewing through the entire, uh, backlog of stuff at that point. And,

>> It'll do it. It's just, it's going to take some time. And I really want certain things done before other things. And I, I can get in the middle of that too. I can tweak my prompts to on the loop to say, focus on,

>> You know, feature X first if there's anything open there. And it'll go do that. But it's just easier to just, I have a new idea, work tree, go do planning over there. I can have nine work trees running at once if I really want with all different things or different projects or whatever. Um, I, I feel like it would almost be better to, and this is just like my initial thought on this, is that just to prevent, so you'll end up with a bunch of merge conflicts inevitably if you do that if you have a whole bunch going on at once, which I, I try to avoid as much as possible if, especially if it's just me working on a project. Um, in which case, it's almost better to just use a whiteboard or or just piece of paper, whatever, and write down, here's a list of the next things that I want to do. And while it's, I, I know that development happens much faster now than it did even like two, three months ago. So this may be less of an issue. Like you can go through the first one fast enough that the second one happens just like after you've thought of it. But even just keeping a list of, here's the features or like a roadmap, like I've seen you do with with tickets, for example, like have a roadmap of these are the things I eventually want to get to. You're not putting all the tickets in at once, but you at least have somewhere to store temporarily, like your ideas down of what you want to work on so that you can eventually come back to that one next, knowing that like you just follow this process and it's most effective to focus on one thing at a time.

>> Yes. Yeah. Um, yeah, there's, there's kind of outside things you can do, I think, to manage it. It's project management at that point. Like, what order do you want things to be done in?

>> Very true.

>> Where are the clean divisions to be able to say, okay, we can actually have two developers working on these things together. Um, or in parallel and we'll integrate it later and all of that bit.

>> Um,

>> And that a lot of that still really comes down to like the human judgment around priorities and what we actually want to build. Um,

>> Yeah,

>> The AIs are terrible at, they have no taste whatsoever. They don't know,

>> Good design. Like they know it's weird, right? Because they know all the design patterns. They know all of the architecture. They know how to build all of the things. But if you just let them make their own choices on the what and the when and the why,

>> It's going to be weird and wrong. And so that's where we still need to be on top of things is the, is the picking,

>> What is the next thing we're going to do and what shape is that going to take? And then whatever shape you want it to be, it knows how to do all of that already. So it can go do it for you. It's, but you have to define that shape for it. So you think for now, the, the role of a, because this is something I grapple with regularly now, like it's an existential thing that we briefly talked about in the last, the last time we talked, but it's, what is, what is the role of the human now? Like, what is, what is a thing that we can uniquely do that is significantly better? And this will pro, almost certainly change over time, but for now, it's our, our best differentiator and the best leverage that we have that's different from what a, you know, an LLM can do, is understanding the, by understanding the like the when, the order, like basically project management and direction, um, is where we can work best versus the LLM is like, a, you give it a task, it can do that task really, really well, but it doesn't know what tasks to do beforehand. It doesn't know why, and it doesn't know when. Um, and, and maybe this is like, I don't need to like diverge into another topic, um, too much, but like, I wonder this, if this is why I'm skeptical of open claw. I'm not, I'm skeptical from a security perspective. Actually, not skeptical. It's like, I'm certain from a security perspective is a terrible idea. However, um, I wonder that's why we haven't heard, like, I think a lot of the excitement around open claw is just novelty. And I, I think that there's, um, and, and all these other like me too, like different versions of open claw, um, that people are building. But I think that there's we haven't seen a lot of like, really great breakthrough use cases. It's almost like what VR was like, right? Like, don't get me wrong, I love VR. I've like, I was a huge user of VR. I still have a bunch of VR systems and I developed in VR. Like, I love that, but it was like, there was never a clear, this is the killer use case for this thing. This is where it is better than anything else that we have right now. I feel like open claw is kind of like in that state. Not that it can't ever get there. Sure, I'm sure it can. The more people pour resources into it, like, I'm, I'm sure it'll get there. But for now, I think a lot of it's novelty because the value of an LLM is still like, they, they can't quite do the things that humans can do when it comes to project management and the why and the, the when and the like, why it matters, basically, and, and what order to do it in. Um, yeah,

>> I don't know. Without diverging too much, that's what do you think about that?

>> Yeah. I mean, so I mean, that does that does feed right into this because they, they can do a bunch of that stuff, which is is a little bit disturbing part. And that's really what these skills lean on is,

>> You know, I, I'm setting ultimate direction and setting architectural constraints and direction and things. I know what patterns are going to fit into the future of this project best compared to other patterns that we could adopt. And which, if I let the AI just do things, it would probably do because that's closer to the normal in the training data distribution or something, right?

>> You know, like there's the,

>> Very common. If you just have Claude go make a, you know, a website for you, it's going to have that weird purple gradient on it. That's like the, the, the right down the center of the training data for whatever reason. And it's going to, it's going to do it in X.js. And it's going to do, like, there's certain things it's just going to kind of default to because that's what it's, you know, that's somewhere in the middle of the distribution there. And it's just going to do it that way. If you want anything outside of that, that's where you have to set kind of the, the, the framework for it and say, okay, in this project, we are using these patterns and these languages and these libraries and we want it to, you know, have this shape to it. And when you, when you give it that though, it's going to, it's going to work great inside of it. Um, it's just if you leave it. People have this the same problem though too. Like, there's a, there's a whole thing around constraints being a driver for creativity.

>> Yeah.

>> And if you just give somebody a blank piece of paper, they have no idea what to do with it. But if you give them,

>> You know, a little bit of, you know, a few shapes on it, they can start connecting dots and draw the owl or whatever the the meme is, right?

>> Um, and it's this, if, if you have too many options available to you, you get into that analysis paralysis thing. And AIs don't really get paralyzed. Able to pick something that's not optimal. Um, and that if that's what you want, that's fine, but I need better than that. So we are, we are effectively like our differentiators as guides of like, um, or oracles, or I don't know, shepherds, like I don't, I don't know what the best metaphor analogy is. But basically like, we, our, our differentiator, like the thing that humans do better right now is like maybe forcing those constraints or picking a, maybe setting long-term horizon goals because perhaps the LLM is limited by the context window and how long-term horizon it can actually think or think, right?

>> Yeah, yeah. No, exactly. So it will, um, it would, it would do great if you can set that for it. And if you don't, then you're going to get, that's where the swap comes in, I think. And that's that's the majority of of the things. And like open claw.

>> Okay. The other thought that I'm having here, I don't, I don't know, have we talked about the bitter lesson before? Has that phrase come up as a thing? Um, so there's this paper from 2017 or so. It's, it's, it's really short. Just look it up. You can look it up and have a have a read through it at some point. But, um, basically the, the core, core thesis of this was that scaling compute has always and theoretically will always outperform any novel architectures or or things that you might think up to make, uh, machine learning do what you need. So rather than getting really, really, you know, fine-tuning your parameters for the model that you're building or, you know, putting a lot of thought into the structure of of the training process or,

>> Like it was very specifically geared to AI development rather than AI use. But I'm finding it's, it applies both in both places of,

>> If you put a whole ton of work into your crazy harness that gives it all of these things,

>> It's not effect as effective in the long run as just waiting for the model to get better and throwing more compute at letting it burn through things. And so my, my whole two things for getting good results, I think kind of mentioned this on on other chats, is everything comes down to feedback loops and how many tokens you're willing to spend.

>> And that if you get the loops tight enough and you're, you let the AI churn away inside of those loops long enough, you will get exactly what you want. And if you're not getting what you want, it's because either you haven't given it mechanisms to get feedback properly and to verify that it's doing the right thing, or you haven't let it, uh, burn enough tokens basically. And so like, that's what all of these skills are kind of geared around is the, um, not the feedback loop part because that's really project dependent. Like if you have a good test harness, if you have your, I don't know, static type feedbacks, if you have browser testing tools that let it go exercise UIs for you, like all these different ways that it can check if it's done the right thing as very project specific. I don't have a general purpose thing for that. But on the scaling compute side, where we really just want to spend more tokens for you, like thinking through the planning process, if I just say Claude, go make a plan for this crazy big thing and it doesn't use any sub agents, it's got 200,000 tokens to work with to do all of that planning. It's gonna, and by the end of it, it's going to hallucinate and forget a half of the things that it found at earlier in the session. And so sub agents come in as a way to just scale up. Now, you know, that main agent can oversee processing two million tokens instead of 200,000. And each one of those sub agents in there might use 50 or 100,000 tokens and it might use 30 sub agents. And that lets it scale out on the compute side, just churning through little aspects of the problem, um, from all the different angles that it needs. And then breaking that up into then more top level sessions. So having a project planning session to get kind of all of the high level things in there, and then reviewing it where we burn another million tokens on making sure that all the gaps are closed. And then we go into a brand new session again to review all of the plans and figure out how to break it into the dependency graph of tickets cleanly. And then finally new fresh sessions with the whole Ralph loop where it looks at the next thing and it focuses exclusively on that one thing and works through it and constantly with fresh tok, uh, fresh fresh eyes in the world. It doesn't have prior any context that it's carrying around, which means that it has to burn more upfront to understand, okay, where am I in this project and how do I get oriented and what is this task that I need to work on and what's all the stuff that's involved in that?

>> That might take you 50,000 tokens to work through at the start of it session before it even gets productive. But then it's going to go do that one thing perfectly.

>> And then it'll shut down. And then the next one comes up. And it might be a new task that's directly related to the first one. And it has to go rediscocover all of those things. But it will do it. And then it'll understand the current state of the world with that first task completed. It knows now here's the second task. It's going to go do it. It's going to do it perfectly. And that, that's the whole thing. It's just how can I make this take more tokens? Is like a regular question I have of, how can I decompose the problem into a few more steps that more sub agents or more main agents can burn tokens against?

>> And,

>> Okay,

>> But like not overthinking things. I just want to, all of the skills are just around decomposing the workflow rather than big, like trying to make the the harness smarter in any kind of way.

>> So just breaking down, like it sounds like a lot of this, the theme that we keep coming back to is focus. Like whether it's about focusing on a single feature at a time or breaking that down into single things that the that the autonomous development loop can actually focus on. It's about focus and keeping that tight. And that, and tight feed, tight feedback loops is another thing. So kind of to say to also repeat what you just said about the two things that you need to get best results. You need, um, tight feedback loops and you need lots of tokens to burn, specifically leveraging those through sub agents so you don't use all those tokens in a single context window. Is that correct?

>> Yes. Yes, that's I think exactly correct. And that's why I had been really, um, like, I, I couldn't get good results out of Codeex for the longest time because Codeex, Codeex has a completely different way of doing context compaction and stuff like that. It does seem to work independently for longer, but it still ultimately has a limited context window. And I mean, GPT 5.4 came yesterday. You can get a million tokens, or is it two million even in a window now?

>> A million, whatever.

>> Um, Gemini's got two. But they all have that whole needle in a haystack problem where as that window gets filled up, it's going to forget important details. And I really don't want to use the big window. I want to have it, you know, as small as possible. Um, and it's totally okay to use the first 30 or 40,000 tokens in the window to get oriented properly for the specific thing that needs to be worked on. So long as then in the next 20 or 30,000 tokens after that, it can just do that one thing and it's going to do it really, really well. And then the window doesn't get too full. It shuts down. It's a brand new fresh window to go on to the next piece of work. And that is definitely more token expensive. It's also using a lot of sub agents during that, like all of these skills are all sub agent orchestration skills.

>> Um, and so every sub agent also has to do kind of the same. I'm gonna go read the Claude.md file and I'm going to go, you know, start finding the right code because the prompt says I need to probably go look in over here, but I know there's other stuff involved. And so the sub agent is burning tokens and doing all of that every time. And the session uses 20 sub agents or something. And so like every one of those is burning at least 20,000 tokens off the start, probably getting oriented for its own little piece of the work.

>> So there's 400,000 right there. And you know, it adds up really fast. Um, but it allows it to be like just laser focused on, I'm going to do this thing and it's going to get done perfectly. Um, like it's really nice like that. And this is whole, you know, it just, I get what I want from the AIs now, at this point.

>> You, Okay, I want to go back and make sure we cover the full skills process of that you're talking about. So you mentioned you've gone through the planning, uh, skill, like the the initial planning skill, then the review skill, um, and then, uh, after that, it's the autonomous development skill. Is it, is that the three?

>> There's, there's one in the middle there that's, it's a really short one, but it's the plan to tickets skill.

>> Right. Oh, I do. So that's a separate,

>> Plan document.

>> Okay.

>> Yeah, that's a separate skill just because I want a fresh context window that it's not going to, um, lose track of some step in the middle of the plan or something and just forget to make a ticket for it or something.

>> I see. So the output of the planning skill is a PRD, like basically a plan document. Is that correct?

>> Yeah. Yeah. Just a big markdown file. It's usually, I don't know, a few thousand lines or something. And then,

>> That's the output of the initial planning. Then there's a review skill that just goes and refineses that same document.

>> Got it. To close up any gaps.

>> And then the ticket skill goes and uses that as the input document. Output is a bunch of ticket files, um, that are just the markdown files using that ticket CLI tool. And so it's got a dependency graph in there.

>> And then after that is just run the loop, um, with the with the main dev skill to just work through the tickets.

>> And that's, that's the whole crux of it. And I just have to frontload all of my thoughts in that planning phase up front, basically of, here's what I want. And then,

>> Um, I still manually,

>> Start up and shut down the Claude sessions for those. So I'll start up a planning run through it. I need to be involved in the planning part because it needs to know what I want. But reviewing it almost never asks me questions at this point, especially Office 4.6. It knows how to close almost all the gaps unless there's something very, very like, there's a big flaw somewhere.

>> Sure.

>> Which does happen sometimes, but for the most part, I just start a fresh Claude session, say, run plan review for X. It goes and it does it. It comes back 15 minutes later, says, cool, we're good to go on. I close that session manually. I start up a new session. I say, run plan to tickets. And it goes and does its thing. And then I close that session manually. And then, uh, bash script and I just let that run. But sometimes I, I want to see it work. So I will start up a new Claude session and just say, run autonomous development. And,

>> It will go do that live so I can kind of watch it work through.

>> Um, I don't need to do that. I just like to do that sometimes, or, you know, I want to see what it's, what it's doing and how it's working.

>> It's also really cool to watch.

>> Yeah. And I used to need to do like that. I used to have to do that because there were gaps in the skill or something where it's gonna kind of get off track. It's going to do a bit of the wrong thing. And so I want to see how it's behaving and then I can refine the skill. And that goes back to what I was saying earlier with it's, if I'm not getting the results I want, it's, there's a process problem here or the inputs that I'm giving to it are not quite right. And so I'm going to go think back. I might throw

Away everything it did. They'll do a hard reset on the branch or something and think, okay, what is wrong with the skill input here? Or what's wrong with the plan that was fed in or like work back to what it was? What is the root cause there? Fix that and then run it all again, which might even go all the way back to just throw away the plan entirely and we're going to start from scratch here and plan it from from fresh.

Um and do that whole process. And so then I'd run the autonomous development skill manually and just let it work and I would see if I get what I want out of it and I you know and just kind of like um uh you know just kind of cranking the wheel manually there and not it's not a headless bash loop that's just doing things. I'm still in the loop there just to verify that it's actually doing what I want it to. But at this point it's super well dialed in and I don't need to watch it. I just like to do it sometimes with it.

Yeah. Then you know and I'll go and I'll check on it and I'll look at the diff or something that came out of it and go oh yeah that that's actually that's exactly what I would have expected it to do perfect and then you know um and so like that's the whole process is plan review the plan make it tickets run the loop and okay um that gets me everything I want.

The one outside of one piece outside of that is I do have a there's a claw.mdglobal file okay in this repo as well and I put that in my home directory so that every every cloud session that I run on the machine has that put into its context and that has this big section at the top uh called the maximal simplicity policy and it's got a bunch of uh kind of there's a bunch of like catchphrasy memey things that is just like stuff that uh from people who know what they're doing in the industry have kind of put out there that I know is in the training data and so if I reference you know if I if I put the word complecting in there which is not a common word but that's a very specific word from a rich hickey talk around simplicity.

No. Um, so it picks up on that word instantly and it knows, okay, we're doing rich hickey things here and we want to think a certain a certain thought process through the problem. And like there's a bunch of these kind of things that are in this document um that are certain, you know, kind of triggers for getting it to be maximally simple. So this is my like anti-slop thing here is if this is always in every context window for every cloud session on the machine.

Um I had it go missing one time. I don't know it it uh like disappeared and I had a weird week where I'm like why is this like I know that this could have been done in 300 lines of code. Why is there it just built a whole new factory service thing and there's 2,000 extra lines of boiler plate around this is and and so I'm saying no stop doing that. why are you doing that? Let's go do the and trying to manually fix it. And then I'm just in my home directory and I notice, oh, the this file's not there. And I put it back and everything fixes itself again. So like I know that it does stuff. It has an impact on on the work output here. And the whole piece of it is just think through every change that you're making for every diff. Is this the simplest possible implementation for this? uh you know if if you're adding an extra abstraction layer that needs to be still the simplest thing that can accomplish the goal here. Uh sometimes the extra complexity is justified because the feature is complex or fixing the bug, the root cause of the bug is a complex thing and so there's some inherent complexity there. But the AIS love to just throw out every design pattern under the sun and you get into these crazy states and so this this whole ploton MD file is geared around don't do that. We we really want can this be done in not necessarily fewer lines of code. Sometimes fewer lines is still more complicated than more lines in a counterintuitive way. So like that's not the only metric.

But you know given the requirements of the task, can this be done simpler? Can this be done with you know a more minimal diff? Can we make the change in a different place that would or can we refactor something else that would ultimately make it easier to implement the real thing over here? You know a bunch of a bunch of that. So like sometimes the diff is quite a bit larger because it did have to go do some construction work to you know make the change the ultimate change easier. Um and there's I don't know if actually the phrase is still in there but there's the the phrase of making um make the hard change easy and then make the easy change or something like that. And so like you get a two-step workflow there. But just just a bunch of these kind of little things that nudge the model into the right headsp space here for not spamming out insane amounts of code to solve something that doesn't need it. Um, and so that's like the the the skills are kind of the workflow piece and then this file really has a has an impact I found on controlling the slot part of it.

Where's the the repo? I'm trying to look at the repo on your on your GitHub. Uh, it's just slash skills, I think. Oh, slash skills, I think. Or it could be cloud skills. Because I was going to ask you, do you have Oh, it is skills. Do you have um you should pin this on your on your GitHub main one? I should. Yeah, I just pushed it yesterday. Some some bookkeeping. I was going to ask if you had instructions on how to go through this. Um I see there's an installation like a setup.sh file.

Yeah. So, I mean the the way that I've been doing this um this is kind of a fork from my main skills directory. And so I have the entire under my home directory there's cloudskills. I have that whole thing as a git repo that I maintain from all my personal stuff. And so then to split this out I kind of had it be the same model of it's just a git repo of that skills that kind of global skills directory. I don't think that's the way people should use it though. I think I run a rewrite most of this read me club just kind of put it out there as a base piece. Um but it's you just basically it's it's skills. So there's a bunch of markdown files in here. You put them wherever you put skills on your system. Um for a lot of folks now that's under the agents directory instead of claude because you're using codeex or open code or something else. Um but so you put the skills wherever you put skills and then there's this uh this global you know prime directive maximum simplicity thing that um I find need like it really has to go into every session. So um that's there as well but you put that wherever you want to put it um to be scoped to your projects and um that's it. It's just

Yeah I'm looking at all the different skills you have. There's more than what you talked about here. Um well like for example the uh investigate blocker create new skill. We don't have to go over that now but I what I was I was curious about is if you had like a I see a typical workflow where you were talking about kind of the steps to go in order but it would be great to have like maybe this is a blog post I don't know um an example of like going through creating a project and you do this step then do this step like start by opening cloud code and saying run this skill to the planning skill and then put a new line here's the the plan. So like going through an actual example of like for someone that yeah like doesn't necessarily know how to use skills as much or doesn't know has never done like an autonomous development loop process like this which most most people haven't. Um it would be great to just have like a reference to be like okay I just import this all I have to do is follow these instructions and I can build whatever I want. That's what I'm thinking.

Yeah. Yeah. Yeah. Um that's I think a place to go with this. I'm not sure how like there's a delivery mechanism there. Maybe that's if you've done anything else with AI before, then you can't just clone this into your skills directory because you probably also already have some then you need to like have it play nicely there. But yeah, there's a setup piece here probably that I need to figure out um what that should look like. But otherwise, yeah, I mean the the process is kind of what we talked through. Um there is a bunch of other skills in here, but most of these actually get pulled in. um I see by the by by the loop and stuff. So there's there's one for test-driven development, there's one for called feedback driven development, which is trying to get the agent to think through, you know, this is a really thorny bug for some reason. And I'm not able to get the visibility or observability on it that I need. And so I need to go create a feedback mechanism for myself to go uh be able to verify that this thing is working or fixed. And a lot of the times simple, you know, test-driven development is enough there. It just needs to write more tests and it will figure it out. But other times it needs new tooling or we need to pull in some other project that somebody else has made other tooling that's better for catching this kind of issue or validating this kind of thing. And so that skill is just trying to get the model into being able to do that itself because otherwise I have to babysit it and make sure that it's got all the right feedback loops.

Yeah. And so it doesn't pull that skill in very often, but it it has done that and it does go kind of course correct based on the results of that. Um, and then there's also um I mean that that's that's really it I think is like there's a few extra skills. Um, there's one called investigate blocker that I don't think gets used much anymore but was needed with earlier models. Uh, Opus 4.6 seems to be able to just resolve its own problems. But before what uh would happen is the autonomous dev skill is running. it's hit something that it can't figure out because, you know, maybe his context window is too far or something, but it's got an escape hatch where it says, "I need help." And it used to just bail out entirely and I would have to go hold its hand through resolving the issue. But instead, now it's instructions are create a ticket called investigation needed as a prefix uh and then describe the issue that it's having. uh it will exit then and the batch script will notice that oh there's a ticket in the queue that needs investigation. So, it's going to run another cloud session but with a different prompt. Instead of run autonomous development, run investigate blocker. And that one specifically will go look at that ticket, go start doing research, doing whatever it needs to do to understand the problem, figure out what the solution is supposed to be, update the ticket or make a new ticket or whatever. It's not supposed to fix anything. It's just supposed to understand what's going on. And then they'll, you know, have outputs of that and then the bash loop will go run again and autonomous development on the next runaround will pick up whatever that new stuff is and then go do it.

And it would do that automatically, right? And yeah, that's all automatic at this point. Um, and that was what got the earlier models through a lot of problems that they were having. They would bail out with I don't know what to do and I have to step in and kind of guide it through that. But most of that I I found my my process for that though would be I'd start a new plot session. I'd say okay the last agent found this issue. Go do research, figure out what we should do and then it would come back with a recommendation and I say wow that looks perfect. Go do that. And so I just made that a skill and I said go, you know, and now we can just go do that. I don't need to be in the loop there because most of the problems 98% of the time are things that it can figure out just by looking more. Go read more code. Go read documentation. go read whatever. Um, go just search the web for how people in the wider world solve this kind of problem and then it comes back and says, "Okay, here's what we need to do and that's what we need to do." So, just feed that back into the loop and let it keep going. So, like that's all automated at this point. Um, there is an escape hatch for that escape hatch. If there's something that's ultimately, you know, it it goes, I can't figure this out. I have no idea what's going on. um then it will create a you know human escalation ticket or something and the the script will see that and then bail out entirely and say you know human input is required on this one and then I can go sit down but I've not needed that once since the new opus came out in November December there um it figures everything out which is wild.

Uh so the I was my point earlier about like having a example of like here's how you go through creating a project we kind have done that already in this this talk basically. So maybe what might be better is in in lie of doing a blog post separately for this once I publish this you can actually you could just do a put a link to it in there and be like because that's basically what we talked about this entire time is how this how this entire project came to be how you use it and pretty in-depth. So if you would like afterwards it might be worthwhile putting that in the repo.

Yeah. Yeah. Well and I mean you this is uh going on YouTube right? So I'm sure I could even just poke Gemini at that too and say so in this video I outlined exactly you know how to use this repository here distill that down into two paragraphs or something and have it do that. Um, and then you know, I I think I want to rewrite most of this read me actually. I don't think it flows very well um and doesn't really that that's all you need. get the skills in there, run a, you know, run ABC D and it's a right now that's not automated. I feel like that could be automated too though of you really only need to run an initial thing where it's going to walk you through the planning process and doing that interviewing brain dumping piece and then after that all of the rest of it could be uh like an orchestrator skill that just runs the plan review. It runs the ticket breakdown and it runs the loop and it you know that can be fully automated at this point and it could notice you know it could then be watching and say okay the plan review become this critical problem I need to go escalate that and bring me in at that point and same with you know the main dev loop if that runs into a problem the orchestrator could watch that and so like then I wouldn't need to babysit it and again that's another how can I throw more tickets at this problem or I mean more tokens at this problem and you know that would work really well. I think I just haven't taken the time to do that because I still like to watch it work to make sure that everything's going smoothly. But at this point, latest top models are ridiculously good. And I'm pretty positive that I'm just doing that for myself at this point. And there's no reason that I should be looking at these things anymore. It's just going to do the work. The quality level is dialed in. and I get, you know, really good code out the other side and the features work and um that, you know, it's just it's just the limit. So, it it really is that. Um and so like I could make this more handsoff, I think, and turn it into some, you know, that's actually going into that whole software factory model of you just feed in plans and ideas on one end and you get working software at the other side. And I think that this is basically that um except that takes more tokens and I'm already hitting my Clogmax uh weekly limits quite regularly. So I don't want to uh hit them even faster with with something that's just like kind of bookkeeping that I don't mind really looking at and I I can kind of queue it at my own pace then. Whereas if I had an overseer agent that's walking through the workflow, I don't need to touch it. It's going to get done really really fast. I don't need it done that fast. It's going to take more of my limit. You know, there's It'd be great to just have some, you know, unlimited spending account with these things, though. And wouldn't it wouldn't it just be the best?

Yeah. Because there's, you know, I think like this is the core of, I think, a process that makes robust, reliable software at this point. And well well for people that for example work in a company like like where I work we have an enterprise plan which doesn't have those same limits. I mean there's still always limits that you can set but like it doesn't have the same weekly limits you get in a cloud code max plan. And so people that are in that particular environment using cloud code teams or enterprise like at that point you can use this particular process and don't have to worry about necessarily hitting the limits obviously given your spending limits of your company but um use this process to kind of like here's your tool chain your workflow the like the bound like the here's the tool that you can use to now go and build whatever software you want to build in the most effective way possible using AI agents.

Yeah. Yeah. And this works with codecs just as well, right? It does. Yeah. So, that was the part I discovered yesterday actually. It was GPT 5.4 comes out and uh they got rid of the Dashcodeex variant. They've just merged it as one big happy GPT family now. And it's so it outperforms previous base GPTs on all the other things. And then it also outperforms all of the codeex variants on coding things. So, it's just one really solid model. Um, and the codec CLI has sub agents and it has good skills support and everything. And so, uh, yesterday I tried this out on a couple of, uh, different things where everything I have is um, all code oriented. So, I still have my cloud MDs and stuff. And so, my base prompt then to codeex was a little bit different where I said from this directory, walk up all the way to my home directory and find all the claw. MD files and read all of those first because that's where all my stuff is. And then I sim linked my cloud skills directory into a agent skills directory so that it has all the same skills because skills are cross compatible now just that directory naming convention is annoying. Um but then I just said you know go find all the cloud IDs just to get the right base context here and you know simplicity policy thing loaded up and then uh you know run the autonomous development skill and it went and it ran it and it chewed through a bunch of work and everything you know and it it did it excellently actually. I'd never gotten actually good results with this process from codeex before.

And the it does the two things I want to say. one highlight uh sub aents in codecs. That's probably the reason why you can do so much more now than you could before. But the second thing I want to ask is like so I just want to clarify you you said go through and read all the cloud MD files. Is that like to convert them to agents MD files or like what sorry where to read them just to just to load them. So like cloudMD or agents MD um depending on your harness would just get automatically injected at the start of your session. Yeah. Um, and so because the the file names are cloudMD, they don't get loaded into codecs automatically. So I go explicitly go read all of these files that it should have anyway. Um, I'm sure I could sim link them or something, but I wasn't also sure GPT used to behave very differently. If you have like strongly worded system prompts and stuff, it would overfocus or hyperfixate on things. Um, and I that was part of why I was felt I was not getting good results from codecs previously is because these instructions have language that works really good with claude and that doesn't mean it's going to work with GPT and in a lot of cases it will actually make GPT perform worse because it doesn't respond well to strong instructions.

Okay. Um, they call that out actually in their prompting guides on on the OpenAI docs is, you know, don't use any all caps bolded stuff in your prompts because it's going to hyperfixate on that as a thing. Um, and Claude does respond really well to that and those kind of things can make it perform better. And so that's the kind of language that I have in these things. Um, but 5.4 here seems to have understood, you know, it didn't hyperfixate on weird constraints or anything. It just followed the workflow that I wanted it to follow. Um, and it has sub agents now. There's a feature that you can enable with the codec CLI to do that. Um, it's had that for at least a month now. I think maybe in January they shipped that as a kind of a hidden feature. It wasn't even in the main features list and it was called collab I think at the time, but you could do codeex features enable collab and then it would um toggle that on secretly in the the config file and um it worked. They should like it was a feature that existed. sub agents, you could make it do things, but I think GPT 5.2, 5.3, they were not good at using that tool. They hadn't been trained on how to do that effectively. And so they would launch a bunch of sub aents and they would be kind of overlapping in weird ways or be checking on them in weird ways or it would be it would say like while those are doing this, I will go do X and it would step on the toes of its own sub agents and stuff. It was really weird stuff I was getting out of it. Um, but from what I saw with 5.4 before yesterday. All of that is fixed. It's using the sub agents really effectively. Um, and I was very surprised actually with how well it was working. So, I feel like all of this was very friendly to codeex now at this point. And that means that you can probably use this with any harness and model under the sun, I would think.

Yeah. But I do think you need like if you want the really the best results, you do need the top model, right? If you if you just go with uh whatever the latest open source ones are or something, they're very capable. All of the stuff was working great with Sonnet 4. Yeah. Even for a period I was doing Haiku 4.5 as my main Driver uh for probably two months there until the new Opus came out and Haiku did very solid work also following all of this. like it does work well with lesser models, but if you really want like the best results out of it, um because Haiku or if you're using I don't know Kimmy 2.5 or something that they it will work. Um they're not as good I think at the upfront planning part. They can probably do the implementation part pretty decently. I see. Um, but they don't, you know, they don't think through things enough or like for the high level picture of what you're trying to accomplish, especially with the the simplicity constraint on it too of okay, we've got this big thing, we want to go build it. What is the like the the essence of it and what is that? And like you know I use Opus for all of the planning at this point and it really spends a lot of time thinking through all of the constraints and possible contradictions and conflicts and things and like the ways that this aligns with the current state of the world and does a lot whereas the other models they just won't do that and that it works. It's just the results are not going to be the same. So yeah I always want to use whatever the top models I think GBT 5.4 before we do this great as well. I'll probably be testing that out more next week and stuff, but for for now, it does seem like we have two very good families of models for this kind of work, which is awesome.

Yeah. Okay, that's amazing. Um, one thing going back to your cloud MD files, do you have cloud MB files in different subdirectories of your project? Is that why you had it read through all those cloud? I still like caught on like Okay, that makes sense then. Okay.

Yeah. So yeah, the clawed um like the the harness will walk up the directory tree um all the way to just root um and any cloud. MD files it finds in there will uh it'll pull in um and then also if you have a why that's just how it works until the root of the project or the root of your the root of your your system like slash you know just so why do you why do you want it to read in all your different cloud files for all the different projects for your just the parents it just goes up it doesn't go to all the different projects siblings oh I see I see okay because so maybe I'm not understanding like it's a it's a scoping thing. So I have one file that's in my home directory that's everything I do all of my projects are under my home directory. So this one then becomes it's automatically injected in all of my work. Um and then okay if I have um like for example for my for like for employer work job work I have home directory MD and then under that I have a company name uh directory where I've cloned all of our code bases. I see. That's solid there. So I have a small MD file there that's the kind of the context of here's all these different repositories that have all these different responsibilities just so that if we're in one project and it knows then that these other things exist then it can say okay well that's actually because this other service is having a problem and I'm going to go look in that directory and understand what the you know the API is or something and so then like that gets pulled in and then the repo itself has a cloud MD for repo specific stuff but then you can also in further. So like if you start your cloud session in the base of a repo and you have under your docs directory, you could have another cloud MD that won't get pulled in unless cloud goes to read a doc file. And when it enters that directory for the first time, it'll load that file, that cloud.mmd. And same with under your source directory or under anything else. And so you can have it you can have all these little bits of extra context kind of scattered all over the place. And when plug goes to read something from that area, it will just get that so that it kind of can orient in that directory properly.

Does cloud do that automatically or is that something you have in the harness here? No, that's the harness does that. It didn't used to do that, but it does that now. Um, and I don't leverage that too much, but there's a little bit of, you know, I'll have an extra 100 tokens or so just to kind of um, and that's again from watching it. So I like I'll see that when it goes into this test directory, it always does this weird thing where it goes and tries to look up a certain pattern and this project is different and it doesn't follow that pattern. So let's just put in a little blurb uh you know two three sentences that says we don't uh try not to call it the things that we don't want and just say in this project we follow xyz and so that one sentence or two sentences gets dropped in anytime cloud goes to read one of those files under that directory and then that, you know, that weirdness that it was doing uh never happens again because it understands what the right pattern is to look for.

Okay. You try to include Claude MD files in different places to basically give Claude context that it might need if it's ever traversing around those different places. Yeah. Yeah. But keeping it super minimal and generic and um like it should it shouldn't be anything that's actually tied to the implementation really. a lot a lot of you know, if you run slashnit in a new project clog will go kind of fan out and it'll come back and it'll make up a nice like um you know directory tree diagram or something that now has to stay in sync forever anytime you go add a new directory you need to go update this MD file because otherwise it won't think it exists you're right so do you know just skip all of that it shouldn't anything that it can easily discover just by uh agent search you know if it can rep it or have a sub agent go fan out and do some stuff and discover it great. But if it's always always always every time it goes to read your test files, it does you know slightly the wrong thing, just drop a sentence in that says what the right thing is just so that it stops, you know, it because otherwise it wastes five tool calls right there. And that's it figures it out. But like that's a bit of a waste of context window that doesn't give you any value whatsoever because it has to rediscover the same pattern over and over and over again. So it's just easier to put 20 tokens in that just gets automatically put in.

So you Okay. So you have this this harness in your skills directory like a skills um uh repo that you're that you shared in there part of that uh which part of that actually goes and references. Is that part of the the autonomous development like uh bashcript? No no no I'm talking about just what cloud code does like the CLI harness? I didn't know that. Okay. So cloud code automatically do. Oh my mistake. Okay. I see now that makes sense. Okay. Um, got it. So, you that that makesense.

I mean, I've got, you know, feelings on a lot of the these little features that cloud code puts in like they keep adding stuff and shipping more stuff and I don't think most of it is the right thing to do for the majority of cases because that goes back to the bitter lesson thing of you're trying to create this really intelligent scaffolding for the model. But if you drop that all away and just let the model do its thing and throw more tokens at it so that it's allowed to go explore more and understand more, it's going to do better. And I find that that's almost universally the case. A lot of people are using uh PI now as a harness. I don't know if you've seen that one. Um pi.dev. Very very tiny minimal harness. Barely does anything at all. Very extensible. If you want to add extensions and stuff to it, you can. But it's it's a very very thin layer. And it works really really well because it doesn't do anything. It just lets the model do what the model does. And pi.dev pi.dev. Yeah, pi like the number. Oh, I went to pie.dev and it actually goes to http bin. Oh, sure. Which is interesting. Someone decided to host that. Uh, so sorry. So you're saying that um the the goal is so you think that uh claude code is shipping too many they're almost taking the Rails approach where they're shipping like everything you could possibly do. They want to put it all in cloud code instead of something minimal and extensible. Is that what you mean? Uh yes. I mean I wish it was even taking the Rails approach because then it would be more coherent. They're just shipping stuff. Um and not all of it's good. Um, they have walked back some things, but I I'm not a big fan of just how many changes come in constantly that makes the models do stuff slightly differently. Um, and I'm not a big fan of the cloud code harness, but I want the claude model because that's what You're not a fan of the cloud code harness. Because I my experience like I find cloud code well I mean okay client in the past was always really good like Klein um was was probably the my first favorite harness for doing any sort of like agentic devel software development. It was really good. Um but then I actually found like cloud code the the harness was fantastic and I've only kind of used that one even compared to a lot of the other ones that are out there. Do you do you still not you don't feel that's the case anymore? Um it was I mean it was the first right. It's so by kind of nature it was the best because that's that's all that there was. But now we have I think most of the alternatives are actually better probably. Um but you're not allowed to use your subscription with those. Yeah. Right. There's a you know nonzero chance of having your account locked out and not being able to act, you know, call the anthropic API on that. And the subscription is such a ridiculously massive discount that I will put up with the cloud code harness just to have that because like the $200 max plan if you're fully leveraging that is actually good for about $3,000 worth of API credits. Um insane value there and that is worth putting up with the nonsense that they keep shipping. Um so that's that's about it on that front. But like if if I you know ever did have to pay API rates for things then I would do something else. I think I just not sure what that would look like. Also I mean the harnesses don't have to do much is the other thing. You really don't even need to give them proper tools. Uh you can get away if if you let it write bash. If you have a single tool called bash that lets it do everything else. because you you can then just tweak your system prompts a little bit. So you can give it the concept of skills. It doesn't need a skill tool. You can say just go cat the skill file under this directory and you can have your your harness can do the same thing that cloud code does where it just grabs the name and description metadata from the skill file and injects that into the system prompt, right? Um, and so like you can do a little bit of stuff around the system prompt management there just so that it knows kind of what exists in the world, but beyond that just let it run stuff and you know it can run all the you know normal Unix code core utils things effortlessly. So that's all it needs really is just a bash tool to go run commands and it can do everything else from there. Um, and that's maybe not the most efficient. I think, you know, there's there's different ways to do file editing and stuff that's maybe that's worth a dedicated tool or two, but for the most part, you don't need them. And then the harness really is doing nothing at that point. And the harness is just running API calls in a loop. It's it's kind of just Ralph at that point even. But what you're saying is basically it's not that hard to build a effective harness anymore. Like um and and the CL you you think that like the clawed code harness is not the best what they're but we're kind of locked in because you can't use your like the deal that you have the the discount you get through using them with quite frankly one of the maybe the best model in the world right now. There's there's no better place to do it. So it it's not bad enough that you switch off of it.

Yeah. Yeah. It's definitely like it it still works. It gets me what I need. It just does a lot of stuff and they keep making it more complicated and adding features and it's it's one of those things where like this could have just been done. You know, it's true. I really like the idea of software that's finished. 100%. You know, like I I never need to worry that the, you know, LS is going to change under my feet because that is a finished program. Yeah, I'm sure that it still gets maintenance doing something right. I don't care because I know the features are not going to change. It's going to list contents of directories for me and it's going to have some flags to, you know, do stuff for me. It's done, right? Most, you know, core utilities like that are completed finished programs that, you know, just sit there. I think a agent harness should be that kind of a thing where you're just connecting it to different models as the models get better over time, but otherwise doesn't need to do anything unless well, there's some specific things like for example, how you have some of those keywords in your claim empty files like there's some specific things that you can do to like improve or or optimize your use of of a particular model. But I think that's becoming less and less common given the RLHF like the the reinforcement learning that's being done on all these models now.

Yeah. Well, so that's exactly the the bitter lesson thing though is that the capabilities of the models are going to increase better than whatever your intelligently designed fancy harness is going to do. Okay. Um and so the harness they do a lot of stuff to try to like babysit it into being better in some way, but that makes it worse in other ways. And there's a weird balancing thing going on there. And just when the next model comes out, that problem's probably not going to exist anymore. But we still have all the croft of that being baked into the harness now because the other model needed it. And you know like you just you only need like the very thinnest lay layer here um to do these things. You could literally do it as a bash script that's just running curl and jq to do the API stuff. Mhm. Um and that would that would for you know especially these top models if you're doing you know new GPT or doing Opus that's all it needs at all and you will get excellent results from even just that.

So when are you going to be making your own harness that uses all these these skills and the approach you use um with different models? Um I mean I've made a couple. They're they're like they're not complicated. No. I I made one uh a couple of weeks ago that's just in pure assembler even because it's really it doesn't need anything. I just wanted to see what that was like, you know, like uh you know what is this? And it's kind of cool because the the binary then the resulting program was like seven kilobytes or something and it uses no memory whatsoever other than what gets you know allocated for handling the JSON messages and right, you know, it's super lightweight. You could run it on literally a potato probably and uh it doesn't take anything is is the key bit right now I think like it it's just the cost yeah the cost of tokens. like it it's literally just the cost on the you cannot call the AP you can right I've done that before too of of you can pretend to be cloud code and go through the oath flow like token but and and I don't like did you see the drama around that back a couple months ago with open code they they cut off every open code user yes I did and I just rather not have to deal with that you know I like my cheap tokens they're really good tokens they're very valuable tokens I would like to keep Yeah.

Yeah. That that is until until more and more of these I mean eventually these open source models are getting better and better. Um, they're probably already caught up to Sonnet 4 in some cases. Yeah, that's pretty damn good. That's the Yeah, I mean that's the silly bit too, right? Is there are models that are equivalent or better than what I was doing last year, right? And so these skills would feed into those models and get me what I want. But you want the best but the there's still the quality is not quite the same and it's just I can you know I better exists by the time you know there will be by the summer or by the fall even an open source model that matches what opus 4.6 would do but there will be an Opus 5 or something and then why do I want Opus 4.6? I'm never going to use Opus 4.6 again. Even though I could get 4.6 quality from a cheaper model, maybe it's a tenth of the price or something. I still want the better one. I get that. Yeah. But and then the question is like at what point though is it good enough for every like if it's doing everything you want it to do which it kind of is right now like Opus 46 right now is building everything you want it to build with your skill system like there have you run into anything with your skills that you have that it has not been able to build?

No. Um one thing I thought uh because I I'm doing some silly stuff here just to try to push it to see where is this actually not capable yet. Um and so I had it writing actually x8664 assembly um building a multi-threaded concurrent runtime inspired by goes where it's got go routines and it has a you know work stealing scheduler that can multiplex everything across operating system threads. Thought this is really some tricky stuff. This is complicated. There's a lot that could go wrong here. Um and it did it really really well but there was a couple of small bugs. I mean it worked through there's big bugs at first and it worked through them. Um you get the feedback loops dialed in correctly and it could just work through them and it'll do that but there was uh there was a heisen bug in some of the multi-threaded code where um it would go routines would silently just disappear like they get orphaned somewhere in space and but if you added tracing or telemetry to try to observe that happening that would change the CPU timing enough that it would stop happening. Oh and so when you go to measure the thing, you change the thing you're measuring. And that was really, really diff. I didn't think it was going to solve that. It was churning for probably 12 hours trying to figure that out. And then I stopped it and I said, "Okay, this is

"I think not working." It was doing Ralph loops the whole time, too. So, it would spin up. It would look at it with fresh eyes. We look at the bug ticket, look at like the previous investigations, they would try to, you know, and they would just kind of keep turning on it and uh I like this is not going anywhere. This is a really hard one. I don't fault it at all because I think people would struggle with this one a lot too, the way that this is presenting.

Um, but I said go open a new T-Mo session and launch Codeex over there and pair with Codeex on this. And they talked through the problem together.

"No."

"And they solved it in about three hours."

"Okay, hold on. Hold on. Hold on. I can't believe you're only talking about this now."

"Sorry."

"I can't believe you're only talking about this now. After we've been talking for like over an hour here, like Okay. So, so, so this is for the people that are are are dedicated to watching this whole thing through. You'll you'll catch this. Okay. So, the walk me through this. You have Codeex and so many questions. So, you have cloud code and you're using your skill system and it's it's building this thing. It is trying to solve this bug which um I want to come back to like how do you do you are you just manually prompting it to try to solve the bug or is there something built into your okay so the skill systems."

"It had it had a goal of here's the plan of implement this crazy multi-threaded runtime thing. Right. And it had a bunch of tickets and it's working through those and everything was good but at the end of the the plan there's validation of like does this thing work and we're doing we're writing tests and we're writing load tests and we're writing a whole bunch of different stuff to try to make sure everything is working properly."

"Um, it came up with this concept of a soak fuzz test, which I'd never really thought of or heard of before, but it's basically a soak is it's a thing. It's where you just let your system run under not under full load like a load test would do, but it runs it uh looping through all of the different permutations of um configuration that it can have. It sounds like I mean it's fuzzing but it's configuration but through."

"Yeah, it's fuzzing through the set of configs basically and so like for a multi-threaded runtime it's you know how many operating systems are we running here the default is however many cores the CPU has but that will you know if you've got 12 cores that will behave slightly different from eight cores which will behave different from one core if you run make it run single threaded so run it through all of those and run load testing things against all of those and let that go. And so it was going and it was running those for you know 12 hours or so and it was finding failures in certain configurations and it's goes okay well what is that it would create tickets then to go track that and then it would keep running the autonomous loop then to resolve those tickets and."

"It worked through almost all of them. I mean it worked it found a ton off the start and then it would incrementally do just test-driven development. So okay the soak found a failure. Let's write a test that reproduces that failure and then figure out how to make the test pass. And it knows how to do that. It did that great until it got to this one final situation where you could run the soak for five hours and at the four and a half hour mark uh one of the results would come back that you know this thing should have iterated 10,000 times and iterated 9,999 and where did that extra go routine disappear to that should have put in that last result? Yeah, there's something wrong here. And that, you know, was very very sneaky. And it ended up being, you know, I think ultimately like a five-line change or something like that. It was the ordering of one instruction. Again, it's assembly. So, it's machine instructions here of, you know, load something into this register, go do this, you know, and it just had one thing that like, you know, it it moved a value on line here when it should have done that six lines earlier or something like that. And it was just an ordering thing there that caused the scheduler to just very, very rarely just drop a go routine. And you know that was incredibly sneaky to track down and it but with a bit of help it got there."

"Okay. I want to understand though how you did this like you had Claude code call to pair programming with cursor sorry with Codeex which was in another through a team session like tell me so was this you had to manually step in and do this."

"That was that was where I I saw like it was spinning on this problem for at least 12 hours. Like I went to bed, I got up the next day. It was still trying to figure it out. Um, and I thought, okay, I don't have a whole lot of confidence you can do this independently. Maybe we need a different perspective on this. Um, so T-Mox is just a command line tool that has uh commands on it for sending keystrokes and for capturing pain content. So you can it can take a basically a snapshot where it gets all of the ASKI that's in the other terminal and then it can read through that. So it can see what Codeex has wrote. There's also a scroll buffer in there. So, if Codeex dumped out a whole lot of stuff, it like COD can page back through it. But basically, Claude then just works with with Codeex the way a person would where you just type like you can do run the send keys command and it types those keys whatever the message that Claude has is. It'll type it into the box, hit submit, Codeex will go do its thing. And so then it's just pair programming and Codeex would come back with I found X. And Claude would go, okay, well, so while Codeex is doing this, I'm going to go look at this other thing. And then they would, you know, be looking at different aspects of the problem and doing all that."

"I didn't give them instructions on how to do any of that. I, by the way, I I just said open a new T-Mo session, run Codeex, and use it as a pair programmer. That's that was my entire prompt."

"Oh my god, that's crazy. So, you just told it to open up a new T-X session with and use Codeex to do pair program through this problem. Did you say like through this ticket, through this problem you're working? Like what did you point it towards?"

"Um, yeah. Yeah, I think there there's a little bit, you know, my wording was probably longer of just like I've noticed you're having a struggle with this one. New cloud session, by the way, like I stopped the loop, started up a new session. I said, here's a ticket with describing a bug. I've noticed it's taking a long time. Um, and we're having problems with it. Open work through it with Codeex. Like, open a new TMX session and pair program with it. And so then it knew what the bug was, I guess, that it was supposed to do. It told Codeex what the bug was then and they just started going and I walked away. I came back three hours later and it was fixed."

"And that was"

"Why do you think that is? Do you think it's because Codeex just gave it a different like Codeex is better at solving these problems or it's giving it a different perspect like a different perspect perspective is a weird word to use with LMS but a different like training model that that like attacks the problem a little differently and having those two together is better than greater than the sum of their parts."

"I I think there's a big factor there. Like I know that Codeex has this reputation for being kind of relentless once it's you know you've given it a task and it will boil all of the oceans in the world to make that happen even if it clobbers 90% of your codebase in the process. Um, it will get that thing done for you. So I don't want it to go nuke 90% of the codebase if it goes crazy here because this is a really weird bug and if you because it's all multi-threaded code too. So this was if you add telemetry or tracing to try to catch it that changes the timing between the two threads enough that it never happens again. And I mean maybe that's a fix. You could just like leave the tracing there and then walk away but that's not right. Yeah. So, you know, I know that Codeex will kind of do this relentless thing or at least that's what I hear from people and but I know that Claude is good with keeping focus on the goal here and not destroying the world in pursuit of the goal like it, you know, does the right kind of higher level thing. And so then using Claude to oversee Codeex to keep it in check, um, I think worked really well. But also they were working through the problem differently because uh cloud still has sub agents and so it would it would send something to Codeex and say okay we should probably you know what do you think of this and Codeex would start going and doing stuff and then it would also have sub agents simultaneously doing stuff and I have no idea where the magic was in the process. Um I'm pretty sure I did actually have Codeex take a crack at it first though and not get anywhere obvious with it. Um, and so then that was, okay, I don't want to go figure this out. Maybe this is the deal breaker. That means that this, you know, this task was beyond the capabilities of the current models or something. Maybe I found my, you know, that's that's what I'm searching for is where does this break? What can they not actually accomplish at this point? Um, and so it was just get them to work together on it. And again, you don't need a person in the loop there at that point. It's just"

"Okay. So maybe the models individually are not quite there but in aggregate AI is there and can solve this kind of problem. Um, that's just weird, right? And then aside from that really, really tricky bug that needed this, you know, AI pair programming situation happening. Everything else, it just does. You know, it can I've not had Opus fail at delivering a good solution to anything I've given it since, you know, since 4.5 in"

"Oh, four or five."

"The end of last year. Yeah. Um,"

"That that's that's wild, man. That's a crazy story. I got to I'm gonna have to keep that in mind for big problems is to have pair programming Codeex. That's crazy. I I've never heard of that and it makes that's just the implications of that are crazy. Um, one last thing. Yeah."

"Just you you mentioned a Heisenber. I've never heard that term before, but I can I can I can guess that it mean it refers to Heisenberg and like cross quantum mechanics where you you by the act of observing it kind of makes the bug kind of not show up. Is that correct?"

"Yes. Yes. Um, and I mean I've seen I've used that term. I'm pretty sure years back with some weird rail stuff actually that again that you kind of go to you try to measure it and it stops happening or or the thing that you're measuring doesn't actually align like it changes the measurement enough that it doesn't match with the thing that you're ultimately trying to measure. Um, but I didn't use that in this case. Claude ca that was Claude summary was there bug here. Um, and"

"Wow."

"You know, so he was trying to work on it, but that's very difficult because you're kind of stabbing in the dark a lot with come up with a theory and see what happens and if the bug still exists or not because when you actually go to watch it in action, it goes away."

"Yeah."

"And you know, that's like the trickiest kind of problem. And I'm sure that humans would have struggled with that same one for quite a long time."

"Um, the best expert humans that have done a lot of assembly in their life have probably, you know, they probably have some mental guard rail there that they know a pattern that would have prevented it or something like that. But that's not, you know, that's that's a hundred people on the planet or something like that at this point. Not a whole lot of folks who are doing you know regular assembly programming scale performing you know yeah assembly programming at this kind of level too. A lot of if you're doing assembly it's because you're trying to work on very resource constraint microcontrollers or something like that. You're not trying to fully saturate a thread ripper you know like it's kind of the opposite problem and you're not doing high concurrency high parallelism stuff like that. And so I don't know how many people have, you know, I've worked with quite a few folks who have done assembly stuff. It's all very singlethreaded. You know, go read this sensor, go put the value over here, go do this, whatever. And they would not have ever touched a scenario like this, I think. And so I don't know how many actual humans would have solved this either. You know, it's"

"Yeah."

"Really weird state."

"But the LLM did, which is crazy. Um,"

"Yes."

"Well, I have like 500 different things I want to talk to you about now, but but I think we we've gone long enough. It's been an hour and a half now is"

"This has probably been one of my favorite one of our talks in a while. Um, and I think that a lot of people are going to get really good use out of the skills repo that you that you put together. I know I'm going to make use of it at the very least in my personal time, if not at work, because I think it just is going to be really effective. Um, and uh, you've given me a bunch of ideas now that like the sky's is the limit on what to build. Like I kind of again every time I think about what what crazy things could I do now that I have like the proper harness and methodology to go and do it. It's just the the barrier to entry is that much lower. So thank you."

"Yeah. Yeah. Yeah. I mean that I would I would really like that to be the thing, right? It's like the barrier to entry for all of this should be nothing. You should be able to just install the app or something. We have, you know, there's a cloud desktop app. Codeex desktop is actually pretty decent as well. Install, you know, whichever. You should be able to just paste in a blurb to that agent and have it go pull in, you know, the skills or whatever and set itself up with all of this. And you just shut it down. You open it again so it gets the new config there and have it do whatever you want at that point. And I think like we're super close to that being the reality for all of this is you can just just have stuff at this point."

"I want to Well, sorry, one last thing. Um, I was going to ask you if you still use like a way of sending notifications to you during development, whether that's through Telegram or like notify.sh or something like that. Do you still do that? And if if not, do you have I I know you haven't tried this, but like would Claude remote not be able to do something similar? It's not going to give you notifications as you go, but it's going to be like a way of monitoring it in a sense."

"Yeah. So, I have um I I SSH from my phone so I can kind of just see what's going on. I think the terminal works fine for that. I haven't tried Cloud Remote yet, but I think it would be roughly equivalent experience aside from if they've got push notifications or something. Um, but I again going back to I'm already hitting my limits. I don't need it to go faster necessarily. So, you can just wait until I'm back at my desk for the most part. Um, but it's also like I like to do it I like to do it so I can see what's happening. I like to see kind of updates of what's happening. But if you I don't know if your skills repo has a way of like in the git commits also doing a git push because if that's the case then I could just what I would do is I just monitor my git repo on github and see as it change like commits those changes."

"Yes, I do have it. It does regularly commit everything that it does. It doesn't push them though and I have that because I don't want stuff just going out into the world that I haven't looked at yet. Um, so usually I'll come back and there'll be 50 commits or something and I'll kind of go review the ID there and see, you know, did all of this actually do what I wanted."

"Um, that used I used to need to keep an eye on that. I used to actually go through it quite thoroughly and make sure everything was good with earlier models. That was when I had notifi set up as part of the the shell script here. So every time it started a new loop or if it there was a human escalation ticket that got filed or anything like that, it would send me a push notification so that I would know, okay, my thing is broken here and I'm going to go back and figure that out. Um, but the things don't break anymore and I don't really need that and I know that it's just going to work. Um, and so there's a lot of it is just for my own gratification of being able to watch it do the magic or something. Uh, you know, but I don't need that. And so I don't have push notifications anymore. I could do um I don't have not going to do the Claudebot thing. Um I have a thing for that though that's maybe for a different conversation. Yeah."

"You know an alternative there that I like a lot better but I'm also not really using because I kind of I want it to be when I'm at my desk you know this is what I'm doing and then when I get up and go do something else I want to go do the other thing."

"You want to be focused on Yeah. You don't want it always on your mind because that that leads to the whole AI vampire problem which we briefly talked about before the call started is like it's very easy to take over your life if you're always thinking about it. You get burnt out really quickly and then it also loses the luster of it. It's like it it takes over and and is no longer as enjoyable because it just sucks everything out of you."

"Yeah. Yeah. Because like you can give it tools, it can text you or something or you can have it, you know, you can put it on a schedule or something to go do a bunch of stuff and then ping you when it's done or do whatever. But I want to not do that. You know, it's just I want to be able to check out and say, 'Okay, my workday is done here. I'm going to go do other stuff. I'm going to go enjoy dinner and do enjoy my weekend.' I might check in on on the weekend or, you know, I can go from my phone and I can say, 'Okay, start the next stage of planning or something,' right?"

"Um, but that's not going to be what I'm doing that day. It's just I can I can spend five minutes to write out a planning prompt or something and then let it go and then I know it's going to be good for the next five hours and I'll check back later. I don't need a ping then because I know that it's going to be done at this point. Like I've got very high levels of confidence in in this stuff. Like it's I know that probably for this level of complexity, I've done enough of these now at this point and I've watched it enough times that I know for this thing it's probably going to take three hours and this one's really big. It's going to probably run overnight and I'll check in the morning or something and I kind of just know where that lines up and I don't need notifications then because I just know where it's at. Um"

"That makes sense. Okay. Yeah, that's more I think for me the notifications are more of like um it used to be for notifying when I needed to provide input, but then it became about like just of interest of us passively being able to go and see all the cool things like I would get notifications on my watch of like what it's built and I just thought that was really interesting."

"Yeah. And I mean like like notify makes that really really easy, right? It's just a curl call away. So you can just have Claude go put that into the the shell script or something too that at this stage in the loop you want to get a ping on it that has this content. Um, but that's that kind of stuff should be optional, I think, and not the default experience because then you just start getting bombarded with all this stuff. Then you end up with Claude code."

"Yeah. Yeah, that too."

"And all the extra stuff. Anyways, I will let you go. This has been so helpful and exciting and energizing and um wonderful and uh I can't wait for the next one."

"Yeah, absolutely."

"Thanks, Greg. I'll talk to you later."

"Enjoy your weekend. Yeah. See you later. You too."

Get Our Mobile App

Ep. 5: The AI Agent Workflow You Can Copy to Build Better Software Today

Transcription