State Of Agentic Coding #3 with Armin and Ben

Is this healthy? Is this unhealthy?

Oh, it's definitely unhealthy. I burnt from my $50 food in a single session.

I have no interest in going faster. It feels really fast. Is our job going to turn into requirements engineering?

Hello. Welcome back to Aenti Coding. This is episode 3 with Ben and me, Armen. I'm a aentic coding addict for a good year now. I started a new company called Arendelle. This is what I work for. uh nothing publicly announced yet, but if you're kind of curious, you can go to aranddale.com and there's our purpose statement and a little newsletter you can sign up for, but we'll get do some updates soonish, but so far still a secret.

Oh, hey Armen. It's good to see you. Always good to see you. So, my name is Ben and I'm a career JavaScript developer. Unfortunately, worked at companies like Discuss and Sentry and now I have my own thing, modem. Arm and I ultimately started this show because we've, you know, going back a year plus, we'd exchange a lot of notes on how we're using AI coding tools and agent coding tools and at some point we decided it was worth kind of sharing our take of um how we see things and that's where we're at.

If you saw last episode, so first off, we recorded that January 8th and by the time it came out it felt a little late. It feels like a ton of things have happened at the end of that episode. We started what was going to be our annual predictions, but we quickly pivoted into our monthly predictions almost as a joke because we were laughing at how quickly things were going. And um turns out that was the right call because some of those monthly predictions came out true. So I made this declaration and we're going to go we're going to go through them. So I made this declaration that wow like Claude code had had a really great holiday and I thought that cursor was going to answer in a big way. You know cursor had been so huge and kind of dominant over the last year and it just seemed natural for me that, you know, in a month's time they'd kind of come out of the gates and say something or or have something. Armen, do you think that came true or not? It doesn't seem like I mean I think that the only update that I saw in cursor in the meantime was the custom harness that they built and with which they then built the browser which was impressive but it wasn't something that uh has an meaningful impact on a user of cursor.

My coworker Justin highlighted like oh they launched a CLI and I noticed that when I was using the editor I did feel like I was getting some prompts but I thought they already had that. So is that true? I I'm pretty sure that someone at one point gave me access to a beta to CLI and I feel like this was in summer. Like maybe I'm mixing stuff up, but it really felt like it was already.

This caused me to go and look up like the release notes for cursor CLI and it had You're right. A beta did happen in the summer. Like it's already I presume you're looking this up like I think it's had multiple versions. I felt like they had a release in the fall but it could have been quiet like I'm not sure. This is going by hazy memories here and this felt like a muted relaunch. Either way, I don't know that it really connected or really resonated. Is that fair?

Yeah, I think one way of looking at this is that cursor now is targeting I think the big people enterprises. They are probably very successful at this. They're just now somewhere else in a way. And I think that there might actually be quite a bit of really interesting stuff happening if you're the target audience. They might just be looking at different people at this point. They're they're pretty big and and I'm pretty sure they're they're no longer necessarily selling to the individual developers on Twitter.

Also true, right? Like we're in a weird headsp space of what we're seeing. So this is a this is a real like VA vibes evaluation like probably inside cursor they're like you know we just added a billion dollars. We don't [laughter]

Yeah. I think it's always good to remember that Twitter does not necessarily represent the market as a total. It's just a Yeah, it's a it's a wild early moving space. Um, and whatever is the hottest is the hottest.

That prediction unfuzzy, unclear. So, let's move on to the next one. You asked me, and I thought this was a great question, especially in hindsight, like, do you think there will be more or less harnesses? We freewheel these questions on the spot, by the way. So, I was I was trying to think on the spot but my argument was you had made a strong case over the course of the episode highlighting that, you know, there's really like a lot of innovative features coming out in harnesses. And so my logical conclusion was kind of like, well, if that's still true that we're probably going to see more harnesses because there's room to innovate and if there's room to innovate, people are going to try it. They're going to try things. I'm looking to you to answer these questions cuz I think you have a better pulse on this answer. Did do we have more harnesses or less?

So we definitely have more or at least I think that you can argue that there are more particularly if we consider that we are moving towards like basically taking the core principle of a coding agent but sort of applied to like a little bit of a greater distance to the code. So that is on the one hand that there's clot core work. It still runs bash it still runs clot code but it's like technically it's a harness. Uh Cordex also relaunched I think the desktop app. I feel like maybe they launched it. They might have launched it. I I also thought that there was a thing before that I didn't try. [snorts] Uh but they got a lot of buzz and then I think at this point at least um pi moved from obscure to there actually quite a few people who are playing with it and there are more uh I mean like it's at least if you if you follow Twitter like every day something pops up and and some of them are just rappers but for instance like I know that was named Will. I think he's a a Python developer that used to write a command line utility abstraction library. He wrote a harness called toad. I see that show up on Twitter all the time now. And lots of people putting cursor into uh VMs on phones and stuff. So uh I think there's not sorry not cursor um code like the whatever the the library that you can rep. So yeah, I think that there's probably there's probably more now than there was at Christmas. Codework is a great one. Whether whether codeex was new or not, I downloaded it and tried it and to me I you know that that's a good example. I give it if it relaunches or not like I at somehow I felt like it was a buzz and I I decided to download and play with it. This is worth talking about. Codex I think is the first sort of like let's call it bigger player harness that has workree support baked in. Is that true?

So actually I think this is an interesting thing. The the clot app that I think very few people actually use that sort of like the chat GPD thingy. If you switch it to code, you can actually on the Mac take a repository and if you then give it a task, it will actually create a work tree. But I don't think anyone is doing that and it's doing that for six months or so. But um in theory you can do that. So like the claw desktop app that they already had for a while, not the cowork one, the regular one has work three and and local agents if you if you use it, it's just totally undocumented or it's not undocumented. It's like un undiscoverable. I guess

I think this is a real highlight to the value and the power of marketing of these harnesses, right? Even if a competitor had this feature for six months, if none of us noticed, it may as well not have had it. And I think that Codeex landed and I saw people promoting that it had work tree support and that got my attention. I was interested.

What a question. I was sorry for the detour. It's like is anyone actually using work trees because like I can't figure them out. So I moved to just having multiple checkouts. I am a caveman when it comes to this. The UX around work trees is horrible. And I feel like there was a period where like a couple of months ago everybody was building their own abstractions over trees and I was also one of them and it just didn't stick. I just I went back to multiple checkouts. So, I actually kind of wonder if if this is how many people actually are really using work trees. Um probably a lot, but um definitely didn't stick with me for sure.

I use work trees a lot via conductor and and we might talk about this later. And I think this checkouts versus work trees we could get into, right? Like doesn't um Boris Aclaude just he just uses checkouts. That's something he's talked about. I think Peter is also one of the clo the checkout people. I have I have strong arguments for why check out so we can go to this later. It's definitely I feel like work trees are sort of like um they they are sticking around in the disc in the in the sort of discourse and definitely some tools are promoting them way more now than they used to.

Last question is and this is this feels like the biggest one because this was in the final minute of our of our 51 minute episode. you made a prediction which is and this was January 8th. You made a prediction which is like I think personal agents are going to be a bigger deal. And at that point in time I don't know what the popularity of Claudebot was because I'm in the bubble where I follow you. I know what you're talking about and I follow Peter and so I knew that it existed but I certainly wasn't ready for what where you know wherever it was January 8th to where it got to even just a few weeks later. So, I'm gonna answer this and say yes, personal agents got very big, super big. I mean, to go from that, you know, to go from that statement of like I think, you know, personal agents like Colomb and and by the way, I don't like dropping names here without explaining. So, can you quickly um I don't know how to pronounce Peter's last name. I'm just gonna be very

Steinbagger.

Thank you. So [laughter] he is also known as a stipete on Twitter.

I think that I think that Claudebot OpenClaw is like pretty well known but like let's just humor the idea that you haven't followed like can you give us like the 60-cond version of this?

So the 60cond version I think of openclaw what's it called now or like in between it was moldbot and claw this and claudebot is you can think of it as like a coding agent hooked up to your personal communication platform of choice be Slack or Telegram or WhatsApp and it basically has a memory you talk to it it programs and runs the code and because it's sort of skill-based it will reprogram itself to some degree. Actually, quite quite a significant degree. And so, as a result, it is turning into something like a personal like an agent that controls a computer. Let's put it this way. Um, yeah, and it got really really popular in the last couple of weeks.

Uh, it drove Mac Mini sales through the roof. I I don't know if that's true or not. Like, people joke about it like, can I not get a Mac Mini right now? And that's also a question which is why a Mac Mini, right? Like why not a Raspberry Pi or you know whatever you know piece of junk Windows machine you have sitting around.

Maybe if we walk her back a little bit. Um, so also part of the reason why it wasn't completely uh crazy to make the prediction early January was that I think a lot of people have tried to build personal agents in one form or another over the last nine months. There were two events in December I think or November somewhere between November and December that uh I think where you have already seen a little bit that uh Peter's approach of a personal agent uh gained a little bit of visibility elsewhere. One was that he uh merged a contribution from a community member that extended what was originally just a WhatsApp plugin to one that goes in Discord. And so he hooked the his personal one up to Discord and he tells a story that there was this moment where uh he kind of forgot to turn it off overnight and it restarted automatically. Um, and people sort of like went crazy with his personal um assistant that was basically having root access on his computer over Discord for like 12 hours and then afterwards he like woke up scared and and read through every message and was actually impressed that it didn't completely screw up. Um, but the reason that story I think is important is because it was running on his Mac. And one of the things that Peter did the nine months before is that he went very deep on CLIs. And so what the um what Cloudbot primarily operates is a bunch of the tools that he wrote before and one of which is called Peekaboo. And because he uses a Mac, Peekabboo is basically a remote control for Mac. Um, so it can click buttons in the UI or use Apple script and all kinds of other stuff.

Is is Peekabboo like a like a independent open source library? You know,

it's like a command line tool is a command line tool that you can use with anything. Um, but um part of also I think how open claw works right now. I mean not not like technically but in in terms of like how people are using it is clawhub or whatever it's called now which is a repository of skills and there was there was a skill for uh remote controlling a Mac [snorts] and if you are into the Apple ecosystem and you want your agent to have access to your iMessage or iCloud or whatever then you kind of need a Mac because that is a closed ecosystem and the only way in which Apple gives you access is via a Mac. The other reason I think is there's always question like why why is it a computer at home? Why is it not um an EC2 instance or I don't know a headset machine or like a lineode is that uh residential IPs have a way better reputation than cloud IPs for a lot of services. So if you have ever tried to launch a browser in a home IP or in a cloud IP, the number of captions

for now for now

for now. [laughter]

Well, actually here here I have a fun story, fun little story. My son spent his pocket money to buy a projector. I'm going to tell the story everybody know. So my son bought a projector, Chinese projector, €60, very very cheap, very good projector. The side hustle that this company does is it operates a VPN endpoint on that machine and sells residential IP access. Um, turns out it's a pretty common system now where either companies or individuals are participating in reselling network access through residential IP addresses for reputation purposes. So, it's it's a market now. It's crazy.

So, hold on. Are you saying that this projector that you put onto your network is basically creating a backd dooror for the Chinese company to gain access to your residential IP?

So I was I was looking this a little bit. It's not a Chinese company that actually comes out of it. What actually comes out of the end of it is a whole bunch of companies including like VPN companies are buying residential exit points.

But is that okay but not a Chinese companies. But is was that statement true? Like you got a cheap projector but you also sort of bought something else unwittingly? I'm trying to understand.

Yeah. Yeah, that's this absolutely the case. [laughter] I found a way to get rid of it, but um yeah, it's it's fun stuff. [laughter]

That's a whole topic we could get into. Um

but it is related to this because if you actually look around on the internet, one of the problems that agents run into is that they want to browse the internet. And to browse the internet, you want to search the internet. And the there's actually only one search engine in the world basically, which is Google. I mean, there's a little bit else. And so coming out of a residential IP with the same trust gives you usage.

Yeah, I use duck.go. Okay, I use both. Okay, I'm lying. I use I use both. So Cloudbot, we did took a little detour on residential IPs, but that was fun. This thing got crazy. I think you did a good job explaining a little bit why. And I and I think Armen and I talked a little bit about this like we didn't want to go too be we don't want to get too deep into Cloudbot. There are so many places for you to learn more about that. But I did want to talk about it from the perspective of agent coding, okay? Because that's what the show's about. I if there's one gap that I feel that I am personally missing as a developer and I suspect other people do too is that I'm mostly doing everything kind of on my machine. And if I'm not around, you know, like, yes, I can kick things off or whatever. Come back later, but I'm not on the road. I'm not walking down the street and being like, "Hey, you know, hey, Claude Code, go make me whatever." I'm not doing that. But it seems like in following some people who've been really engaged with Claudebot, sorry. Open Claw. Open Claw. Open Claw. Um, really easy mistake to make. Okay. really easy for stake. Every single one of these episodes has us basically ba basically asking lawyers to to not come after us. It seems like there are people uh who are doing, you know, working with their personal agents to build software. And I'm curious how true that is. Are you doing that? Is it a good experience? Um, should every should [laughter] everybody go out and get a Mac Mini? What do you think?

I think everybody should try to build a personal agent. I think that's true. Mostly just because I think that this is sort of I think going to be a topic that's with us for a while. I don't think you should buy a Mac Mini. [laughter] Mostly just because in the grand scheme of things that's a little bit of a crutch. But I think there's if you're into programming I think like having at least made the experience of like toying around these distinct things is probably a good idea. Should you use it for programming? I think it's a different question. I don't think it's the right structure necessarily. Part of it is if you actually just want to use it for programming like even the claw app gives you or codeex gives you like an incloud make me a pull request kind of thing that can also run code which just has a better user experience I think but each to their own I think like you can definitely use a personal agent program programming but I don't think that's where the strength is necessarily

I'm going to recant what I said earlier where it was like oh I don't really do it on the That's true but I have experimented with it using the using the the apps that you brought up codecs and claude web but my experience has been let's take codeex on the web for example and I think I think cursor agents cursor cloud agents similar thing right if you've used that you have to do some work to make sure like to give the agent like an environment where it can run your tests and do that kind of stuff I've struggled to get it to I think emulate kind of what I have on my machine and I always find as a result that the I find that the results are lacking like I'll give you an example on the modem codebase we made a decision to all of our unit tests use Postgress so um which is a whole topic we could get into and maybe we maybe we'll get into that a little bit now when I started using cloud on the web when that came out and I think it's still in beta cla web agent, I don't know the exact name, has a a more locked down kind of environment like they have almost like a like a network firewall for you can stop it from making outbound requests. And I found that even if you gave it the most permissive network connectivity from that environment, it still struggled to connect to our like uh cloud Postgress instance, they're using some kind of um proxy and I just, you know, I spent an afternoon on this, I couldn't get it to work and I gave up. And so that's an example of like even though I use Claude predominantly, I couldn't quite get the same environment going. Um, and you could imagine, you know, hitting more and more challenges. So the idea of, hey, I could run this on my Mac Mini or my Linux machine or or whatever or my VPS and it's, you know, I control the environment and I know that it's going to work. and the idea that I could chat with it over Signal or WhatsApp or whatever the heck they're doing. Um, that sounds appealing. So, are you kind of like, does that make any sense or or you're like, you know?

Yeah. I mean, first of all, you're right. And I also don't use it. Um, and I think like the biggest biggest reason why I'm not using it is because like the def is not really set up to work on this on demand kind of online in status. hard to look at diffs

hard to look at but but also like let's say for inance you do want to have like coding on the go there's a very obvious alternative that I think like people are using which is like you have just a computer at home and it just runs the thing and you connect to it remotely and that is actually in many ways not completely insane if you're a developer and I think quite a few developers have a setup like this the other thing is that um one alternative form factor here that I've seen pop up is for instance things like row code or robot or whatever it's called which also sounds like a bot but it's basically if you have an Android phone you can download the thing which I think it's called I forgot what's called termox I think it's basically a terminal emulator you can run on Android and then it can run your programming agent and then you set it up once to connect to your thing and you can literally vibe code on your phone as if it was a terminal also probably not the greatest setup But you you have full control over it, right? Um and I have I've seen some people do stuff like this. I personally I just feel like I already programmed too much. So try to do less on the go in a way now. Um, but yeah, I think they're like better ways than that. I think

so. My take away from this and is that you know Cloudbot great personal agent but probably not the you know if you're looking for it to be building software for you is not it is not the form factor you should be reaching for. I think like there's a there's a there's a conversation we should go into later, but I think that's sort of like maybe a little bit of foreshadowing which is there's a huge desire I think by a lot of people to just and I was in this too which is like like you almost feel like if you're not shipping on the go what is going wrong with your life basically and uh I I feel like I personally moved past that point. Um, [laughter] so my my strong desire to actually keep doing that as excessively as I was doing is is a little bit less now than it used to be. And and and I've seen other people also slowly fall into this pattern.

That's where it's revisiting because I think even when we started this show, which is not that long ago, you know, and we weren't even sure if it was going to be a show, we were just kind of like, you know, let's just talk about it. was um even you who spends a lot of time thinking about this, we declared ourselves to be uh less crazy than others. And I think that we got I I feel this way. I feel like I got a little maybe over the last month I like got more AI pill than I've ever been. And and it's [laughter] and it's not healthy. And I've even noticed um like a trend of people talking about this like sort of the the healthy ramp, you know, is this healthy? Is this unhealthy?

Oh, it's definitely unhealthy for sure.

Yeah. Um

my my my fancy bed tells me I don't sleep enough. This is how I know.

Yeah, I know this. And um I don't know how much other people know this. Um to stay on top of this, maybe this is worth addressing. like to truly stay on top of this is like madness.

I feel like for for many years in my career, there was sort of like a there was like a a path and the path was a little bit like when I was like between 16 and like 22. I was like working a ton, sleeping very little and and it just felt like the the world is an oyster or whatever. And then I just got old, right? Then I sort of became responsible and I was like, "Okay, this is just how like like you're getting older, you're doing less of this." But actually, what I realized is that like the career that I was going through turned from like really fun hacking around on random to being a lot more organized. Um, and now I feel like I'm back to how I felt like in in in in sort of the time between 2004 and 2009 where actually I think our industry was a little bit less well defined and and I I actually there's like of course like the addictive effect of the agents, but there's also like all of a sudden I feel like I can do so much and I want to try so much. Um, and and everybody's in the same kind of mindset of like I can do everything I want, right? I built a a an an Ethernet driver in JavaScript the other day to sort of sandbox my agent and I would have never done this before, right? And so I don't think it's just the agentic addiction in itself. It's like okay, you can do so much, but I think it's also like now everybody's in this mood of like someone said like it feels like the early days of the internet and it really does feel like this to me again. It's like it's the early days of like our industry in a new way.

I agree and that's what's making it hard. I don't have a great answer. I think we got a little bit off topic. So, you know, this is state of agentic coding. So, first of all, let's just say this was a crazy month. This just felt like the craziest month.

I want to talk about pi a little bit for like 30 seconds.

So, look, we've talked about pi on this show. I suspect for many people, you've even introduced people to pi because it was pretty lowkey there, but I understand that pi is a pretty big part of open claw. I I don't know if this is true or not, but my understanding is it has certainly gotten Pi has gotten more attention uh as an Asian harness lately. Whether that's open claw or not, I don't know. And you know, so what's going on there?

I think that around when like everybody was looking at Open Claw, I have seen some people write about Pi. I I know that I released a blog post that got a lot of views. Um and I also saw Toby from Shopify retweeting Pi. Pi was officially blessed as a coding agent by uh OpenAI. So it has uh full access to the codec subscription with its own system prompt. So definitely seem people have sort of started looking at this and you can also see this a little bit by the GitHub stores and and a bunch of people sort of jumped on to sort of explore and and and tried it. So, it definitely has um like a hot 15 minutes of of popularity, but I think it's also like more people now looking at I think minimal coding agents as a sort of building block to build their own agents. I think if you really want to build an agent loop yourself, if you actually look like how do you do it, you actually don't have a lot of choice in many ways. You basically have Pantic in Python, you have the Vela SDK, you have PI I think and that's it. Or you just go straight to the underlying libraries. But but I don't think like there's actually that much choice in the market today. Um so I think it is actually kind of interesting.

I'll add to that though that if um there are other choices, but I think they're cloud-based like Cloudflare workflows arguably I think is like having an agent loop. Is that fair? But it's for running it in the cloud, you know. I don't

But I think you still have I think you still have to like an to actually talk to the AI provider. I think they I don't think they have an AI SDK. I'm not sure.

Yeah.

Because we'll come back on that.

Yeah. But but maybe like so so why is it even relevant at all that you have a thing here which is like how hard can it build to be an agent? Well, it's not super hard, but particularly with entropic, if you do it wrong, um it's going to be way more expensive because you get the caching wrong, for instance. And it's actually there's a there's a bunch of stuff you need to handle, or like Pi, for instance, does automatic image resizing like a bunch of other things are doing so that um the agent doesn't break. Um so there's a there's a little bit of stuff you actually do need to get right and and PI gets the core parts right and then you can can explore with like really creative solutions around it if you want to. And I think more people started to to to to play around with it. Which was really interesting to see.

So, we'll check back on that. I guess it's like we'll see where where Pi is. Man, there's a lot of news to cover. I think Oh, we did ask like what did people we asked on on on X or Twitter like what do people want to talk about and somebody I know I somebody raised a a comment I left a comment that was like there's we have written down so many things that like I don't even know that we're going to get to it, but we're gonna we're gonna try and do it in a quick way. So other news that happened in the last month somehow cursor built a browser from scratch. Enthropic built a compiler C compiler that can uh compile the Linux kernel and I guess it I don't know if it compiled it compiled Doom. This feels like a change where now we're seeing these you know we're having agents complete large projects. Um and we've obviously had a bunch of model releases which we've we haven't even touched on that yet. What does that mean? I guess maybe this is our good way to bring us to the model changes.

Yeah. So that have been someone else's prediction, right? I remember Simon was talking about the predictions on the oxide and friends podcast and uh at the end of the year he was like this will be the year where someone will build a browser from scratch. And the reason Simon did this prediction, I think a lot of people did similar predictions was there was a growing consensus that if you have a good test suite then uh you can you have like a something to develop against. And that was definitely true for the browser because there's a there's a browser test suite that uh Chrome and Firefox and Ladybird and I think a bunch of others are building against and obviously uh if you build a compiler then yeah tests from other compilers and then at the end there's a a kernel that can boot or not. Um, and so there's there's there's there's a north star to work towards. I think what's really impressive about those is that they did it in much quicker time I think than people anticipated. I think the browser I don't know but it was like less than two months right and so it was definitely impressive and I think is this is kind of it's like this a roar shock test of like where you're standing on this because like a lot of people are like oh it's not impressive at all because it's in the weights and the other one's like oh it's super impressive but then it's like actually it's really really really slow. It doesn't even hit like baseline performance of like an unoptimized GCC. I just think it's cool. Uh I I don't want to like judge it in any way, but it's like it's definitely like it it was not able to do that six months ago, right? And it's able to do that now. So let's see where we're in six months. And a lot of people are now talking about like how do you orchestrate these sort of like agents of agents and stuff like this and like even code released an experimental multi- agent mode with T-max and like a bunch of teammates or whatever they call it. So So that's happening. We'll see.

So question quick question. I didn't see what uh so anthro so opus 4.6 came out not even a week ago codeex 5.3 we're going to come to this quickly so anthropic built their cmp compiler using opus 4.6 what did cursor use to build the browser like was that composer or was that something else as far as I know it used a combination of opus and codecs but I can check it's not their own model for sure uh cursor browser model uh should be 5.2.

Okay. So, this is fun. Cursor comes out and announces that they built a browser uh with GPT 5.2 and then like what a couple weeks later that's deprecated and we got GPT 5.3.

Yeah. Um do you remember we had we talked in December I think was like everybody all of a sudden was like Opus 4.5 is so great. But it was actually I think in many ways like four months four months four weeks after it actually initially launched that took a while for people sort of to acclimate. I feel like it's the same right now with with these model releases. Like I I have no opinion on them right now. I'm using them, but um like I think it will take us a little while to to realize like are they different? Are they the same? So I don't think it's deprecated. I definitely right now I think I use 5.3 exclusively instead of 5.2 because I don't see a reason why I should use 5.2. I guess um it feels faster. This is what I'm noticing. It feels noticeably quicker. Um, and I really didn't have much of reason to use the other one, but I don't think its programs any better. Maybe it does, but wouldn't wouldn't really have noticed. But I noticed the speed.

So, obviously model improvements are are a factor, right, in terms of building these more complicated pieces of software. But I have to believe the harness too, right? So how much you have how much do you think like there's sort of what I've learned is I think everybody has learned is that harness innovation is very real and probably like a really um impactful part of um the ability for for you know agents LLMs to build software. So both these experiments, both the browser and the compiler, they use the custom harness and custom harness that basically is, as far as I can tell, two loops. One, which sort of continuously cranks out tickets in form of files for other things to pick up. And also, I think they also all just use the file system to to dump tasks to to places for the other things to pick up. There's obviously a lot of talk about gas town and things like that which is like the sort of a more colorful version I guess of a sort of this thing but maybe the slightly less absurd one is is just a built-in support in cloud code now which this team work team member team team lead whatever they call it. So, it's there. Um, and I tried it and it's kind of cool, but there's innovation in that. And also, I think maybe we should talk about the fast mode in cloud code.

I'm going to tell you this episode this episode feels very challenging because there is just so much to get into. So, look,

we should talk about fast mode. I forgot about this.

Okay, so look, let's just let's just start this off. Let's talk about Opus 4.6. All right. Are we I presumed you meant Opus 4.6 fast mode. Let's bring everybody up to speed here. Okay, so OP Opus 4.6 launches what? February 3rd or 4th? It was Thursday, I think last week.

I agree. Armen brought up like normally we take some time to to decide whether things are good. I have been using it kind of like a a a ton and I think this is a good moment especially if you haven't if you don't know us or you haven't been following us or or or listened to this before is like you know Armen and I are both trying to build software businesses here okay and we've been doing it for for months um modem is a real product I think that when we talk about evaluating models I think that we're talking about it with real stakes in the sense that we're trying to build products and get them out there. We're not just vibing um you know some indie project that uh you know makes $2,000 a month and then you try to sell for 2.2 million. [laughter] That's side joke. I guess I'm sharing that to say that like I think there are real stakes and when I'm evaluating a model like it's important, you know, I've got a team I work with like it has to work. It has to solve our problems, right? And is that fair by the way? I don't want to speak for you, but is that you know?

No, I mean like it's true. I just also don't necessarily know if like going to like put a lot of scrutiny on a model. It's just like does it work? Does it not work? If it wouldn't work, probably would go to back what we had before. Just doesn't seem like it's objectively worse that I have to go to an older version. I think that's sort of my take on this right now.

Right. I think the one thing it does do for me is like if something is working and I feel like it's given me great results, I'm not super motivated to like I do want to experiment. I do want to listen and try other things if I think it's going to sort of advance like what I'm working on, but it has to be really good. So, I've been using I think I had something like I think I merged like 80 poll requests over the over the weekend. That's a lot for me. I squash commits, by the way. So, [laughter] it's quite a bit. I think Opus 4.6 is really good. I think it's like I don't know. These are all vibes, right? But it it could be like objectively on a on a benchmark maybe 10% better, right? I don't know. But 10% could be dramatic, right? In terms of your ability to work with things. Uh like my pulse check is that I I feel like I can operate at a slightly higher level. I feel like I feel like compaction is good in terms of sessions being able to go longer without feeling like they're and this is with Claude Code. You're right. I feel like compaction is really good. And that's like my vibe test. It's like am I am I shipping a lot of stuff? Do I feel like I'm am I, you know, do I want to do I want to keep going because I'm having a great time. I'm achieving a lot. I think that Opus that's like my benchmark and I felt I feel like it's pretty good. So

yeah, I mean I don't have a lot of like I don't have a lot of opinions on on on like base opus. I think like the main opinion that I have right now is they also released the fast mode. I'm not ready to pay money for that I would say but uh that's new in the sense that if you pay six times as much in API credits you get two and a half times the throughput

and I burned for my $50 included in a single session.

I wanted to I wanted to segue to fast mode because look I think that I think that already the agent is is like it's good alone. Never never mind fast mode. Thursday, you've got in my mind a, you know, a better a better LLM, a better a better model and a better experience with Claude Code and probably with a lot of the harnesses. And then on top of that, what was that two days later? Fast mode. So now here we are. So you have that new model and then on top of it, you can just use that model way faster. How many times faster?

Two and a half times for six times the cost.

For six times the cost. And you blew through it really quickly. Did you like what you saw? I'm a little bit conflicted on it honestly. It is cool. It is really cool. It's like it's objectively cool to see it go at like Oppo's performance just like double the speed, right? And and I think it's really cool because like you can sort of feel like if we actually are going to get models that are this quick, that will be very nice. It's also at the same time kind of like telling the story a little bit that speeds like that is going to be really expensive. So maybe we're not going to land with like the $200 a month sort of price point. Maybe it's going to be a thousand, right? This is sort of like I feel like this is sort of like this slowly sort of getting us used to the idea that we're like this is going to be 10k or more a year. I feel like this is why this is happening. But we'll see. Maybe not. Who knows?

I think for me and I'm, you know, these days I'm usually running three to four different sessions at once and I'm kind of like a crazy person jumping around to each and kind of like spinning plates, keeping them going and like pre, you know, testing them, bouncing around. I have no interest in going faster. It feels really fast. [snorts] And I'm personally bottlenecked by my ability. And I think this depends entirely on, you know, what you're building. Are you supporting users, for example? Like if you've got no users, doesn't matter that I'm, you know, I'm performing QA or, you know, sometimes I'm tweaking a UI and I want to I want to like with my eyes, I want to evaluate it. Like I I think that human in the loop and taste is really important and I don't I I you know, two and a half times faster. I don't need that. So what I do think though is that there is a class of problems for which that is probably very valuable and it goes back where we started this section off which is big agentic projects like building a browser or building a compiler and projects like that need you know very defined sort of like win conditions about what success looks like that the agent can kind of keep going until it has achieved that and that win condition or whatever you want to call it is is in code whether via test suite or or something right it is not a human eyeballing and saying looks good

Is that fair? Yeah, there's a bottleneck, right? And the bottleneck is me. But there are also types of tasks where I'm just actually just waiting there, right? So it's like, I feel like if I could press a "turbo" button, like on demand, it's like, "Now go quicker." I there are situations for sure where I would press it, but overall, like this thing is idling most of the time at this point because I am not capable of context switching that much and actually setting a quality bar that I'm at least okay with.

Somewhere that is a segue over to like, "The Future is Terrible" offer, because like, I'm freely feeling like this right now, but it's because everybody's so convinced that they have to ship faster and faster and faster. I think it's like, you sort of like, you get into this mood that they have to, but it's it's also very, very hard to call the quality bar. I agree. And even though I think Opus 4.6 is really good, and I think I think it's smarter, or I've just deluded myself to think that it's smarter. Like, I still find that it, and all the coding agents, make bizarre decisions all the time that you only really that you only really uncover by digging into the code and looking closely, and then, you know, that's when sort of a little bit of the illusion gets pulled away.

Um, well, they they write terrible code. Like, there's like no denying. Like, as as much as we, I think, like the reality is that if if you have an agent write, 95% like, I can objectively say that my codebase is not good quality standard. I wish it had, but also it's good enough for what I'm trying to do right now. Mhm. But I I think if I if I have to rely on that kind of model for the most critical pieces of my infrastructure, I would have a problem, right? And I think that I hope in like six to nine months that's less the case. But yeah, it's like, you you cannot vibe yourself to to everything. So there's there's an engineering part too. Um, and there just doesn't work yet good enough. At the core of it. Like right now, I'm building the sandbox, and it's it's it's good for a lot of it, but like there there's a lot of cuffs I need to rewrite because it just doesn't it doesn't work.

One of my reactions to seeing the um, the big agentic projects like the browser and the compiler was, man, I wonder if I can frame more of my problems with like, you know, these sort of uh gates or wind conditions so that the agent can be more successful and go further. And that maybe, you know, call it test-driven development or or call it something else. If I work that way, maybe now I could achieve more parallelism or, you know, it makes sense to go faster. Is our job going to turn into requirements engineering?

I don't know. I I really don't know. I think like right now, I'm sort of like on the Gartner hype cycle. I'm sort of in a slump a little bit in that um, I I feel like we have hit a point. I think it's going to get better for sure, but I feel like right now we have hit a point where we have figured out how to make the thing loop, right? We have figured out like like this, you're you're going to write a bunch of this like requirements documents. It's going to do something, but we have not figured out how to make the thing actually produce code that is actually good. And so the limiting factor right now is not my ability to write PRDs. Um, my the limiting factor is actually I need to do less of that and actually do a little bit more engineering. That's that's the state that I have right now. Is it going to be different in two months or three? Probably. But right now, I'm actually pulling back on that quite significantly because of the problems that it would cause if I would lean in more on that.

Um, in part because like you actually get incredibly lazy and don't look anymore until you sort of like go back at this like three days later and like, why the hell is why is this feature in there? Why is like, why is why is it doing that? Um, and it becomes problematic when it starts breaking, right? Because like all of a sudden you're like, "No, that like there's you can build yourself into a codebase where nobody in the company actually knows what it does internally, right?" And that's scary because like there will be the moment where you have to understand it. So I I don't think that I want to build myself more towards like what you just described, which is basically uh clear clear passing gates only validated by machine. Maybe in a couple of months, but like not right now.

Okay. You're on a detox is what?

I'm on a I'm basically on a detox. Yeah. But I don't think it's like I don't think it's like because I made a necessarily particularly bad experience. It's just like I take the model for what it does today, and there are clear limits to it. Um, and I guess the question is, and I heard there a bunch of people say, is like, is can you actually just vibe your way through until the better model comes around that just cleans up all the mess that the old model made? And my suspicion is not yet. So, I don't know. Do you want to take that gamble, I guess, is the question. And I I I don't take that gamble right now. That's like a parallel to like pollution and many other uh many other problems that humans create for themselves. You know, someone else will fix it. Yeah.

All right. So, look, we've been recording for a while. Man, there was way too much to get into. I feel like one, I do think that there seems to be almost like a almost like a model allegiance divide. Like, we've talked a lot about um brands, but perhaps even styles. Like I noticed that some people are really into the Codex way of building things, and some people are really into the Opus, you know, way of building things. And we didn't even get to KamiK2.5, which is also very substantial. Um, and another way perhaps of building things. That's a whole topic on its own that I would love to know more of, because I know that you use Codex more lately, and I've, you know, kind of just leaned in on Opus. Not enough time to get into it, but it's something I'd love to get into on like another episode maybe next month or in between. And if that interests you, like somebody say something in the comments.

All right, so let's get into predictions.

Yeah.

Yeah. So, let's go into predictions. And I guess we're just, man, we're just doing it. We're doing monthly predictions. Armen, you want to go ahead?

So, I think in a I think I think in a month, we're going to see more people either launching sandbox solutions or talking very prominently about sandbox solutions. And the same thing is going to be true about um, how to orchestrate like, basically a productized version of Gas Town. Like, either the coding harnesses will themselves ship a version of it, or there will be like a a plethora, if you call it that way, of third-party solutions to orchestrate basically self-running doom loops of of agents. Um, I think we're going to have this by the end of by the early March.

I think that's a reasonable prediction for two reasons. One, I barely know what Gas Town is, and I know who's behind it. I know ostensibly what it does. I just haven't had time to really think about it. That's it, right? And this is always, by the way, I think this is actually a good uh way of sizing up our predictions, because we've tal in in in doing these shows, we've talked about like, you know, Armen is the 0.1 and me as the 1.0. So, I'm behind, right? I'm I'm both ahead of probably like a lot of pe I'm definitely ahead of a lot of people, but also behind. So, you know, I was behind on Clawbot, for example, right? So I think I agree with it from that perspective in that it's not like a huge thing right now, right? It could be. The second thing is this. One of the reasons why this mania, this like collective monthly mania is happening, which we should I we could talk about this, but I don't want to say just influencers, but like there is an influencer set. There are the model companies. Like they really want something new to talk about all the time. Okay. And the reality is enough time will go on, there's kind of like hunger and demand for like what's next. And I do think that the natural kind of like, you know, conclusion of like, if we take open claw forward, is that, you know, sandboxing will be a bigger deal, and we'll have videos about that. Here's how you sandbox, etc. So that's like my like, I guess I'm plus oneing. I'm plus oneing your prediction. You're better at predicting. Although, you know, I think it's I like the questions because then we get to think about it, right? I would have never sat here and said, "Oh, of course there's going to be more harnesses." But because you kind of phrased the question that way and we could think about it, I think it it led to a decent answer.

So here's my question for you. By all accounts, Google is a big winner of the AI era. You know, their their stock is doing well by all accounts. They have tremendous models. Um, their image model is great. Nano Banana, VO, um, they make TPUs. Like, no one is questioning that they're sort of a frontrunner in this in this race. However, and I do know that I like I know that people do use Google models for code, but they don't seem to hit in the same way that say the Anthropic models have, or maybe even Codex. I'm not sure. By the way, Google lawyers, I'm not sure. I'm just these are all vibes. But my question for you is, do you think in the next month, like do you think Google is comfortable with that? I think that there are teams over there that would love to do more of this. Do you think that we'll see them kind of like approach, you know, this agent coding market in a different way? Like they have, to be clear, they have Jewels, they've got Anti-gravity, they've they've had like approaches that are out there.

I I don't think it's going to change in the next month.

Not at all.

I mean, I But you ask like, do they want to? I'm pretty sure they want to, but both Microsoft and Google are deeply dysfunctional in actually being able to somehow ship this to uh in into an environment where like they're really, really resonating with the user, which is shocking because like they have all the resources and somehow some something in the company prevents this from actually happening. It's wild, but but it's hard to change, right? And and I don't think it's going to change this month, but I think they will they will put a lot more effort into it because like they they put so much money into infrastructure investments and everything, and I don't think they want other people to to collect all the like the the value. Uh, I think they will have to eventually walk up the chain and build an end-user product that someone wants to pay money for.

I can't comment on the dysfunction thing. I have no idea. But I do think there's a branding challenge. Like there's already a lot of players. Is Anti-gravity, and by the way, Anti-gravity is their IDE that released a few months ago, I think. Um, is it good or is it bad? Man, I have no idea because I got no time to pick that up. I think that's a bit of a challenge. And I think there's like a branding and a positioning challenge that they have. Like it could be that it's fantastic. I don't know. Right.

Yeah. But that brings me to the other prediction that I have.

Okay, good.

I think it might be hard to say like this will be in this month, but I think the discourse will very quickly shift towards the death of the IDE.

Yeah.

I I see already now that AMP, for instance, killed their IDE extension this month. So that's gone, or it will be gone. It's on self-destruct. And you already mentioned at the beginning that Cursor has a they advertised the IDE, the sorry, the CLI extension. Now I I feel like that if not this month, then next month, we we are starting to like really, really talk about we need different tools than an IDE to work with agents.

This is a great prediction. And I think this presents probably illustrates why my prediction last month where I was like, Cursor is going to like, you know, they're they're awesome, they're going to come back. And you could argue they have, right? This is the same thing. I think the same problem as Google, just in a different way, right? They do have a CLI. I think I've heard some positive things. I don't have time to do it. And the other thing, reason is, what is Cursor? Cursor is predominantly known for template complete and our IDE that is sort of their position in the market is it's the best editor. It's the best editor, editor, editor, right? And now it's hard to say that you're that thing and to also say, by the way, we've got a fantastic CLI, right? Um, this positioning is what works so great for open code and pod code too, right? And it arguably works well for Codex because they never really had an editor. And um, there's something about sort of committing like if you believe that this is the future and you believe that there you're not going to have an IDE in the future and you're just going to vibe your way to a solution. I know that some people use vibe in a very dismissive way. I use it mostly to describe what you just talked about, like I'm chatting with an agent, I'm building code, I I'm still looking at the code, I guess. Right. Right. So, I don't I know that that's not like sort of the origin of it, but I guess I'm referring to this kind of workflow where I'm I'm mostly doing the kind of chat-based kind of flow. What do you think about that?

I'm definitely not open to the IDE much, right? It's like I'm I'm sort of talking to it. Um, I'm I'm I I like it after a while. I like the the TUI, but I also think like I think we have to be somewhat fair to Cursor here because like I think they know that, right? Like I I know that if you go to the website of Cursor now, it it it there's no mention of an IDE. The the the UI that they show you is basically their agent view, right? So I like they they I think our perception of what Cursor is is in some ways also no longer reflective of what the product wants to be. And I don't think that the thermal user interface is is definitely the right choice here because it's it's very limited, right? But I think the the sort of having a bigger distance to your code and and finding other ways to review it. There's just a lot of pressure to move towards that.

Well, look, I guess this is also where we can bring in a little um a little inside baseball. It even if Cursor internally knows that and they want that and I think they're a great, you know, company, they build great products, is that it is hard is this kind of like branding challenge is that if you're known for one thing, it's hard to move into the other thing, especially quickly.

Right. Especially quickly. And at Sentry, we had a bit of this problem because we were known for so long as the error monitoring tool. And when we started bringing in other products like, you know, tracing and session replays and now they got logs. Um, you know, it was it was hard to actually get anybody to escape that idea that it was more than error monitoring. We've lived that and it's interesting to see that play out now in a different way, just like on a wildly accelerated timeline to these other products.

I think it's fair. It's

Yeah.

I feel like building in the AI space to some degree. It's like you built this great new castle and like three months later, someone is already digging at your foundation with an even bigger castle, right? I feel so it moves up there in in sort of ambition and possibilities. Um, and if you are known for the small castle and someone builds the bigger castle, then

Yeah, it's it's it's it's interesting. I think

if I made an annual prediction, by the way, I think that it would be like what's after the CLI? And I think it's just it's just the chat app. It's almost like open claw is like a is like a year early for this, like where we started off. Just to bring it full circle, there could be a future where you are just so you are you have so much trust and you're just Jarvis building with with your with your bot, and we're not there yet, but it could just be that that is the future UI. I guess the question to you then is, do you think you're going to have the same app for putting your kids' PDF that you got from the school into your calendar app as your coding agent, or are these separate things? Cuz like in some ways you're describing the Uber app of like this just the thing I talk to and magic happens.

I don't know. I guess maybe I'm I guess this is lending lending uh credence to the theory that yeah, maybe there will be the Omni app. And um, you know, the software industry is in trouble. Stay. And that feels like a good place to stop. Like, stay.

All right. Thanks for being with us and leave some questions. Maybe that gives up some insights for next episode.

Bye. Thank you. Wife.