Casey Muratori Doesn’t Care About AI (Here’s Why)

Casey, last time we spoke last summer, you stated, "I couldn't care less about AI." I really couldn't care less about AI. It's completely uninteresting.

But then recently, you just started a podcast on AI. "Hello everyone and welcome to the very first episode of Waiting Through AI." What changed?

Uh, nothing really. And I think uh people who have watched the podcast kind of know I play the straight man on this podcast. My friend Dimmitri Spanos who's been a AI researcher for decades now, you know, before it was cool. He came to me and he basically said, "Look, I want to be able to talk about AI." And he doesn't, you know, he didn't have a public platform. He he wasn't, you know, a YouTuber or anything like that. And he was like, "Do you want to do a series where we talk about this because, you know, I want to I want to be able to talk about it." And I said, "Sure, because I love Dimmitri and I love talking to him." You can see for people who've watched any episodes of that podcast, he's just like a real straight shooter. He knows his stuff very well and he really doesn't have an angle. He's just like he's always been interested in AI and he's kind of motivated to try and just talk about it frankly. So, I've been really enjoying it and it's also educational for me because since I don't work with AI, the structured time when Demetri and I talk about it, which we might not otherwise do, right? And so, we get to have an hour or two of of of talking about AI every couple of weeks. So, that's really it. Um, but I'm I'm sorry to say for folks out there, I still don't actually do any AI work myself. I know people are oftentimes like, Casey, you should just go try it. Like, you know, see see what you think. And uh, it's not a value judgment. Again, like I think I said on your program, it's not me saying I don't think it's useful. It's just me saying it's it's not my thing, basically. So, so I don't know. Does that make sense?

It does. I When we spoke last summer, I had not yet been AI pilled. Okay. And so for me that occurred late December of 25, early 26 where I finally realized that I've been moving the goalposts in terms of AI development. I'm like, "Oh, okay. Well, it can't do leak code questions." And then it started doing leak code questions. Then I'm like, "Well, it can't like pick up a bug and then fix it and then like check it in." And then it started being able to do that. Then I'm like, but it can't take like a BRD, like a business requirements document, and then like decompose it and then like start picking off the items on that BRD. And it's like, actually, it can do that now.

And so I'm like, wait a second. I'm the problem here. I'm the one that's saying that it can't do these things. I got into a little bit of a funk to be quite honest with you because I was just like, well, I spent 20 years in this industry writing code. Like I've been a trad coder for 20 years and then now the AI can I can just see I can just see this line of sight between where it was maybe four years ago to where it is today. And then you I mean I just can't help but just draw the line and project it out into the future. It's starting to do some pretty amazing stuff now. And so I'd love to to get your take on it as well because it seems like something has changed when you maybe not as large of an amplitude as say me, but something has changed with you.

Yeah. No, really nothing. Um, uh, so if you remember on I actually did two shows around the same time. I did your show and I I had done one with Marco and he asked a similar question because it was kind of it was an emerging technology at that time. It was it was it had kind of been established that it could do some interesting stuff already. It wasn't like new new like GPT4 or something. It was more like okay this is clearly working right now. It's debatable how well it's working but it's definitely like working. It wasn't n it wasn't really nent. Said the same thing in both programs which is like it's just it doesn't interest me. That's really the thing. I've never really been someone who wants to make a bet on what it will or won't do. And the reason for that is I think that's really difficult to do. Even if you were much more knowledge about it than I was like say Dimmitri he's also unwilling a lot of times to make a firm that's like okay will it be able to do this won't it be able to do this when will it be able to do this right for various parameters and so to me I think it's always been just a mistake to approach AI from the standpoint of like I do or don't want to use it because I do or don't think it will eventually be able to do something or it won't eventually because the truth is nobody has any idea right it could be able to do everything eventually or it could hit a wall or it could be anywhere in between those things, right? And so I think that's the wrong way to approach it and I've always approached it as like am I interested in doing this thing and I'm not. Right? And so that's what that's why I said in your program that's what I still say today and it's also why uh you know if you watch the the podcast I'm always just like yep I'm just here to kind of like get the information out of Dimmitri. he works with it every day and he actually does research on like AI sort of repeatability like getting determinism out of it and and you know that sort of thing for industries where that's important. So I would say nothing's changed and I think you know if I went back and watched my answers to your uh questions at the time I think I'd say the basically exact same thing.

Well, let me push it back on that a little bit. Like you don't see a progression from where it was to where it is because Yeah, absolutely. What I'm hearing you say is like, does I have the capability that I care about? Yes. Now, yes or no? No. And in a little bit you're like, does I have the set of capabilities that I care about? Yes or no? It's like, oh, some new stuff happened, but no. And so it's this sort of like point in time thing, and I'm trying to say like over time.

No. Uh, so again, like I'm sorry to accuse you for not watching your own program. What I said on there was that it doesn't interest me. And there's a difference between those things. I think I gave the analogy of like guitar playing for example. Yeah. So uh you know you can go to other industries you can see where you've had times when we've had computer or you know it doesn't matter computer but technological things where you could now do something that you previously had to do by hand but now you can do it using some piece of technology and typically they look you know kind of similar to AI in a lot of ways where it's like you know if people I'm I'm pretty old so I've lived through uh the 80s you know at that time you had like a like a drum machine you know like you know in the 70s they had them 80s they had them they sounded pretty bad. You could kind of tell it was a drum machine. And at that time, you know, if someone was looking at that going, "What do I think of a drum machine?" There's two ways you can look at that. You can look at it as, "I don't like a drum machine because I want to play the drums." That's been my opinion on AI, right? Or you can look at it and say, "I don't like the drum machine because it doesn't sound enough like drums." And that is an opinion that can change because if we improve the drum machine, like today we have much more sophisticated drum machines. We have drum loops. We have things that try to simulate what a real drummer does, you know, based on actual how drummers would approach particular pattern, all these sorts of things. You may now change that opinion. But if you were someone just wants to play drums, that doesn't matter, right? That's the goal was not, and I think I said literally this on your program, the goal was not to get a drum sound that you wanted. It was that you wanted to play the drums. And that's why I'm not interested in AI. It's why I wasn't interested in AI then because like I I do think there's some pretty cool stuff about it even at that time like the natural language like anything that can really take a natural language thing and produce anything useful has traditionally been a very difficult problem uh in computer science I think I said that on the show as well and so I've always thought that was really cool but I don't really care I I'm not super resultsoriented when I do these things to me the answer doesn't change even if the AI is doing like a better job than me it's an activity I want to do. So, you know, if I want to play drums and someone's like, "Well, this drum machine plays drums better than you." I'm like, "So, like I you know, there's also drummers who play drums a lot better than me and that never stopped me from wanting to do it, right?" I don't play drums, by the way. Using that as analogy.

Well, let's let's let's poke at the analogy a little bit. So, programmer is the same way. And the same thing about there's programmers who are better than me. That never stopped me before. So, even if the A got to the point where it's programming better than me, that wouldn't really change the fact that it's not my thing. They put me out of work, but it doesn't change.

This episode is brought to you by Linear, and they just published something that I think is worth talking about. They opened with the new manifesto, issue tracking is dead. And what that means is that the behaviors around it, like the manual logging of issues, the prioritization back and forth, the grooming sessions where half the room is just trying to figure out what's even in the backlog. These things are a thing of the past with Linear. I lived this nightmare at Amazon. We had mechanisms for everything. And don't get me wrong, I'm a big believer in mechanisms, but more often than not, the process of tracking the work took as much energy as doing the work. You'd come out of a sprint planning meeting and think, "We just spent 2 hours so eight engineers could figure out what they already knew they needed to build." What Linear has done is rebuilt the platform so that the stuff happens automatically. Your code, your decisions, your customer feedback, all of that context all lives in one place. and linear actually works from that. So when something comes in, the system already understands what it is and where it fits. Nobody has to triage it manually. If you're an engineering leader and you want your team building instead of administrating the work, check out Linear. The link is in the description.

Yeah. I mean, okay, so let's talk about So from what I understand, the drum machine was there because like drummers are a fickle personality. What you really need is somebody keeping time and a beat in the background. Traditionally, the drum has never been the focal point, right? It's very important as like a base of a band. And for songs, you needed to keep time, that sort of stuff. From what I understand, like the the first people that were using these drum machines are like, "This is great. Now we don't have Casey trying to go off and doing a drum solo and trying to do something interesting because we need him to do like the most boring thing in the background which is to like keep time on a loop. And so like is the unit the band or is the unit the drummer, right? Like you write software. I get it. There there are better software developers than you. There are worse software developers from you. You just want to make the software. But the software is in service of a thing, whether that's a product that we sell or you know part of like a larger organization or team that we are a part of right and so like at some point if the drum machine is just like better than you yeah it's fine I just wanted a drum but like but what if nobody wanted a drummer anymore this is why I say it really depends on what you are thinking you're doing right so if you are thinking that the reason that you got up in the morning is to to make Microsoft Word. That's the thing that you wanted to do is to make Microsoft Word, right? Then really the activity that you do do during the day is doesn't matter to you. You know, this is very philosophical kind of, you know, where do you fall on on on the philosophy of life, right? Is it the journey or is it the destination? So, if you get in the morning and you just want to make Microsoft Word, then it's like, well, then it doesn't matter, right? Do do I go to work and I I write some code? Do I go to work and I manage some people who write some code? Do I go to work and I tell an AI to write some code? But do I just tell the AI to write Microsoft Word and get Microsoft Word back? Right? They're all equivalent modulo whatever the quality is. And so if that's your perspective, then what you're asking is the point in time question you were talking about before. Like what you're asking is, is the AI producing a good enough Microsoft Word? And maybe it's not. In which case, that's a legitimate criticism of the AI. Certainly there's also the was the AI built ethically. That's a concern we talked about your programmer as well. That's another concern. But at the end of the day, those concerns are different than am I getting the fulfillment that I wanted out of what I was doing, which is getting the Microsoft Word. And if the AI gets to the point where it can produce the Microsoft Word, I am because that's what I said I was in this for, right? If on the other hand, what you wanted to do was program. That's what you were in it for. Then you don't really care about those other aspects, right? Obviously, you would like to produce a good version of Microsoftware when you're programming it because probably if you like to program, you also want to do a good job. So, it's not like you don't care at all about whether or not you're producing quality work. You do, but you have to be the one doing the work. Otherwise, you didn't care about this. You weren't there just to make Microsoft Word. That was a byproduct of the activity you actually wanted, which is programming computer. And I think this is kind of the bifurcation about why, you know, some people care about AI, some people don't. Anyone who just cares about the the end product, I think they can shift over time. They could legitimately have said they didn't like AI a year ago and legitimately say they like AI now. And that's not a philosophically inconsistent position, right? People who are just in it for the programming. I just want to program a computer. I don't see that attitude changing because it wasn't about who was doing a better job or whether the AI was screwing up and that's why we didn't like it.

Do you style yourself as a coder or a programmer or do you style yourself as like an engineer? I don't know that I style myself either way, but I'm definitely someone who just likes the act of programming. So, I would say a programmer or a coder. So, I think also software developer, for example, would be a term I wouldn't use because a software developer is someone whose primary goal is to develop the software, right? And especially like if you look at what I do even today, a lot of what I do is sort of more like research and development kinds of stuff anyway. It's like, hey, let me do these analyses for performance and let me show people how these things work. And that sort of educator is actually probably more of a better term for the kinds of stuff I've been producing over the past two years. And again, that stuff doesn't come specifically from some notion that I'm trying to build this specific product. Does come from the idea like I think people should care about performance, but at the end of the day, it also just comes from the fundamental curiosity of like I like to know how the machine works. And again, ask AI how a machine works or ask AI to do the performance work for me. I'm not getting that same tangible benefit of the exploration and the thing that I enjoy.

Yeah. So hopefully that makes sense. I mean it's it's pretty simple actually. My opinion my opinion is not complicated, right? It doesn't

Well, I'm just saying I'm just thinking to myself, you know, when I got AI pled, right? Like yeah, who is still around that's denying like AI? I think you were the only person that I could think of like on the internet that's basically like, yeah, I'm not interested in AI, right? So, I was like, okay, I have to have this guy back, especially after I saw your first couple of podcast episodes.

Yeah. And uh, you know, I think on those podcast episodes, I said similar things, right? But that's more a chance for Demetri to talk. So, I I try to, you know, interject opinions or philosophical things where it's appropriate or technical commentary where there is some that's not specifically about AI. But mostly I expect Dmitri to you know to provide any of the actual technical commentary because I don't work on the stuff definitely in the market but I think also in yours at that time I I said exactly the same things. It's like let's say we take we take out my perspective of like how I personally see it. Uh because that's not really relevant to how good AI is. So you take the other one. I think what I said is okay there's another thing which is I'm concerned about people adopting it too soon. I think I said that on your program. actually do still have that concern and I think that's a concern that I can sort of move forward on right so like someday I may say I'm not concerned about that anymore like you know as a passive observer what I see what's happening it seems like the eyes are doing as well as the humans were doing and so you know maybe I would like them to be better because I would have liked the humans to have been better as well or something too but I'm not criticizing the AI as AI so I would say like that. I still am a little uneasy about when I look at sort of the sorts of things that are happening with AI from a kind of distant observer's perspective. I'm not necessarily getting the sense that it's actually doing all that great. Again, this is a very uneducated opinion because I don't use this stuff myself, but you're since you're asking, I've heard some pretty good programmers say basically like I babysit this thing very heavily if I want it to produce things that I think are good. It sounds kind of like that's the state right now. And so, you know, if you told me that you had a really good programmer babysitting this thing, but you had it write a lot of the code, I'm like, okay, I've heard a lot of people whose opinions I trust say that that's probably fine. If you just said like we had a bunch of people who had no idea what they were doing just typing prompts into this thing and something worked that came out and we're going to ship that. I'd be like, "Well, you know, again, I'm not really the person to say yes, you should or no, you shouldn't do that because it's not my not my monkeys, not my zoo, or not my circus, not my uh monkeys." I guess it's the correct way to say that phrase. But, you know, it also sounds like there's people who do use AI who are good programmers who say like maybe don't don't be doing that. you know, like let's wait till we see some better results from those kinds of, you know, oneshot highle prompting kinds of things before you go hog wild, right? I feel like that's a pretty sane take, but I'm the wrong one to say. I'm just looking for sane takes to be quite honest with you cuz okay, you know, when I got AI pill, then I'm like, okay, well, what's going on on X Twitter? I've never really used it as a social media platform. I'm like, I know that it exists. I, you know, all of the social media that I'm a part of, like links to exposts.

And so I I go in there, I'm like, what is going on in this? Yeah, this place is wild. Then I had to like stop myself from it. But people are just going out like they're way ahead of their skis on a lot of this stuff. Yeah. And you get rewarded for that because that that's the thing that that ens snares you that ins snares your attention. And I think what your take is is just like, okay, I I'm open to new things, but can we just be a little bit behind the curve instead of in front of it? When you're in front of it, that's where all the danger lies.

Yeah. I mean, I guess what I'd say is I tried on on your show and on Mark's show and also just online in my posts, I've tried to kind of just say that I'm staying out of this because like I said, if I'm not interested in something, I don't really want to be out there telling people to do or not do something because my opinion about whether you should use AI or not is really just completely uninformed because, you know, I would want to be I would want to be out there trying, you know, to do crazy stuff like, you know, the the Anthropic recently They were like, "How close can we get an agent swarm to build a compiler? Like what does that look like?" You know, that's what I would want to be doing every day in order to tell people like, "Here's where I think we are like on this trajectory. Like what realistically can you listen to do?" And I trust Demetri on that. And he says like, "Look, you have to basically give it prompts where there's no wiggle room." He's like, you know, that's what you actually want to be doing. So I believe people when they tell me that if it's a programmer I trust, whatever, but I just don't know. But I totally agree like you know the discourse if you look at people who are just unrestrained who are like I'm just gonna have an opinion on AI even though I'm not an AI researcher even though like I don't even like I couldn't even describe right like what a basic you know what back propagation is or something like like basic concepts of the technology but yet I can just tell you that in two years it will do X right yeah I think there's a lot of risks to that and the reason there are a lot of risks to that are because at the end of the day if they were coming more from my standpoint, but on the opposite side. I just want to use AI. I love AI. It's fantastic. That's all I want to do. And so they're like me saying, I just want to program. I love programming. That's all I want to do. They're not really harming anyone, right? They're just out there on the bleeding edge doing stuff and that's generating information for us, right? It's like, hey, we're just seeing what they do. They're like the forward deployed people who are just, you know, feeling it out for us, right? That's actually pretty harmless. If on the other hand, you're like, "Oh, you got to like fire all your employees and switch everything over to using these AI, but you don't really know that what that's going to produce for the end user and it could be really bad in six months if everyone does this thing and then we're stuck with all the software that no one could fix or you know, bad possible outcomes that can be very dangerous." And so, you know, do I know how to temper that or get people to try and approach it more reasonably? No. But I share your concern and I share your reaction cuz that's exactly how I feel about it too when I read it. I'm like like uh okay maybe it'll be good, maybe it won't. I don't know. If you just got your performance review and it wasn't what you expected, that feeling isn't random. Amazon, Google, Meta, they all have unwritten rules about who gets exceeds ratings and promotions and who gets managed out. But nobody tells you what they are, not even your manager. I'm Steve. I spent 18 years at Amazon and sat in on nearly a thousand interview debriefs. talent reviews and promotion panels. Me and my team of principal level coaches know exactly what they're looking for. If your review stung this year, don't waste another cycle to get the same result. Apply to work with me below. Let's talk about Dimmitri. How do you know I'm like what have you learned from him during this podcast that was maybe a bit surprising that you didn't understand? on the podcast versus not on the podcast is a little hazy because you know I don't always remember like which are the and we've recorded some stuff that we didn't air yet or like or won't air because we were just doing like feeling out what was going on things like that so it's hard for me necessarily remember what's on the podcast but if you have a friend who's really smart and they're into a discipline that you don't really know anything about it's fantastic because it's so much different than going on X and just hearing people say like the AI will crash immediately and it will completely be nothing in one year and then someone else is like it will replace all human labor in 6 months and and you're just like what like what how do I make sense of it right so talking to Demetri has been great because like one of the biggest things kind of broad perspective things that he pointed out that I thought was really useful for thinking about stuff was talking about the AI bubble he had a really great way of looking at that and what he said was like both things are true is kind of the way he looked at it he's like the way you have to think about something like an AI bubble is it's not that the AI won't work the AI bubble the AI probably will work. It probably will change a lot of the ways that we do things because it is effective. And I think like like I said, like I think we can all agree even if you have a very pessimistic take on AI, hopefully you can at least see the technology is doing some very interesting things that we couldn't do before, right? So he's saying it will be that, but there will also be a crash. And the reason he said that was because it's like look, somebody will come out as the person who's going to provide this thing. He hearkened it back to the webs. like there was a web crash that didn't mean we don't all use Google now somebody will be the Google right of AI and all the other search engines that were there the Yahoo Alta Vista all the ones that you used or what ask Jeves right there will be this big crash because all the money that went into those things they go to zero but that doesn't mean AI doesn't happen what it means is there's one player and they were left holding everything right do you think that AI is like the web Or you think AI is like search in this analogy?

AI is like search in this analogy just for the purpose of the analogy. It has it it doesn't have to it can be a much more profound effect on humanity. He's just talking about what happens to the companies. Right. Sure. But I think like yeah there is one winner in search. I think there was essentially one winner in e-commerce. Right. There was one winner in streaming like when we eventually you know got their incumbents and stuff like that.

So and you could do that for each discipline, right? like what the crash means you invested a ton of money in a lot of companies and a lot of them are going to zero but one of them is going to 100 in that category that's the way he was talking about it and so he's like so you have to be prepared for both things the AI bubble will burst it does not mean AI will go away what it means is there'll be a few really big players who are providing the AI technology for a large sector and all the other people will become also rans that's pretty smart. I like that framing because like it makes a lot of sense, right? Might not be true. Well, I I think the the thing that pops into my head, which is I think that's a that's a great take, but also part of me and I sound ridiculous saying this, but like maybe it's different this time. Could be the thing that's different this time is that like AI helps you make or break the competition, right? Or beat the competition. You can either beat you can copy the competition. Yes. Try to clone it faster. Yeah. Because you know if you if you look at say the uh the deepseeks of the world right so like you know there was a ban on chips and then so China goes and makes a very decent LLM for $6 million whether those numbers are true or not there I think there's some smoke and mirrors that are there but just goes to show you know there might be an efficiency like somebody like these companies seem to be leaprogging each other well certain companies uh have been a little quiet recently so maybe the leaprogging will will cease the thing that we're talking about now is like there are no modes or there very few modes when it comes to AI on the software side of things. It may not be a software company, right? And so the other thing to remember is like it could be that it has more to do with who builds the most efficient machines for this and that is more moike, right? I mean, we were already living in a world now where we effectively have one leading edge fab in the world, which is TSMC, and almost nobody else, like Intel's sort of getting back on track there. But like you think about how few companies are actually capable of producing leading edge process nodes for processors for example, that number has dramatically small even for all of the, you know, stuff that sits on top of it. Dramatically small. It could be another thing like that where it's like yeah again, I don't study this area so I have no predictions but it could be that like AI doesn't produce a winner at all in the software side and it kind of just like means that space doesn't have an outcome of that way that the bubble completely bursts there and they're all down to zero but the hardware that might not be right it could be that like everyone needs these chips now so the winner are the people who are supplying that which is two hardware companies who designed the best TPU used or whatever it is, right? Who knows? If it stays data center oriented, which it might, then it could be who are the people who built the really big data centers and built the most efficiently, you know, provide the lowest cost per token. Those sorts of things. I don't know. That's so far out of my like area of expertise, though. Your guess is as good as mine. See, your guess is better than mine. Way better than mine is how I would say it.

Let's talk through the C compiler. So, I went and read the the blog post from Enthropic. I read the headlines on X, don't get me wrong. So, so I definitely knew about That's all you need to know. Why'd you go read the blog post when you could have just read someone's summary of it on Because I was I I had watched your your episode and I think you you brought it up on Prime as well. And so I was like, okay, let's go read the blog post and then watch the marketing video. The blog post didn't actually have that much about the C compiler. It was basically this bash for loop, right? to try to get this like agentic loop going so that it could go and earn $20,000 worth of tokens and then you know different ways to to do it and then at the very end it's just like oh by the way you know we there was some smoke and mirrors you know there's an a linker an assembler I think the thing that you guys came on to you're like wait a second there's no type checking that's pretty important for a C compiler

Well, and yeah, like I think where you were going with that because you watched the video too as well right the video's kind of leaves a very different impression, right? Yes. And so, yeah, the reason that we were talking about that that C compiler thing was just because the blog post itself, which was sort of I don't know how much marketing was involved with it because I don't have any insight, you know, insight into what anthropic does, but it sounded more engineeringy. Like, it's pretty straightforward. It's not really trying to dress up what they did. It doesn't really try to sell it as more than it was. And if you read that whole blog post, I think you'd come away with a pretty reasonable expectation of what you could expect Claude to do if you tried to do something similar. You wouldn't be completely confused. If you watched the video, you might think you can type in write me an entire new C compiler from scratch, return, and have it just be like, you know, GCC, right? And that's it's not there yet. Now, again, like sometimes when you say stuff like this, people who are very strong AI proponents, they get mad. You say this like, well, it will be there. And so like I'm not saying it's not there. I'm just saying if you thought it was there right now, you'd be wrong because it wasn't. Right? And so what we were trying to get at in in that episode of the podcast was just the information is often there to take away the right things. like some programmer at Enthropic, some somebody on the R&D side probably said the reasonable stuff like the truth was there and then marketing got a hold of it or social media got a hold of it or Dimmitri was pointing out the CEO got briefed and didn't really get briefed fully, right? And said some things and that's can be very erroneous. But it wasn't like the original thing really wasn't that misleading. It was more just the coverage of the thing that becomes kind of misleading, right?

I was gonna ask you about that. So, I think your distaste for the marketing is clear. Sure. Yeah. Yeah. But like, isn't the thing that they did kind of cool? Yeah. Like like you prompted an AI and you put it in a in a tight for loop. It almost came out with a compiler.

Yeah. I mean, I guess there's two parts of it. And if I had to think about it engineering brainwise, I think I'm a lot less impressed with the part that people are impressed with and more impressed with the part that they're not is the way that I would say it. So the the reason I say that is because if I if you told me, okay, you're going to burn $20,000 worth of compute time. I've got all these tests. I have giant like source code repository filled with compilers. I want you to write something that just like stochastically tries to put those together to pass these tests. Oh, I'm like, I don't really know if that's that hard. I'm like, when I'm thinking about that, I'm like, yeah, you know, and because they've been working on this thing for years. I'm like, you know, if you gave me three years to write something that could do that, that's nothing to do with AI. It's just a thing that's about stochastically putting code snippets together, looking at what they do, back propagating like, you know, how you, you know, doing it with the ASC or whatever. I'm like, that doesn't actually sound like that hard of a problem to me, right? And maybe that's because that's the kind of thing I'm more familiar with. I don't know. The other part, which is I want to type in, I want a C compiler and I want this thing to know what I meant by that. Right? That part I'm just like, I have no like I'm like, forget it. Like I I don't know, right? So I'm way more impressed by the NLP part of things for whatever reason. the building the compiler part I'm actually like it's not like I think it sucked or anything, but it's like I'm not as impressed by that part of it for whatever reason because again, the way they did it was they had extremely detailed test suite. So this thing could literally stocastically try stuff and like just do mutations if it needs like you wouldn't really need it to know that much about what it was actually putting together potentially too. So at that point I'm just like, I'm less impressed by that. Right? That's not really me being pessimistic about the future AI, though. In some sense, it's me being optimistic about it because I'm sort of saying like, if anything, I think it's kind of underperforming on that back end. Probably I think there's improvements that could be made there that don't sound far-fetched to me. If it wasn't understanding what you meant by C compiler, I'm a little pessimistic about like, how do you get it to know what we meant by C compiler, right? Because again, I don't know that much about NLP. I don't see how that goes forward. But if you're just like, yeah, the code's kind of crappy, I'm like, I feel like we could fix that. You know, maybe we just need to like rearchitect some of those parts where it's got a little bit more knowledge of like how the work or something. Maybe we can code some of that, or maybe we use it to code itself and we guide it better or whatever we want to do. Seems like there's a path forward there is I guess all I'm saying to get better than what it was doing.

Yeah, I think you know maybe this is more on the progression thing in my head. So I also think it's a miracle that you can say I want a C compiler and it just like it kind of knows what that means. Maybe maybe I shouldn't be impressed by that. But that's the most imp because NLP was such a long-standing hard problem. That's what you know what I mean? Yeah. Then you get, you know, these transformers and these attention heads and all this matrix math and suddenly it works and you're like, "Wow, that was surprising." And it's like you can actually be like, "Well, it it actually does understand what that means in some sense." Yeah. I think the other one is also really impressive. Like if I had 20 grand to burn right now, I just, you know, 20 I pulled it out and I'm like, I I want to see as far as I can get with the C compiler and I threw it on the table. I'm not getting that far if I'm truly paying people for their time. If you let those people cut and paste from existing compilers, they'll have it done in an hour. Right? So I don't know about that. You have to keep the stakes fair. It's if so let's say it had never seen any C compilers before ever. It was not in the training data set. Okay, that is a lot more impressive, right? The fact that it's all in the training data set means that if you're comparing apples to apples, you have to give the humans all those things and 20 grand is you overpaid because you could just literally check out GCC and you've got it.

Okay, so you can check GCC out. But by the way, GCC's got 50 years of optimizations built on it. I'm not making heads or tails out of that that code base without like major study. Sure. So yeah, I mean you do have to like factor in the cost of like building the LLM and that big model or whatever, but like I don't think you get very far with 20 grand, but you literally have the whole compiler. You're done literally just by getting it from GitHub, right?

I mean maybe we have to go into the words of the prompt which is like make me a C compiler. Yeah, it didn't say copy and paste a C compiler, but it did like people have gone and looked at the source code and lots of the things are like this is exactly like this C compiler does this, right? So, it wasn't like it was doing it without actually going back to those sources. Now, you could say that, well, if we really wanted humans, they'd have to all be people who were kind of experts in compilers but didn't exactly have the source code on hand. But then, I mean, honestly, 20 grand for a compiler that kind of sort of works, not that hard. I mean, I don't know. I think it's just on what level of the philosophy you want to get on. It's just like, well, it used syntax that has been used before and like, you know, if it took snippets, I think that's fine. Like, I don't know. There's there's not an original thought in my head. Like, that's that's kind of the way that I think about it. Every artist has been inspired by other artists. They they all steal like artists, right? It's just like, "Oh, I saw a really cool painting 20 years ago and then I had an idea and then I made my version of it." Well, it's mine. It's inspired by the other painting that I saw. Well, this sort of gets into a little bit of a bifurcation. So, I think we have to remember what we're talking about. We didn't ask it to make something new. We asked it to make something that already exists. So, if I asked, What does it mean to make something that already exists, right? Like a C compiler. So in other words, we didn't say, I'd like you to invent your own programming language and then write a compiler for that programming language, right? Because you can then make the comparison you're making, which is like, oh well, when we do that, we are acting as the artist. It's not that I've never thought of the existence of a programming language before. Obviously, I've seen them. It's not that I have no idea what a compiler is. I've seen them, but I am trying to do something new. In this case, it was not trying to do anything new. It was literally just trying to do something that already existed and there were no bonus points for doing anything special. In other words, it it literally could have just produced the source code for GCC if it had the capability to do so. And that would be legitimate because nobody the prompt didn't tell it like try not to make it look like any other compiler you've ever seen before or anything like that. Right? So,

Well, is that just a direction? So, like what if the prompt was go and invent your own programming language and then from that programming language write a C compiler. Is that novel or is that just one layer of indirection? It's more interesting. So, this is why I say like if you asked it to do something that wasn't a direct copy of something else, I think it's a more instructive test. A very legitimate criticism you can have of work like this from a productivity standpoint is it would have been way more efficient to just use GCC, right? And unfortunately, I I am the target of

This kind of criticism all the time. I'm usually like, "Hey, let's try making this thing and let's do it our own." And they're like, "Why didn't you just get it from GitHub? It already exists." And I'm like, "Because I want to try and do that." Right?

So, this is why I you'll notice I didn't criticize the entropic people for trying this. I'm like, "Yeah, it's a very valid thing to try. We want to see if we can get this thing to reproduce it, right?" But at the same time, from a productivity standpoint, what we actually do want to know is can this thing produce something that we don't already have? Because if it can only produce things we already have, that's not particularly useful as a tool, right?

And so, that's why I say there's like a bifurcation there. If you're just asking about for $20,000, what can I get for a C compiler? I say for zero dollars you could download GCC and you're done. Right? So, it's that's why I said like it's the wrong way to ask that question. The right question to ask would be what if you needed something that was very different from GCC for some reason. How much money do you have to spend to get that? And the answer is we have no idea because this one wasn't really a working version of that.

Now, some people who I trust I think would say, and I'm a little bit putting words in their mouths, but I think they would say, we could have used Anthropic or Claude code. Well, maybe not Claude code, but let's say Claude in general, however they wished to use it. I don't want to specify how. We could have used that to build a very good new C compiler, but I would have told it all the things I wanted. I like, "You need to make this kind of token look up and it should use this algorithm." Like that's really I think where it's worth noting that it's not like AI is useless at building a C compiler just because this particular one was like this was a very specific test to see what it could do on its own. And the answer was it didn't do that well if you actually wanted to use something in production. But again, I just want to point out that people I trust would say, "Well, we could have gotten good results." And I believe them if we had guided it, right? If we had told it all the steps.

So, just also wanted to point Well, no. I Yeah, I think it's fair like you're like, "It didn't make a type checker." Yeah. It's just like, "Well, because it fed like the the input it was syntactically correct. You knew it was all you know that was in the test suite, right?"

>> Yeah. And so part of me is just like, "You just gave it one prompt and then had it loop in this tight loop. Maybe you looked at the output and you're like, 'How about you throw a type checker in there?'" Like exactly. And but this is why I say it's worth that's why I wanted to bring up that part is because what they were trying to do and and I think this is important too. So I I think they were trying to do something they should be trying to do. It's why I don't want to come across as criticizing the experiment while we're talking about its shortcomings. They were doing the right thing. And the reason for that is because one of the things that they're selling this as, like one of their pitches, right? And one of the things AI proponents' pitches are is to try and enable people don't know what they're doing, right? People don't really know that much about what's going on to do it.

So when you look at something like a C compiler, it's like, what we want ideally is for the AI in all of its training, all the things that it's read to understand that when we ask for a C compiler, that includes the type checker, that includes an optimizer, that includes register allocation, right? Things things that didn't include. Oh yeah, let's just use the one register.

>> Right? Like just like and you know that's something that we want. So that's why this is a good experiment and you know the hope is right that as they make improvements to the way that the AI works and you know only they know how hard that's going to be for them to get to like maybe it's incredibly difficult. It's going to take them five years or maybe they're like, "No, no, no, we got this, like six months from now we're going to have a good one." I don't know how hard of a road this is for them, but hopefully the idea is over time they will be able to get it to the point where when you say something like C compiler, the meaning to the AI in terms of what it's supposed to produce, it doesn't require you spending weeks producing a test set. It doesn't require like it's like it just knows like, "Okay, these are the things that are part of a C compiler. I need to generate my own tests. I need to, you know, live up to all those tests. I need to go read through the code to make sure I didn't skip anything." Like, and I'm sure that's what what they want to do. And I'm also sure that's why they did this. You can't get there by just snapping your fingers. They have to know these steps.

So, well, and to be honest, like again, maybe this is kind of stupid of me, but honestly, like I just get more impressed by the AI stuff that's even like separated from coding. Just looking at like, "Okay, this thing can carry on a conversation so much better than like anything that we've ever been able to do before." And it does seem to be able to do that in a structural way. Meaning, it's not like it does actually understand that it can use sort of a meta structure of the conversation. It doesn't have to be about logical things that it's seen before. Like it can it can take a complete fantasy thing that I create or something that's weird and doesn't obey any of the sort. And as long as I give it enough information, this is what I remember playing with like GPT-4, even could sort of do this. I think that's still the coolest thing about it to me. And sort of the coding things are just, I don't know, because coding is more mechanical and has a lot more ways that you could imagine making tools to do these kinds of things, especially when you're talking about spending this much compute time on them and this much like search space stuff. I'm like, "I don't even know if what the AI is doing is really all that most efficient way that we could be doing this thing if this is what you wanted, right?" And if you were going to spend trillions of dollars to accomplish it, right? I'm like, "We might have been able to do something else that's like just the GitHub munger that like does this sort of thing in a different way, right?" So that part has always been a little less impressive to me, which is again, I'm trying not I don't want to say that it people are wrong who are saying this is cool. I'm just saying my re my gut reaction is that's always less impressive to me than some of these things that other people just maybe just don't care about as much.

I think people forget that the Turing Test. Yes. Was actually a really big barrier that we just flew through. Nobody talks about it anymore.

>> Yeah. So the Turing Test from what I understand is basically like, you you want to come up with these double-blind experiments where people are trying to figure out whether you're actually having a conversation with a computer or a human. I'm an '80s kid too, right? And so, you know, in in the '90s, it was just like, "What an impossible thing." And I think everybody tried to like pack up some Perl and with some if-then statements to try to like see if it can't repeat basically what was said back to them. And but like nobody could really crack that nut and it seemed like almost an impossible thing. And then we just blew way past it.

>> Yes. And to me, that's the way more interesting thing. Like I understand why we blew past it, right? I I'm not surprised that even enthusiasts of AI and things like that don't care about this all that much or appear not to care that much about this. And that's because there's not a lot of use for it, right? It's like there's not a lot of business case for thing that you can have an unusual conversation with and it doesn't screw up. Because when we're thinking Turing Test, like you and I, we're thinking like, "Oh, I'm going to give it some tricky things, right?" We're not just talking about something that kind of spews out things that sound like conversation because, you know, maybe that's not that hard. Still kind of hard, but you know, but like we could say like, "Okay, you know, imagine there's a purple dinosaur, right? And it looks in the mirror, but the mirror changes purple to black." Like, you could give it all these things and it'd be like, it it has a fair shot of getting them right. Right. And that to me is like very, very impressive because for a very long time, there was no line of sight to that. Like there really wasn't. And I think no matter what AI does in the future, it will probably be hard to impress me as much as that. To be completely honest, that was really like the most impressive thing I've seen for AI. Maybe it will do something else, but that one I was like, "That's really damn cool." And like like you, it's surprising to me that we didn't celebrate that aspect more, but we were kind of just like, "Yeah, I could do that. All right, how can we replace jobs with it?" Right? It's always a little bit pessimistic, right? It's like, "All right, well, I get why you had to do that because this costs a lot of money, but at the end of the day, I was like, the purple dinosaur was what I was cool."

Yeah. You heard it here first. Casey Moratori blown away by the chatbot.

>> Yeah. No, really. Like I know that's silly, but like that was the coolest thing that I've seen from AI and it remains, I think, very impressive to me. It remains impressive to this day. I assume it's only gotten better, but it was good enough at that in my opinion back in GPT-4 or whatever it was. It was good enough that I was very impressed.

Let's talk about the Amazon incidents. This is near and dear to my heart. Since >> you are an Amazon alum, are you not?

>> Yeah. >> I was there for uh 18 years. So, if I remember correctly, what happened was Amazon built their own AI called Cairo or something. It decided that it was just going to delete a whole bunch of stuff or change the configuration, probably change the control plane on something. Maybe it deleted some environments, but it whatever ended up happening, it led to these really large outages. And then there were these big meetings where all of the execs and leadership got together and it's just like, "Okay, all code that's generated by AI from, you know, needs to be reviewed by a senior before it goes into production." Does that surprise you at all?

>> No, it's what I would expect, right? I mean, the reason for that is just because, not to be too much of a grump about it, but I've seen kind of how web stuff went. And in general, it was not an it was never an industry that asked, "What is the like robust performance solution and we will only deploy when we have that?" That was never their culture. And I don't think they would have said that it was either. Like I don't think I'm criticizing them in a way that they were like, "No, of course we did that." I think like, yeah, like "move fast and break things" was the thing that people said as a catchphrase, right? In a lot of circles, and they I think they mean it, right? I mean, I don't think they were just saying that, I think they meant it. And so to me, one of my fears about AI is very mundane. It's literally just that people will continue to deploy it in advance of what it should really be deployed to do wherever the thing is that's like, "We could reliably use AI in this way that bar may be here and they will be doing it here, right?" They just because that's the nature of how they do things. So, it doesn't surprise me at all. That's exactly what I would expect to have happened. And my assumption is it not knowing anything about it because I don't have any inside information about what actually occurred. My assumption would be it happened exactly because of that. It's like they decided to there was a way they could have used AI to improve productivity by 5% and they wanted to improve productivity by 25%. And that's what happened. Right? It w it wasn't that there was no gain they could have got from AI and not have outages. It was that they wanted more than that. Right? That's my guess. And so it's both a positive and a negative take on AI at the same time. The negative take is, "Yep, AI screwed this up." But the positive part is, "I don't think it had to."

You know, my take on this is I don't think I believe that the AI deleted environments or like whole services.

>> Okay. You think that may have been like an misreporting kind of >> I I'm I'm gonna go out on a limb that so I do know that if like every team needs to do code reviews whether AI is involved or not. So you may you can get it reviewed by a junior. You can get it reviewed by a mid, seniors, principles, whatever.

>> But some human is looking over this thing. You did not >> I believe so. Maybe they were maybe they had some agent in the cloud that they said was safe and then let it go off and and then just in that type loop that we're talking about to go and do that thing. If that was the case, so if it's the first case where it was just like regular development and it would have been reviewed, I don't believe that at all because nobody looking at the code would be like, "Okay, cool. We're just going to delete this environment." Like that's that's not okay. If it was an agent in the cloud that had free reign to go and check in automatically, maybe I would believe that. But then like who thought that was a good idea?

>> Yeah. >> So, someone had to approve that check like the checkin that was like, "Hey, we're going to like change this this uh basic like configuration of this thing to allow this." Right. Uh >> so that that I think there's some accountability because you gave you gave it agency to go and you gave it your social security number and bank account information and said like, "Hey, be safe with this thing." It's like, "No, that's on you because you gave him the keys to the kingdom." Okay. Do I believe that like an AI changed some configuration that blew something up and led to an outage? That I believe.

>> Sure. >> The reason I believe that is because, you know, if we're talking about a control plane configuration, right? So, this is like how like what is what is your setup, right, for your system that's there, that's a high-dimensional space.

>> Just think about ffmpeg for instance. FFmpeg, you know, you it's a command line tool. You can go make do a bunch of video stuff. There are about a trillion different parameters that you can give to FFmpeg to like to change the the encoding that's going on to change like different quality settings. You can write PhD thesis on and and research papers on like the the optimal way to optimize like video so that you can see Game of Thrones that one episode where everything was just completely dark, right? And then taking into account like, you know, the different ways that people watch it, basically it was they were all just turning into ffmpeg parameters.

>> Right? That is a very high-dimensional space. >> Yes. >> And if you think about what AI does, it's basically a lot of its reasoning is just a search. It's a search over some space. And so if you said like, "Hey, here's a here's a control here's a configuration for a system like go find me some optimal parameters for this configuration," I think it could screw that up. There's it's there's no way it can like really pull in all of that context and like come up with a good configuration. So I think it's a terrible freaking use case for AI.

>> Right? Is like coming up with configuration values. >> Again, like I said, since I don't really know much about that. I don't really know much about what was actually going on internally, it's hard to guess. But what I could say is at Amazon, I did try to read as much as I could um because it came up on another show that I was was doing and I was kind of curious about it. They had an outage that was due to the fact that there was like a DNS like they this was a very very big outage I guess many many hours there was a DNS entry that ended up getting set wrong and as a result nobody could hit the AI API endpoint because it just didn't resolve anything.

>> That might have been when I was there. I think is that was that or is it a recent one? Oh, no, no. Uh, it would it would have been Oct. >> DNS is hard to get right, by the way, folks. >> It was October 2025, maybe. It was very, very recent. It was a big big outage. And so, I read up on that one. They had a a write up. Wasn't very detailed. They then did a they had a presentation on it that was more detailed, but still kind of omitted a key fact. But then someone actually wrote into me and was like, "I think I figured it out because they did a test on thing," and I was like, "Ah, okay." And what I can say from that was that entire bug basically came down to the fact that uh their DNS thing, Route 53, what is it called? Route 53. >> That thing when you are using aliases, so when you have effectively entries that point at another entry, if the entry you you try to update it, you say, "I want to change this alias to point somewhere else." If that somewhere else doesn't exist, it just hard fails. It like error. It doesn't update the entry to point to, you know, a non-existent IP or or a non-existent name, I guess, would be the case. The IPs all exist in some sense, but the the name, right? If it doesn't know what that name is, it just fails. So, in other words, like even a simple thing like, "I'm going to create an entry for this guy and I'm going to put that in the table," and then I'm going to create an alias that points to him. That will work. Do it in the opposite order. It won't work because when you try to get the alias, that name didn't exist. It would fail that. And if you weren't smart enough to retry it afterwards, right? I don't run DNS servers, so I'm not going to say I know one way or the other, but that to me was like that's very brittle and scares me from a robustness standpoint. Like normally I don't really like things that do that because it means that small mistakes can lead to very big problems. And you thought you were producing something that would fail loudly. Sometimes that's not such a good idea for fault tolerance. And that was exactly what ended up happening in this particular case. But the reason I bring that up is because so that's an example of something that actually exists inside Amazon. Now imagine this poor AI.

>> Yeah. >> That you'd say, "Hey, it would be really great if you could sort of take over the updating of our load balancing alias table right here." Like, "Just, you know, take a look at the machines. You've got access to the dashboard, read what the machine loads are, rebalance them as necessary. If you see something suspicious, paying an operator, I don't know, like, you know, you give it something reasonable." If that AI just it didn't realize that Route 53 had this behavior because, you know, I'm sure it read that somewhere in its master training corpus, but, you know, an AI isn't a perfect memory machine. It's a sto it you know, it boils things down to statistics at the end of the day. If it just didn't quite intuit it what that meant, so when it was producing things, it didn't realize it had to do it in that order and missed that that's an easy way that an AI could instantly take down the entire system, just in the same way that the humans did.

>> Right? And do we blame AI for that? >> Absolutely. Absolutely. >> I wouldn't. >> No, I I do think that the, you know, I just think back to my time. So when I was a principal engineer at Prime Video, we were launching a, you know, live streaming product. You know, we were streaming sports. We have this big Q4 code freeze.

>> And also we were like launching these really big uh sports properties on the system. So we were basically like, "No code is allowed into the system at all. Anytime there's a change, it's it's a risk. We're not okay with these risks. If if the Super Bowl goes down or something, everyone will kill us. This can't happen."

>> Yeah, we spent many billions of dollars on the rights for these things, but also the trust and and all of that stuff. But I was on the live uh the live events team. >> And so we would lock it down and then of course we we were in a service-oriented architecture, like dozens of teams raise their hand and they're basically like,

>> Guys, we need to put this code thing through.

>> Yep. >> Right. And so, but my boss was and you know in the in the leadership team was basically like, "We said no changes. The time for those changes had passed." And they're like, "No, we need the changes to go through. Like here's the reason why." So my boss was basically like, "Okay, Steve, you're the principal engineer here. Your job is to approve or deny all of the code changes that are coming in." And I'm just like, "I didn't sign up for this job."

>> I'm like, "I don't want that responsibility." So I was basically like, I was basically like I was like, "Dude, I was there. There's like 50 changes that that people are trying to go through."

>> And so basically we came up with this thing that was basically like, "Okay, you have to go and get your VP to say that this change needs to go through." Then when it kicked back, there were only like 15 changes they were trying to go through. And then for each one of those, I was able to like wade through it. I kind of quit my job before AI became a thing within the company. And so from what I understand now, like especially with the these outages, if you're a senior engineer or above, like your whole day is now like reviewing all of this AI code that's come out. That's not the job I signed up for.

>> Yeah. Well, I mean, I hate to say it, but now you sound like me. I mean, that was kind of that's why I don't really like this stuff personally, right? Like I'm just like, "I don't really want to do that." And in a sense, I feel very lucky to have lived through the period. I think I literally said this on your last show. I feel very lucky to live through the period where what I want to do, like what I enjoy doing also gets paid well. Like they will pay you well for this. That may be going away, right? The new job even for people who are very good programmers might be, "We need you to babysit these AIs. Like we need you to review what they're doing and try to get them into a place where they're producing code that's up to the standards that we expect." Some people might be fine with that, right? Some people that might be fine because they didn't like the act of actually doing the code. They liked the act of having the thing done, right? And they're just like, "Okay, as long as you present it to me. I like reading it. I like seeing if it's correct." And that's fine. And those people, I think, will probably be I mean, if I had to guess, those will be like your most valuable people. People who really knew programming very well, but never really cared that much about the actual crafting of it. They had to get good at that because they wanted the results, but they weren't tied to that activity, right? If there's people like that out there, those are probably going to be your most valuable people because they won't burn out. They're okay. Like, they like doing this and they have the skills necessary to fix the problems that the AI is causing, right? People like me, they're probably out, right? Because they're like, "I like to code and I'm not coding anymore, so I'm going to go be a plumber or whatever it is, right?" And then I guess I don't know what you do to produce more. Like the question is like, if the AI doesn't get good enough in the short term. This is something that Demetri and I had actually talked about one like, "What do you do for training the next generation?" People are going to review the code. They're not good enough yet to babysit an AI. They don't know. They don't know half the things the AI knows yet, right? That's another that's like this pipeline problem which probably we don't want to get into yet. But I mean that's another huge problem with it, right? And I don't know.

>> We can talk about the pipeline. I do think that there's a step after the pipeline problem that I wanted to pose to you. Okay.

>> So during your podcast, you're basic uh Demetri had said, "Okay, an AI can basically do an entry like a junior engineer's job."

>> Yes. >> One to 2,000 lines of code, maybe 5,000 lines of code. Well, however you want to describe the complexity there. You give it very structured inputs and give it very structured success criteria, and it can go and knock it out. And so therefore, we're like, "Okay, well, that's the death of the junior engineer."

>> Turns out senior engineers come from junior engineers. We now have a demographics issue, right? This was Demetri's thing. And and that again, unlike me who is self-described, not knowing anything about AI. Demetri is I would consider him an expert on this stuff. And that comes from his own test. He runs test suites himself against these AIs to determine where he thinks they are on exactly that. And this is why I say people I trust him being one of them. He says it's like close. It's close to a junior engineer now, right? And that's the problem in exactly what you described.

>> Yeah. DHH said similar things. He was basically like, "I thought that it was another tool," but he's like, "I have now promoted them to junior engineer or something like that," in his, you know, typical Viking like way of saying things. And I agree with him. So now I think companies like Cloudflare, they decided that they were going to hire, 11 interns for their next intern class.

>> 1111. >> Yeah, that turns out that's their DNS, right? Uh IP. Nice PR move. And it turns out >> sure >> likely they're funding. I think that represents like a 10 time at least a 10 times increase in the number of interns that they hire. They're taking that uh the funding for regular headcount and then moving it over into the interns. And the reason from what I understand is that this new incoming class, they are AI native.

>> Mhm. >> So if if you believe what Demetri said that the AI is essentially entry level at this point or intern level, entry level, something like that, if this new incoming class basically knows how to harness that, they are actually useful. And so you can actually bring in like a large incoming class. I heard at at Gerge's AI conference that maybe mid-level engineers are the really the ones that are going to hear feel the pinch. Because if you don't lean into the AI tools now, you get marked from what I hear in all big tech at all companies. Management has dashboards about how many tokens you've burned.

>> I have heard this, right? I mean obviously I don't know, but that that has been said. >> And yes, you can just go and game it. You can be like, "Hey, AI, I need to burn this many tokens, otherwise my boss is gonna be on my deck. Can you like create a bunch of plausible pull requests so that like we're good here?" Yeah. >> Yeah. Sure. So there's this arms race there about like this meta work.

>> But it's not clear why you would do that instead of just telling it to do something useful. Yeah. Okay.

>> But I do think that there the these mid-level engineer. So basically what is I think the demographic issue now or the the issue now is like mid-level engineers that don't go all-in on AI are going to be left behind. If you weren't building or at least looking like you're building at this sort of breakneck speed, you're definitely going to get left behind. And I actually don't see a big problem with mid-levels getting to senior. I don't think there's a there's a big step change there. And then you know, you see you have this new incoming class. I don't I don't know if it's pervasive within the industry of this new incoming class of like AI pill that are coming in whose job is as the AI gets better to leverage them to to get the work done that's there. Maybe the maybe the mid-level engineer is the one that's getting squeezed now.

>> I mean, that's the kind of thing that I just have no idea about because I just don't understand big company dynamics, right? I mean, your take on that is probably going to be pretty accurate because you've seen it. You know, yes, you left before AI, but you can probably imagine like knowing the people you knew, you can probably imagine in your head like what were these people going to do when AI? You could probably have a pretty good guess is my feeling, right?

>> Well, here I'll just talk out loud and you tell me where I'm >> I was going to say cuz I don't know, you know.

>> So Jensen says, so Jensen from Nvidia and obviously he has a he is biased here. He sells he sells a particular product,

>> but he says, "If I have a half a million dollar engineer, a $500,000 engineer, and they're not spending $250,000 in tokens, I'm going to go apeshit." That's what that's a quote from him.

>> Okay. >> And I think that's kind of what CTO's thinking right now. They're basically like, "Everybody tells me that I need to be doing all of this AI stuff, but it turns out that AI is really expensive." If you think about like the total cost of a developer, you have to pay their salaries and benefits and office space. I think that's fair.

>> One of the nice things about working at a big tech company like a hyperscaler like Amazon is like you generally have an unlimited AWS bill that you can run up,

>> right? >> Which is pretty sweet,

>> right? You know, there were I think there were a couple people they got fired for like mining >> Bitcoin and Ethereum on AWS instances. And so they they, you know, they not unlimited, but likely unlimited effectively being doing something productive for the company.

>> Yeah. >> But you have to factor in the the the AWS and and those are marginally cheaper than like on-prem racks that you would manage on your own. So like now it's a SAS thing. It's just another subscription that you have to buy your developers.

>> Yeah. And then you have Slack and then the Office suite and Outlook and all of the enterprise software that's there. But now you have token costs.

>> Yeah. And it turns out that well depending on who you ask, like the the token costs for these frontier models may or may not be sub uh subsidized by these big AI companies,

>> right? And we also kind of don't know what their long-term cost will be because it's a very, you know, presumably as AI data centers get more optimized, as the chips get better, at so on so on, that that cost goes down. But then also, we don't know, will we be asking the to do harder things that then we don't know what that curve is? Hard to predict, right? But presumably it will go down. I would think even if it's subsidized now because people have not been trying to optimize the token cost very much.

>> Well, I mean, we can just go into >> Google, I guess, you know, maybe that could be wrong. Never. Well, I think we'll talk that other thing first. We'll talk about token cost later because that is interesting.

>> But that's just an extra line item that just popped out of nowhere.

>> Yeah. >> And it turns out it's pretty expensive. So we're talking about like say 10% of the cost of a developer on token cost. It just popped out of nowhere to 50% if we were to believe

>> Jensen. Jensen was 50%. This is very high.

>> So let's just say it's between let's say it's between five and 50%.

>> Okay? >> At some point you're going to be like, "Where's my freaking productivity?" And it has to be it has to be 50% more if I'm spending 50% like I should get 50% more out of my engineers than I was before, right? Which

>> yeah, there needs to be some sort of benefit. So I'm wondering just like the market. So the market can stay irrational longer than you can stay solvent. You know, you you can be betting that the market goes down, but you may be right. You just got the timing screwed up.

>> Yes. But I do think at some point there needs to be a productivity increase or some larger benefit increase within the corporate layer because you you can't just write this check and then just get the the benefit from it because it seems to be ethereal. It's we've been always trying to measure this developer productivity. Turns out lines of code is not the way that you should be measuring whether a developer is good or not. I also don't think that like how many tokens you burn necessarily correlates to how effective a developer is.

Well, also, I mean, I would point out the sort of thing I could say about, you know, because this is true across pretty much everything and that is that whatever you set as your metric, that is the thing that will be optimized. So for the time being, it may make rational sense to tell your employees that they're trying to burn a lot of tokens because you want them to experiment with the AI and get better at it or whatever else. Eventually, you're probably going to tell them the opposite. You're probably going to, you know, five years from now when AI has sort of been worked into the system in however which way they want it to be, then they're going to be like, "Let's try to get the AI cost down, right?" Then all of a sudden, you're going to be able to do in less tokens, right? And so, I think we can probably all see that coming. In general, I would say that

>> from the outside looking in, I still want to see the productivity that I'm supposedly getting. Right? As someone who doesn't use AI, I'm looking at other people's software. When will I start seeing the software appear to be better than it was because, you know, I'm used to interacting with Amazon's website and it really doesn't get any better. Like, it hasn't improved in quite some time. In fact, I would say my experience on it on average is worse today than it was 5 years ago or something like that, right? I'm totally willing to believe in a year from now it will be way faster or the the descriptions will actually match the products or, you know, whatever whatever the thing is, right? That that you could imagine like, "Hey, the site got way better or whatever it is." I haven't seen that yet. Now, a lot of people say there's the hypeers who said it was useful for the past three years, like they're just been like, "It's amazing, whatever." And most serious people think that was not true. The serious people have mostly said from what I've heard heard that it only really started getting useful around the time that they were able to get agent workflows set up and actually work. Like it took a while to get the tooling in place. And they say that was around November, December of 2025. So if that's true, then I'm totally willing to wait a year, right? Like it's if you just got the tooling working now, then obviously you're not going to have anything great in March. It probably means you're people who are saying things like 10x productivity. They're obviously wrong because you should have had a year's worth of stuff done basically by now and you didn't. Right.

>> Yeah. >> But it doesn't have to be 10x to be something. Like I said, I'm not an AI naysayer. I'm someone who just doesn't interested in it. There's a difference between those two things. Maybe it's a 20% productivity boost. That would be amazing, right? Maybe it's a 10%. Maybe maybe it means in a year you get an extra month, right? That still be really cool. You would have to adjust the token prices to make sure they're in line with that. But if each engineer is spending 10% on tokens and they're getting 10% more productivity, that's a net neutral. And everyone would have to do it because you wouldn't want to fall behind by 10% every year, right? In a general large or that's a win for the AI industry, marginal win for society, probably. Maybe lost jobs isn't great, but if everyone just moves 10% faster, then we didn't really lose job. There's a there's a theoretical win there somewhere, right?

>> I'm not saying that's true. I'm just saying it's a it's a plausible thing, right? Yeah.

>> I'm not making something wildly, you know, ridiculous.

>> Well, I I think what you and Demetri said that was really great was this um like the profit going from negative to positive like those dynamics.

>> So >> yes, Demetri was talking about sign change.

>> Yeah, I thought it was great. So, like you have a dollar. If you invest a dollar and you get 99 cents back, you're doing that trade exactly zero times.

>> Yes. If you have a dollar and you get $1.001 back, you are doing that trade infinite times. And so we're at this point where the AI has gotten to like say intern or mid-level and it can do a whole bunch of this like low-value stuff that would just longtail stuff that nobody would ever do because the software is so expensive.

>> I see. Yeah. >> Right. And so then you're like, "Okay, now we're just going to do a bunch of this low-value stuff because it's profitable." And then we're going to do a whole bunch more of it because like, you know, we can just spam that button and we get that. So maybe all of the low-value stuff is there. And so I think maybe that's why the productivity is diffuse is because it's all this low low order stuff. It could be, but like I said, also I think people who are saying that they do 10x whatever, I think those people are probably wrong. So it could just be a lower percentage and we just haven't seen it. It would be hard to see, right? If if everyone got 10% or also it might take a while for people to learn to get the 10%. It might be 6 months before you get the 10%, right? In which case, we won't start getting the 10% till summer, right? So, it could it could be that it really is going to be more productive just in general. But the thing you're talking about is I think Demetri used the term Jevons paradox. Jee Vo N. I had never heard before I don't think until he brought it up. And that's basically that like when you get a technological improvement sometimes it affects you know it comes from the bottom up. That's the most likely thing because the technology as it improves, well, it's going to improve good enough to do the lowhanging fruit before it improves good enough to do the hard stuff, right?

Well, I think the paradox is that when something becomes cheaper, people end up using it a lot more.

>> But it's the low value part. So, it's the fact that it ate the low value stuff first because of that, it means you get a lot more low value stuff.

>> That's what I'm saying, right?

>> So, as a result, it's like, "Well, that could be bad. It could mean that there's just like now there's gonna be tons of crap on Amazon.com that like nobody really wanted all that much and it clouds it, right?" So, yeah, possible, right? But I don't know. It could also be that it won't be that bad. Like it could be that it's just like, "No, it could be a general productivity boost." And maybe that productivity boost isn't that high yet. We may never really see it. Like as someone like me who's sitting outside of it, I may never really see it. If Amazon improved their productivity by 10% every year, like not compounding 10%, but like for the next five years, they do 10% more work each year than they were before. I'm not I'm not going to know. I'm not going to be able to see that. But it was real, right? If you actually were looking at like how much they got done over those five years. So you be like, "That's a significant chunk, right?" It's also possible that like if we get to December and we just the software doesn't really seem that much better, even I wouldn't then declare it a failure. I would say, "Well, that just means it wasn't 10x. It means it wasn't even probably 2x. It means it was 20%." Right. 2, right?

I'm of two minds of this. One is if there was a like a 10% productivity boost, I would be like, "All of this hoopla for 10%?" I agree, but only because of the hypeers. Only because people are out there hyping it so hard because it's their business and they need to make all they they need to make all back all this investment, right? Pretend that those people hadn't been annoying, right? This this is the Rust community's problem, right? Pretend that there weren't annoying people shilling this product, right? Would you have a different reaction to it? Right? And a lot of people would, myself included, right? And I think AI suffering from that problem right now. And it's of their own making. I'm not excusing it at all. I'm as angry about AI companies as anyone else for other reasons, right? But I I really don't like their behavior. I said that on your show before. I'll say it again and I'll keep saying it for sure. But if you took all that out, if you took out the unlikability, if you took out the fact that maybe we're spending too much on it, people are sick of hearing about it. If you took all that out and you just had somebody you said like, "Steve, I'm going to make your org 10% more efficient," you'd be ecstatic. There's very few things that you can buy that would do that. So, I would say that again, if I'm objective about it, I'm not going to be upset about a 10% productivity boost because there's very few other things that can do that. And the only reason that I'm going to be pissed off about it is because like you just said, "All this for 10%." It's like, "Well, yeah, at this point we would feel that way." Right. So that's on one hand, I was told 10x. Somebody somebody planted 10x in my head.

>> 10x. Yeah. >> You know, and but then we're we're talking about a marginal gain.

>> Yeah. >> The other mind of me is like, 10% is a lot.

>> Yeah. And if it's compounding, if you told me, "Hey, you have a you have an investment that compounds 10% every year and it's it's guaranteed," I'll be like, "I'm going to be rich soon."

>> Yeah. >> Well, I mean, eventually I'll be rich.

>> Yeah. Once it compounds.

>> Yeah. >> And so, I think

I'm I'm in the same boat as you. Like, like the proof is in the pudding. Like, show, show it to me. Where is it? Like, you should be seeing it.

Now, I have heard that, you know, mid-size tech companies are saying, like, 10% of all of our code or 16% of all of our code is AI generated. Well, one, I think that's impressive on its face, but then on the other side of it, I'm like, so what?

Exactly. Like, that's not a measure of anything. We, like, we don't care how much was written by any. What we care about is how productive were is Uber Ubering more? Is Amazon Amazoning more? Is Google Googling more? Like, that's kind of like, why is it so diffuse?

I totally agree with you. And like, I said, for me, it's just a little bit too soon to ask for those results. It's not too soon to ask like the AI hypsters for those results because they've been claiming they've been right. So, I to anyone who wants to go after some shill who said they were getting 10x back in January of 2025, go right ahead. Right? I have no love for those people. And if anything, I think they're doing us a disservice, right?

Totally true. But for the people who are actually serious about AI, like, I'm thinking about the researchers who have not been out there making invalid claims, you know, not talking about the CEOs who can be just as annoying as the shills because they're doing their own shill bit, right? If we're just talking about somebody who was an AI R&D guy and they re, they cared about this technology, they weren't out there shilling it, they're trying to produce this thing, it's possible that they will get you your 10%. It's possible, right? And it's just a question of, they, they do need some time. None of those people that I'm aware of have ever said that they were getting these productivity gains until very recently, right? They were saying Agentic Workflows sort of the end of 2025 was the first time they were really seeing Demetri used the term dancing bear. First time they were really seeing anything that wasn't just a dancing bear. And for those who don't know that analogy, it's like, if a bear dances, you're just really happy to see that it danced. That's impressive. We don't ask how good the dance was. And that's kind of where AI was. It was like, it's just really cool that it can do any of this stuff, yet it's not doing a very good job. So, the first time that I think the more honest AI researchers were ever saying that this was actually producing something was until very recently. And so, I think it only, I think for those people and those people only, I think they deserve a year at least of people trying to figure out if they can make actual gains out of this before we look at the software and go, "Still sucks." Right.

Right. But I'm totally willing to do that. In a year from now, we have this sit down again.

Yeah.

I'm totally willing to be back here and be like, "Yep, I don't see any difference. It didn't work, guys." Right. Uh, but, but I do want to give them that chance 'cause I think that's only fair. And especially, like I said, as someone who doesn't use it, I'm not prepared to say it's not working. Because maybe I would go and use it and be like, "No, it's really great. I figured out how to make it work well." Right. And, and, you know, so I don't want to say things negatively uh until I've given them chance to to show.

Yeah. It's so, it's end of March 2026 right now. So, we'll see where it goes here. March 2027.

Show, show us the 10%. We're calling it 10% maybe, right? I mean, even 5% because it's hard to show 10% is the problem. It's hard, hard to demonstrate.

Developer productivity is

hard to measure.

You can't measure it or it's very difficult to measure. And then, so we're saying like, some increase on this thing that's difficult to measure, but I, I think we should see some side effect from it. We should see something.

Yeah. I mean, like I said, my fear for them is they do deliver the 10% but no one can tell. If I was an honest researcher who was just trying to do a, a cool piece of technology here, it delivered that and then everyone again because of the shills, right? Is goes, "This sucks. It's not, it didn't allow me to just make all my software by typing in a prompt and it all worked perfectly like whatever the shills have been saying, right?" If it does produce anything close to like what's being claimed, like the 10X's, we should be absolutely clear. It should be like, we have revolutionary software in, in March 2027. It should be completely different from anything you and I have ever seen before, right? That should be happening.

Well, I, I mean, I think even if you are going at the efficiencies from the bottom up, I would expect something like, we have no bugs.

Get rid of the bugs. Yeah.

There, like almost every company that's out there has a really, really large set of bugs and issues that they've triaged and basically said, "Okay, well, we're going to fix these ones. We're going to wait and hold on these ones and then these ones we're not going to fix." Right? I would expect if AI agents are actually good, at least on the, the small bug side, like it just shoots through the roof. That's what I would expect. And it does seem to me like, I mean, I guess the only problem is it depends on how large scale the code changes are to fix some of these bugs, right? So bugs that are more localized, I feel like that's a pretty reasonable ask. It feels like the AIS of today should be able to make a measurable improvement in just finding and fixing localized bugs. And if we can't figure out how to do that by March 2027, that would be embarrassing, I think.

Yeah, because that seems very achievable with what, what they have right now. Whether or not some of these bugs are bigger in scope, I mean, then the AI, you know, you get into things about when the AI has to make larger scale changes. I think one of the things that I hear people complaining about now is that it doesn't necessarily create manageable code bases, right? So, if it starts to do larger scale changes, yeah, maybe it fixes the bug, but it creates 10 new bugs. That's a problem, right? And so that's the part that I don't know about for like the larger scale things is it could be that there's just this, yeah, it can do these things but it creates these other problems and then we have to wait for the AIS to be improved to the point where they don't create those problems, yada, yada, yada, right? So, but yes, I agree with you, at least small scale bugs, that seems very achievable by next year.

I'm just going to give you rapid fire bullish or bearish and then you tell me if you're bullish or bearish on this thing. AI generated game assets like the percentage of AI generated game assets within a game

will go up almost certainly. I don't know that I will like that. I don't necessarily even know that gamers will like that.

Yeah.

But it will go up.

Rust replacing C or C++?

I see them both being around. I really do. I think it's pretty hard to replace C and C++. Just, I mean, we're still running Cobalt, man. You know what I mean?

Well, maybe with this AI thing we can.

Uh, but at that point, yeah, I don't know. I'm not really a big C++ fan to be honest, but I see it sticking around.

Okay. The mass return of performance focused programming.

Oh god, no. I mean, obviously not. I wish, I wish that were true. I've said previously that it would have been great if the AIS had been all about like just taking existing code and optimizing it. Like that's what we were working on. I think there is a market for that. It's not the huge market that is, I type in an English sentence and it makes my software for me. I understand why they're going for that first, but maybe someday we'll teach the AIs how to optimize and then it'll be good. Maybe someday.

So, bullish or bearish, Casey Merriator ever using an AI coding assistant for real work within the next 5 years?

No.

What would need to change for that?

I just don't really think that's something that I'd realistically be doing. This is not really a quick round question is the problem. We've already talked about my philosophy on it. So hopefully that makes sense as answered there because again, I'm not evaluating it as to whether I think it will produce better code. I'm evaluating as whether I want to use this thing or not. But one thing that we didn't talk about is also the thing that I like about programming is me learning or not even learning but figuring out a new way to do something. And in order to do that, I feel like I need to be there doing it myself. And that's not necessarily because I don't think you could imagine creating an AI system whose design was designed to help people like me. They obviously aren't doing that currently, but you could imagine someone going like, "Oh, for some reason, I don't know why, but you know, because there aren't very many cases in the world compared to people who just want to just want to type in make me a C compiler or whatever, right? We're targeting the types of programmers who like to figure out new algorithms and new ways to do things, and we're just going to make AI assistance for them. So, it's all about helping you like do your test cases and help and it figuring out new things for you or such things. I feel like part of the value add if I have one is that I'm in there thinking about it myself. And once the AI can do that, I don't think I would be necessary anyway. I think the AI could do it on its own. And so for me, in order for me to remain being valuable for the length of time such as that I will, which who knows what that is? Could be all the way till I'm dead or it could be six months from now. I'm not, I'm not necessary, right? Who knows? Part of that is that I was in there like the kinds of things that I do are about me being there and looking at it specifically myself. And so I also just think like I'm probably pretty useless at that point. And so at that point, I would just be doing it for my own amusement. And if I'm just doing it for my own amusement, I don't need the ADA.

Let's use the guitar player analogy. You like playing guitar. You like coding. You like

I don't like playing guitar. But yes, let's suppose I did.

Turns out there's some annoying parts about guitar and then there's the parts that you really like about guitar, right? What if the thing about guitar that you loved is just like just shredding, you know, just doing that like like the cool part like the the guitar solo. You love the guitar solo so much. That's the that's the learning part for you. Is the guitar solo. What if the AI just gave you you could just play the guitar solo the entire time? Like, and it took all the annoying parts about programming that you maybe didn't like or you were neutral about and it just gave you the best stuff. Like, so somebody wrote like an LLM or like some sort of GPT or rapper. They made an IDE plugin that just gave you the best stuff where you were that you could just do the learning part, the stuff that you liked and all the other stuff, it just kind of went away. That's more of like a product development perspective on what's going on. The way I look at it is anything that I didn't want to do, that's letting me know a place where there's something to be learned. If you take a step back and look at how people are currently doing software development, like 99% of it self-inflicted. It's like, you chose to have 17 package managers that run in series and like one of them breaks at every 5 days or whatever. Like that's all we could just not do that, right? So there's a bunch of things we could do to make life easier in that way, right? And then people say, I want to use an AI to automate that. I'm like, okay, yes, from product, sure. But what I want to do is get rid of all those things. I want to like, why do we even have them in the first place, right? So we take a look at that. When I'm doing my own stuff, obviously I'm not using 19 package managers, but there are, to your point, things where it's like, okay, I wish I didn't have to do this thing because, you know, I'm programming in C or whatever and C sucks at this thing, right? To me, that's just like, oh, well, if we're at the point where AI is doing all this code for us, then what I'm actually do is like, then that frees me up to go spend my time to figure out how do I design a better seat that doesn't have that problem. I want to know about that problem. I want to experience that problem. I want to figure out what is the thing that we're doing wrong here. Like, let's get back to a separate topic about the natural language that we didn't touch on, which is I don't like explaining how to do things in natural language. Actually, that's my day job. Like when I make educational materials, my job is explaining computer stuff in English. I don't want to do more of that. When I get to sit down and I'm doing coding, I want to program in a programming language. It's so much more precise, right? And it's so much more direct. I have great control. I'm the guy who wants to. It's like, oo, I kind of want to go write this in assembly instead of C because I didn't like what it did and I'm not sure, right? Like I'm that guy. So the chance that I'm going to want to explain this in English is like zero. And so that again, it's just, it's a very different perspective and a different philosophy on what programming is and why you're doing it. I don't disagree and have never disagreed that that's not the trade-off you're making if you're just trying to make Microsoft Word, right? If you're trying to make Microsoft Word, then all you really care about is did the AI do a good enough job and did it do it quicker than I could have done it alone. And when that trade-off happens for you is going to be different from for everybody. And to what extent you're going to leverage, it's going to be different for everybody, too, because it depends on your skill set. It depends on how good you are at like reading code, how fast, like there's all these parameters, but

So, bearish.

It's not bearish. Definitely not. It won't happen in the next five years. Will it happen after that? I don't know. But next 5 years, it'll, it won't happen.

Well, Casey, thanks for coming in and having a level-headed conversation about AI. We're just, we're just increasing that small pie slice of the pie that's like level-headed AI discourse. So, really appreciate you stopping by today.

Thanks for inviting me and thanks everyone for listening. I just want to say again, like, in case I didn't make it absolutely clear, like, I'm just, I'm not an expert on this stuff. So, I always happy to talk about it because I find it's always interesting to talk with other people in the industry about things like this, but I have no idea where it's going. Right. Every year it's going to be something different and I think predicting it's very hard.

Just the last thing, it's like, it just feels like you're, you're my friend that like never watched Star Wars and it's just

That's exactly right. That's exactly right. Like, okay, so you're gonna start on A New Hope. It's episode four. You're like, "Well, why does it start on four?"

Yeah. Yeah.

If that's the first one.

Yes.

You're like, "Oh, God." Okay. So much to talk about here. Everyone who's annoyed right now, everyone who's annoyed by the fact that I don't use it, just think about how useful it is to have to have that person because you can show me in 10 years, I'll be the boy in the bubble who never saw like, you know, like never saw any of it and I'll be really impressed. I'll be like, "Wow, that's really cool." It's like, "Oh, like Darth Vader was his dad." Okay. Anyways, blew my mind. Spoiler warning. Oh, sorry. Spoiler alert.

Steve.

Okay.

Awesome. Thanks so much.