Transcription
Really, there's nothing that is working. There are a lot of toys, a lot of experiments. There are things that are marginally working.
The way to overcome slop is for the platform to expend more effort and for the developers of the platform to imbue the agent with taste. If you want to move fast and you want to ship things, you need to throw away a lot of code. It's a lot of work. It's a lot of work, but you have to do it in order to get to market.
Vibe Coney is taking off partly because it's the best way for companies to adopt AI in a real impactful way. There's always this push and pull between what the model can do and what you need to build on top of the model to add value. You need to be building sort of semi-working products, crappy products such that the moment there's a new model that unlocks a capability, the product that wasn't working or wasn't working very well immediately starts working and you're first to market. And by the way, you can't have a road map. You can't have an amazing road map. I mean, that's a big change.
The TAM for AI agents is not software. It is labor. And that's in the trillions of dollars. You get into 10 trillion. Fast forward 5 years. Does the junior developer role exist at an enterprise company or has that entire tier of labor been absorbed by the Replet agent?
>> Yo, I'm not, I'm not sure. This is Venture Beats Beyond the Pilot. Enterprise AI in action. I'm Samine.
>> I'm Matt Marshall. Today's episode is presented by Outshift by Cisco. Cisco's emerging tech incubation engine and driver of Agentic AI, quantum, next-gen infra, and beyond. Today we're talking with Amjad MSAD, the CEO of Replet. Replet has a habit of living in the future. They put the entire development environment in the browser when critics said it was impossible. They launched Ghostwriter, an AI coding assistant before ChatGPT made generative AI a household name. Now Amjad is making the biggest bet yet: the shift from AI as a co-pilot that you supervise to an AI as an autonomous agent that you manage. For us, what makes this story so compelling isn't just the technology. It's Amjad's honesty about the current state of the market. While the industry is drowning in hype, Amjad argues that most enterprise agents being built right now are effectively slop, unreliable toys that fail the moment they touch messy real-world data.
Welcome to Beyond the Pilot.
I'm John. Thank you for having me. You've historically been months, if not years, ahead of trends, right? You mentioned recently that you try to predict the future by building it. When you look at the current state of Agentic AI, what is the specific signal that you're seeing right now that you think everyone is missing?
There are two things that are absolutely working. Two roles that are getting automated or augmented, depending on how you look at it. Uh, those are support, you know, there's startups like Zendesk and others that are automating support. Uh, Parah help is a good one we use. And then, uh, software. Outside of that, it's really there's nothing that is working. There are a lot of toys, a lot of experiments. There are things that are marginally working. I think a lot of people just call anything an agent that does AI. The way I define agent, which I think there's an original definition from the AI scientists that were thinking about it 30, 40 years ago, it's something that should be autonomous, means it acts without human supervision. And second, it should be able to take action, not just retrieve information. Information retrieval is a chatbot, ultimately. So chatbots are working. They don't add a lot of value to the enterprise. They're not really automating work. They make people more effective. They save time in summarization, writing, things like that. But when we talk about enterprises actually automating work and reducing the need for more people, more labor, more complexity, more systems, more software, the bottleneck tends to not be the AI. The bottleneck tends to be the sort of the messiness of the data infrastructure across large enterprises and governments and so on.
>> Right? I, I want to get into that. Um, so, so you moved from co-pilot to full-on agents faster than almost anyone.
>> Right? This move to Agent V1 and V2 and, you know, recently V3 about to come V4.
>> Was there a specific moment or a model capability? I think we've referred to Sonnet 3.5.
>> Yeah.
>> Um, where you realized, okay, the human is no longer the driver. It's going to be
>> the manager of, of this agentic process.
>> Yeah. Yeah. So there, there are a few things that happened in LLMs that were on the path to unlocking agents. Uh, I think around 3.5, we started seeing something called tool calling. There was a lot of hacks to do tool calling. I actually kind of prototyped the first way of doing it with GPT-3, which is you just have GPT-3 output like a piece of code and then you run that code in an execution environment, and that's how you did tool calling. But they started training LLMs to do tool calling. And so that's when you say, when you look at it and you say, okay, now we can actually take actions, but it was very unreliable. The problem also was context size. The context length, how much you can fit in the immediate memory of the AI was very, very small. At the time, it was like 2,000 tokens. Now we're at a million. So there's been multiple orders of magnitude of jumps.
October 2023, I had given a talk just like a couple months before I realized that we're on the cusp of agents and I described how we're going to be making software in the future and described how autonomous agents will work. Still the future that is coming with Agent V4, which talks about parallel agents and all of that stuff around, you know, New Year 2024. I just had an intuition that we should really try to do it now. Like it felt like it was almost possible. And I have this belief and I keep telling my team this is very important, and I think every AI company should be doing this. You need to be building sort of semi-working products, crappy products, such that the moment there's a new model that unlocks a capability that you care about, the product that wasn't working or wasn't working very well immediately starts working and you're first to market, right?
And the other thing that was happening is that the business was in, in a bit of a trouble. And I always thought that like Replit's mission is about making programming more accessible, making it so that people don't have to code a lot or code at all in order to build software. That was always the intuition with, you know, Ghostwriter. It was still an assistant. It wasn't really doing the whole thing for them. And so people got excited about it. We got quite a bit of usage, but people could not get to production applications. So 2024, it was both a necessity. We need to transform the business, and it felt like it was around the corner. So we started with GPT-4o. 4o was really good at tool calling. The context length was not bad. And we built a system and I remember the first time I ran it, it was able to build like a very simple app. It was like an app that like pulled cat images from the web and there was a button and generates a new cat image every time. And it would run for 30 seconds. It could build an app in 30 seconds. If it couldn't build in 30 seconds, it would just crash, go into crash loops and start speaking Chinese and doing all sorts of things. Becomes really unhinged.
But here's where the luck happened. Or maybe you could think that the prediction was correct. June 2024, Anthropic 3.5 came out. And there were two characteristics about Anthropic 3.5 that were important. One, it wasn't lazy. It could write large amounts of files and not make mistakes. So it can spit out an entire project. So your first prompt, more likely than not, it will succeed. Unlike with the, you know, GPT-4o at the time, it can only, you know, give you a block of code and then yield back and ask you a question, is that what you wanted? It was made really to be truly a chatbot and not something that could take actions. It was clear that Claude, that the researchers at Anthropic wanted to build a more autonomous agent so it could work for longer. Two, it felt like it was more directed, so it could chain more tool calls with each other. It made less mistakes between these two tool calls. Uh, and that gave it coherence over long periods of time. Still not that long. Like it could work for two minutes at a time and then, you know, if you let it run for longer, it will go off the rails. But two minutes is enough to get a lot done. Right. The moment we plugged it in, everything got better. We went all in on that and then we launched Replit Agent. It was the first coding agent on the market that you could try. There were a lot of experimental open-source things that were very hard to use, but Replit was the first consumer-grade product that you could try. It was still not that great, but we launched it. And the cool thing about it, it wasn't just writing the code. It was provisioning the infrastructure for you. It was provisioning the database, migrating the database, configuring the deployment, doing all that stuff for you. So, we talked about that and it went, it went super viral. At the time, Karpathy called it a moment of field AGI, because you could tell that agents are starting to work. And so that was September 2024. Claude 3.7 came around November, December 2024, and this is when computer use started working longer, coherence started working. And then we're off to the races from there.
>> You mentioned Andrej Karpathy, well-known former OpenAI um leader, uh, in many other positions, and now kind of independent, focused on education. He recently called a lot of agentic work slop, right? Uh, and provoked a conversation again in our prep. You agreed that, you know, just now, outside of coding and customer support, there are very few agents that actually work in the enterprise. And we saw the risks of this in your own high-profile event in July, I think, right, when there was a famous developer who had their entire database erased, despite a code freeze. So the question is, is this a fixable guardrails problem, or does it prove that non-deterministic models simply cannot be trusted with write access in production enterprise environments?
>> Yeah. So I think it is, in a way, it's a more complex answer. Perhaps it's both. It's worth defining slop. Slop is when models generate generic things. So if you give it like a lazy prompt and it just one-shots something, usually that thing that one-shots is not great work, and also it is generic, like everything looks the same. So it is kind of sloppy. And most AI products don't go that deep, and therefore, like they generate all kind of, there's a lot of sameness out there. Everything kind of looks the same. All the images, all the code, everything. The way to overcome slop is for the platform itself to expend more effort and for the developers of the platform to imbue the agent with taste. Relying on the underlying foundation models will get you the generic answer. But we have our own prompts. We have our own design systems. We have our own way. Like, for example, for a design system, we classify depending on what you want to build. Do you want to build an e-commerce app? We classify against a library of different design systems that we pull into the prompt. So we do a RAG over design systems, we pull into the prompt, and then also we spend a lot more tokens to make sure that the output is higher quality. That makes it a little more expensive, a little slower than others, but the result is better. The other thing is, if you introduce testing in the loop, you can give the model feedback and have the model reflect on its work. So, Replit really is the only product in the vibe coding space that can test its own work. So, after we generate the first generation, we kick off to a testing agent. The testing agent boots up a browser, looks at the page, tries to click the button, do whatever it needs to do, and then the testing agent will tell the coding agent whether the test was successful, how the page looked, whether there was something sloppy about it. And then, and then we loop a few times, and then we can generate the answer. And that way, the product that you're giving to the customer is high effort and less sloppy. Obviously, we also teach customers how to prompt and how to imbue their own taste into the product. And we, you know, initially our marketing was sort of more on the magical side. It was like, yeah, prompt to app, and that's still true. But even in my podcast and in my appearances, I talk about the effort and the creativity needed on the developer side. And a lot of them are not developers, but they should think of themselves as making something that requires iteration. So if you don't like something about it, change it. And we're adding a lot more tools to give people more creative space. Often times, what you want to do is you want to pit the model against each other. So the testing agent is a different model than the coding agent. So you're relying, like from a mathematics, statistics perspective, every model has a different distribution of knowledge. And so when you pit them against each other, you can find different distributions, and that way you generate more variety in these outputs.
At Replit, as much as we invest in AI and we have an amazing AI team, our DNA is infrastructure. We think about reliability from an infrastructure question. Now, you have to take for granted that AI large language models are statistical machines that will always make mistakes. That's, by the way, not too different than humans, right? Like errors will happen, hallucinations will happen. How do you recover from it?
So there are a few concepts in computer science that are already there. Um, when we had that incident in July, we had been working on isolating the development environment from the production environment. At the time, you could use the same database in development and production. It required extra steps to isolate the databases, and our developers don't know how to do that. So we started doing it by default. So now when you get a development environment, you get a database embedded in your file system, and that's a development database, and you could do whatever you want with it. The agent can break it, can delete it, whatever. But the moment you hit publish, we generate a migration based on the data model, but not the data itself. The data in development stays in development, and then the data in production is not touched. The only thing that's happening is we're migrating any data model changes that you're making. And so that isolation makes that mistake that happened with JSON to never happen again. Right? So development databases are only accessible within the development environment. Production databases are only accessible from the code in production. So that's one way you can think about reliability is isolation. The other way is reversibility. It's very important then that any action is reversible. In the development environment, when the agent makes an error, we have a button that says roll back, and you can just click on that, and the entire state of the machine rolls back, including the database, including the storage, including the file system, everything goes back in time. We spend a lot of time refining that. So now when you're in development, you can sort of give the agent as much autonomy to make a lot of changes, but anytime it makes mistakes, it's just like a one-click rollback, and that's very important. You know, that gets to the kind of larger topic that we're talking about with not everything is reversible. Some things are easily reversible in a development environment. A bank transaction might be really hard to reverse. I think that's part of the reason we're seeing a lot of adoption in software, because in software and development, you reverse things. But in a lot of enterprise workloads, some things are not entirely reversible.
This series is presented by Outshift, Cisco's emerging tech incubation engine and driver of the Internet of Agents, an open, interoperable internet for agent-to-agent collaboration. Learn more about the Internet of Agents and explore how agentic systems are the future at outshift.cisco.com.
In AI intelligence, there's a squishy divide where, and I think you've argued this too, where, you know, that the AI agents work in code because there's this verifiable unit test, you know, either compiles or it doesn't. It's yes or no. But with, you know, many CIOs of enterprise companies are trying to build agents for squishy stuff like HR policy, you know, legal strategy, marketing analysis. Um, so, you know, can we be honest with them?
>> Yeah.
>> Right? You know, if there's no unit test for a marketing strategy, are those agents doomed to fail? Should enterprises stop trying to use these agents as judges?
>> That's actually a profound question that gets at the heart of the question of AGI. If we train models that are really good at the verifiable domains, is there a transfer of knowledge and they become generalizable to more squishy domains? Right. Um, so far, it's not been the case. So far, when you make a model really good at like math or software, it gets better at reasoning on these things, and it gets marginally better at reasoning about the less squishy end of the continuum, like legal gets like a little better. But by the time you get to marketing, it's like you're not getting any generalization. And so I think that's a question that all the big labs are going to have to really innovate around or figure out, because what they're doing right now is they're building these reinforcement learning environments that have these verifiable rewards for math, for science, for that. That's why you see Sam Altman harping on science. It's like, oh, we're going to see scientific discovery. That's actually easier than having a, you know, a great legal expert or HR expert. Right now, without the ability for the models to have really good taste, really good reasoning in the squishy domains, it's really hard to deploy agents in the enterprise. And I think that's a big part of it. There are ways around it. And what the big model companies are doing are they're buying data packs and RL environments with a human in the loop for every particular domain. So Gemini 3, for example, had this huge breakthrough in in web design.
>> Yeah.
>> And we built a design experience around it, and it's actually really good. My guess, and I don't have any insider information, but my guess is that they went to a data partner. They said, "We want you to go get the best web designers in the world, create these RL environments, and have them train, generate data for really great web design." So, Richard Sutton came up with the bitter lesson. AI used to be built in a way where we're relying on human expertise. Think about rule-based systems and expert systems in the 80s and 90s. And then the reason LLMs were felt like there were this path to this general intelligence is because you can just throw internet data on them and they'll just get better at everything. But up to a point, things have slowed down. Now there are rumors that we're back in the pre-training, and pre-training works continue to get better, but it's slowed down. And what the labs have shifted is back to sort of pseudo-expert systems. That's why you see Surge and Scale AI and Merkle and others like have these really large revenue ramps, because they're so hungry for data. They're going out and buying. So they focus on web design, they got better at it. Maybe they'll focus on marketing. So there's a way to get there. It's just not going to be easy, and it's going to be very domain-specific.
Yeah. And I've seen you being a little more bearish on the timelines for AGI as a result than some of your other brethren, and others that are running companies that kind of count on the hype to actually keep raising money, right? Um, you know, I wanted to move before we get into some of those other things you were just talking about to this question about vibe coding versus specialized agents. Um, you know, you've popularized vibe coding, obviously, this place where you're starting and then just putting something in the prompt and having it code an app for you, where the human just manages the vibe. Um, but how is vibe coding consistent with an enterprise that needs specialized agents, right? It is it a compliance agent checking a payment agent, right? So, so I guess the question is, are you starting from a single vibe coder, or do you see a future where there's a meta-agent that governs a team of specialized sub-agents for enterprise usage?
>> Yeah. So right now, Replit's usage in the enterprise is someone who's a domain expert, an HR specialist, salesperson, marketing person, is sitting in front of the computer. They know the kind of steps they need to do to do a particular job. Someone that works as a deal desk person, configures quotes and things like that for salespeople. She built a quote configurator on Replit that automated her entire workflow. And that's a combination of a UI for the salespeople to go in and kind of put in the information about the deal, that app pulling information from a CRM like HubSpot or Salesforce, and then a bot that posts the PDF on Slack, and then there's a discussion or review in Slack, and then they give feedback to the agent, and the agent goes back and makes some change and then uploads a new document. And so it's a fairly complicated system, but it is not like a true autonomous agent. And so the way employees in the enterprise are increasing productivity is they know they have to do a series of steps, and they embed them in a piece of software, more traditional software. So I'm using agents to create software that makes my job easier, and that saves me from going out and finding a vendor that I'm going to onboard, and typically it's not going to exactly solve my problem, and I'm going to ask for support. Maybe they'll build the thing that I want. And so SaaS has been painful in that regard. And so this really supercharges the enterprise's ability to create custom software, solve problems. The ideal goal is to be able to describe the entire workflow to the agent and for it to do it autonomously, and do it over and over again autonomously without human intervention, without even a human sitting in front of a computer and using a software.
So we're not there yet. And I think that the reason we're not there is one, the frameworks around agents, everything that we described for software, there's the same set of challenges for every other domain. Data is messy. The enterprise right now, we're building a lot of data connectors, and we're trying to figure out how to surface that data and the schema correctly to the agent. You know, you can put in a really high-level prompt like, you know, I am a marketing person. This is my job. This is what I do. Like, create agents that can help me do my job on a day-to-day basis. It's like automating real labor. It's still, we're still a little further away from that. And the way you're compatible with current enterprise codebases and environments, I assume is somebody then codes that, vibe codes it, and then takes that code and then plops it into their environment.
>> Yes. Yeah. They can one-click publish, and they'll use it. Other people will use it. We are building an agent environment. So we have a new feature called Agents and Automations that allows you to build these automations. It'll visualize automation for you. Gives you a debugging system to help you find the traces. But it's still early. We're starting to see some really interesting usage, especially again around information retrieval and things like that, like Slack bots and very simple triggers like, when I get a Jira ticket, I want you to summarize it, enrich it, somehow post it somewhere else. So a lot of it is just pipeline and plumbing. But when I think about agents and how agents can transform the enterprise, it needs to be agents that are taking actions on behalf of, like the autonomous actions and doing real work, and we're still a little further away from that.
>> Before we move on from that starting point, that vibe coder sits down and then they're dealing with Replit's, basically starts as presumably there's a meta-agent that starts and then delegates everything. Can you just confirm what that reasoning model is? Is it Sonnet 4.5? I mean, you're close with Google. You know, they've got Gemini 3, or do you switch them depending on the case study?
>> Yeah. So we think of ourselves as a company as like an agent lab. We find the best models for the best job and use that model. The backbone of the agent right now, the coding agent and the orchestration agent is actually Claude Opus. Opus came out a couple weeks ago. It is a little more expensive, but it's also more efficient, and so we standardize on that. We used to give it as an option for a high-power model, but now we standardize on that. And that's like really the backbone of it. It is the best software development agent, and I think it's definitely the best sort of reasoning agent for long-horizon work. Um, but now we have a lot of sub-agents. So, for example, the testing agents, I forget whether it's using Google or OpenAI, but we use like Google Gemini Flashlight for codebase search.
>> Right.
>> Because it's long context, it's cheap, we can put the entire codebase there, we can search in the codebase.
>> Extremely quick.
>> Extremely quick. That's very important. We use Gemini 3D for design. So we're constantly finding new use cases for every different model, and that's what our job is, evaluating these models. And but definitely the workhorse is Opus.
>> Okay. Yeah. I want to move now to kind of the mechanics of autonomy, right? This idea of building these autonomic autonomous workflow apps in the enterprise and some of the limitations that we're seeing, specifically, I mean, digging into some of these concepts we've already talked about. Let's talk about computer use, which you brought up, is a big challenge. You know, you mentioned recently, general computer use. You know, when we talk about computer use for our viewers, I mean, this is basically moving a mouse around, clicking on pages. Um, it's expensive, it's buggy. Yet you've mentioned you built this custom framework, or you called it a hack, I think, in one of your interviews, to make it work for Agent V3. So the question is, is this a new moat, perhaps, or one of the potential moats? Did you have to build this as essentially a prosthetic limb for the model because Claude or whatever the model you're using isn't good enough for that? How much cheaper and faster is your method? I mean, I think I saw a reference to say three times as fast or 10% as expensive or something. Could you just—
>> Computer use in general, like there are a lot of problems in software development, but also just generally in the enterprise that require actually moving a mouse on a screen and typing on the screen. That's why—
>> Well, on testing if an app actually works.
>> If an app works, or robotic automation, that's why it's a kind of a big industry. Um, so not everything can be interacted as an API as code. A lot of things need to be interacted as if it's using a computer. That requires vision and all of that stuff. If it really worked, it would have been hugely disruptive because QA is a huge industry. Um, you have so many office work that's very routine, and you can just do it with AI, but it doesn't work very well. In some cases, actually more expensive and more error-prone than humans. And it's a very tough problem. I know they're working on it. They'll crack it at some point. But when we wanted to add this verifier in the loop, we knew that it needs to test apps. When we plugged in a generic computer use model, it just didn't work very well. It was buggy, expensive, and slow. If you're the developer that writes the code, you can add certain annotations to the code. And so our coding agent annotates the code, and we have like a vision model that's looking at the app, just making sure it works. But when we go actually do computer use, we're not using a model that's moving an actual mouse on the screen. We're doing it programmatically.
>> Right?
>> So we're writing code in order to test that. So that's why it becomes a lot cheaper and faster. Um, and I think it is something like 10x cheaper and like three times faster or something like that.
So is that a moat, or is that just now generic so that people can come? I think that the larger moat is our ability to solve these problems quickly and be first to market and be ahead by months before AI solves that problem.
But where, so where is that moat coming from? Uh, so the infrastructure. So, for example, the way we built that computer use system is of all the vibe coding tools, Replit is the only one that built infrastructure to boot up virtual machines in the cloud very quickly, to be able to fork a virtual machine very easily and merge it back. And so when we do the computer test, computer use testing, you have the coding agent coding the development environment. We fork the development environment. We have the testing agent running in an isolated environment because it might break it, it might do something wrong. But that happens really quickly. We start a fresh sandbox environment. We do the testing on it. We break it. We do whatever we want with it. And then we come back with a summary to the main coding agent. If you don't have that infrastructure, and by the way, the infrastructure should also stream back the results to the user and to see the visualization of it clicking on a mouse and doing all of that stuff, it's going to take a lot more work in order to get there.
>> So the moat is more time. I mean, you've just compiled these steps. It's not necessarily secrets or innovations that no one else can innovate on. It's essentially a compilation of all these steps before anyone else has really been able to get to it. Would that be accurate? I mean, Vercel is an example of a competitor who's kind of emerged very quickly, but they haven't had the time, presumably, to build a lot of these things that you've been working years on.
>> Time and talent.
>> Yeah.
>> Uh, I think a lot of the new vibe coding tools, a lot of them are really good at product design, marketing. But we have infra, deep infrastructure engineers that we hired from Google Cloud and AWS and all these places. And so Replit, underneath the covers, is a true cloud company. And so we have a lot of innovations that are really hard to build. So, for example, the file system that I just described, a copy-on-write file system that can fork on the order of milliseconds, that took us three, four years to build and mature. And it would take as long time. So, yeah, when you think about moats, often times people think about network effects and things like that. Those are really lasting moats. I think technical advantage is often a gap that could be closed. But if you're constantly innovating and you're constantly hiring the best people, and you really care about this, and the leadership is really focused on innovation, you can continue having that gap in perpetuity. But a lot of companies don't. They get bigger.
>> Yeah.
>> They get fatter, they get slower, and that moat sort of goes away.
You've mentioned this concept of the verifier loop that you stumbled upon. I think it was, you know, last year you talked about Nvidia's using DeepSeek in a verifier loop, um, and then you jumping on to that as another way to help remove hallucinations, um, and increase autonomy.
>> Increase autonomy. So you, so let's talk about this. You've talked about moving from two minutes to 20 minutes to 200 minutes, more recently, but it's actually now up to 20 hours. I mean, how do you have an agent running for 20 hours fixing its own mistakes without getting stuck in some sort of hallucination spiral?
>> Yeah. So, uh, so that Nvidia paper that you're describing, what they did is they were trying to get it to autonomously write CUDA kernels. CUDA kernels are really hard to write. They found that it can one-shot a C kernel would be working like 50% of the time. That's no good, right? That's not really autonomous. You need a human looking at it. And so they added a compiler and a verifier tester in the loop, and they were able to like do the first shot, test it, doesn't work, give feedback to the agent, do another shot, test it. In theory, you can do this ad nauseam, and you increase autonomy forever. In reality, you run out of context length.
>> Right?
>> So another thing you need to do is context compression.
>> Okay?
>> And that's really hard as well. Doing good context compression without losing information is super difficult. So we spent a lot of time figuring out the best way to do—
>> Is that compaction or—
>> Compaction? Yeah. People are working on that. Google's working on that. Yeah.
>> Companies like Weka or Vast Data, others, I think KVach—
>> Hits and so on.
>> Yeah, compaction is very, very important. It could also be very expensive because you destroy the KV cache because you're entirely new prompt. So doing it in a cost-effective way is actually fairly difficult. But if you get these two things right, that's when you get autonomy for running for hours. It does a bunch of work, goes into the verifier. The verifier gives it feedback. If it succeeded, it yields. If it doesn't succeed, it gives it that set of feedback. If it ran out of context, you need to do context compression. You need to carry over the information that is important. Uh, now, do you have to worry about that or no? I mean,
>> or is that taken care of by your infrastructure? You're a partner of Google. You rely a lot on their runtime and—
>> Yeah, so the thing is, compaction as a service, compaction APIs weren't there when we did compaction. So that's the thing about AI. Often times you're solving problems that will eventually be part of the platform. But again, if you want to move fast, you want to ship things. It actually is a fundamentally different way of working than most SAS and software companies. You need to throw away a lot of code. It's a lot of work. It's a lot of work, but you have to do it in order to get to market. And then you have to have no ego about it. Like the moment the models do it better than you, you're like, "Okay, good riddance. Let me move to the next hard problem." And so it's a very different mentality. And I think a lot of companies get stuck with that. It's like, "No, we innovated this, we need to keep using it."
>> Does that mean there's an advantage by staying really tight with maybe your— I mean, you've got this close partner, Google. Do you have—
>> We have a close partner position to know what's on the roadmap.
>> I mean, did they open that up?
>> Uh, we get early access to all the models there.
>> Okay.
>> And it's important, like I mentioned before, it's important to try to predict where things are going. Like we know for a fact, computer use will get better, cheaper, faster. Right? But is it going to be a month or a year? Can we wait on the roadmap? We wanted to launch it today. I expected it to get better on the order of like 12 to 18 months, and so that innovation will last for like a year or more, and so it's worth doing. Although at some point, we got to throw it away.
>> Is that pretty much solved that—
>> Yeah, like memory compression.
>> Oh, no, memory compression. I think those APIs, the SDKs that they're launching, they're pretty good. I think ours is better.
>> In, in—
>> You're talking about Google versus yours?
>> Uh, Google, Anthropic.
>> Anthropic. Yeah.
>> The thing about compaction is that if you understand the problem domain better, then the generic thing was not going to be very good for you. Because they're trying to do compaction for software engineering agents. They're also trying to do it for chat. We care only about agents that are making apps. And so because we have a narrower problem, we can do a better job at that. But at some point, the general solution will be good enough, and then you can offload it to them.
We talked about the mechanics of autonomy, and now we're moving to the infrastructure. Um, so there's this tooling gap that you've, you've talked about all these levels of things that you've touched on that are stopping these autonomous agents from happening. There was some news today, literally this morning, I was reading about Harness. I don't know if you caught it. You've been on the road, you were in Dubai yesterday. of Aqua to build software delivery knowledge graph for agents. They want to tackle that trust and observability layer. They want to be a Switzerland versus probably the solutions being offered by a Google or AWS, where the solutions kind of stop inside their cloud environment. You know, the question is, do you get a sense that companies like Harness are close to cracking some of these other issues, these layers that are needed for enterprise to generate some of these apps?
I think all these tools need to exist, and it's something we can plug into. If we can, you know, that's a very important problem. Understanding the policy around, you know, DevOps and IT. Understanding the data policy and data privacy is another very important problem. I think Databricks has an agent or like an AI API that you can call that gives you like a description of the schema, which is really, really important. Good, because like most database schemas are really hard to understand. And so for us to build like really good agents, really good software on top of existing databases that companies have, it's important to have descriptions of these policies, of these data schemas, and how they're used, and all of that. And then there's like the business workflow stuff, like which is written down in random Notion and Google Docs and Word Docs across an enterprise. That's the hardest. That's really the hardest. And this is, I think, going to be even after we solve a lot of the bottlenecks around reliability and verifiability and things like that, that's still going to be a big, big problem. Because again, this is the ultra-squishy things. This is the things that humans understand. We're like to go find certain things, and they talk to each other, they'll figure things out how to do a certain workflow. But for us to encode it in sort of workflow automation and agents, it's really difficult. So if someone can go solve that problem, that'll make our lives a lot easier.
>> Yeah. Yeah. So I'm, this is super useful, right? Um, so if we were thinking about agents with a, say, a plumbing analogy, often people talk about the plumbing of infrastructure for enterprise. And you mentioned that for agents to scale, there are these reliability issues, verifiable issues, we talked about the squishiness, all these things. And I'm wondering, um, for the architects in enterprise who are listening to this, what are the other pipes that are leaking, like in this analogy? I'll throw a few out that maybe we haven't touched on. There's context layer or kind of state management. The data layer, right? You mentioned these docs off to the side, right? These messy SQLs that an agent can't read. Um, is it the auth layer, hitting a login screen and not being able to log in and dying? You know, access control more broadly, right? Like people in the enterprise, like different types of users, and these users have different access to different data, different levels of access, and that's also super important to, um, to kind of have in the context of the agent and a platform like Replit to be able to kind of have the right privacy controls. Um, yeah, I mean, there's there's a lot. Legal policy, you know, HR policy, all of that stuff. I think needs to be platformized. And for AI to deliver on true productivity gain, if AI is going to truly affect, you know, GDP growth, not in a way in the build, like right now it's affecting GDP growth in terms of cost. But in terms of efficiency and productivity, then all these systems need to get exposed in a way that allows us to build better software, to build agents, to build workflow automations on top of that.
>> Right. Yeah. So it just feels like there's a lot. This is going to take until well into 2026 to—
>> Oh, to really figure out. Much longer.
>> Much longer. Yeah. Yeah. Yeah. So okay, so, so then with that in mind, right, let's talk about the—
>> which is why vibe coding is taking off in the enterprise.
>> And and you might have seen, you know, Replit is the top software vendor in the world. Ramp is this finance provider, bank, bank, essentially finance provider that's releasing data on what apps are actually being paid for by their users. They're anonymizing the data, but they have some really good data that they're sharing about what people are actually paying for. And you were at the top of the list in terms of new customers, as I saw. I also looked at the data and I think fastest growing as a percentage of existing. So on both metrics, number one. That's amazing. So continue the point. Like why is that happening? Vibe coding is taking off now.
>> Yeah, vibe coding is taking off partly because it's the best way for companies to adopt AI in a real impactful way.
>> Right.
>> Because all the other stuff that we talked about is messy, hard, and it's really hard to do all this stuff. The easiest thing is to give domain experts, to make everyone in the enterprise a software engineer, to give domain experts. And even when they don't have access to the right amount of data and the right everything, oftentimes even just uploading PDFs and text files and CSV files and downloading things, and they can solve a lot of problems and they can automate so much and they can rely less on SAS. And that introduces better efficiencies. And so it is the most impactful way that when we go into enterprises and we do hackathons. Like, for example, I was just at, um, in Saudi Arabia, doing a hackathon with a Public Investment Fund, PIF manages a trillion dollars. And we had the executive team and the governor of PIF build software on Replit. And, you know, their AI strategy team, they were talking about different ways.
They're going to adopt AI, and the frustration that we sensed is like, there's so much hype, but all we see is chatbots. Chatbots are an improvement, but they're marginal improvements to work. What can we do right now in order to automate, create more productivity, and so on? And so, uh, really having a vibe coding environment like Replit accessible across the enterprise and having people solve, uh, daily problems, uh, that they're having is one of the more impactful ways to use AI in the enterprise.
I think, uh, orchestration versus the model. There's, there's a debate right now. You've touched on it. Do we need complex orchestration frameworks? You know, LangChain and SDKs, uh, ADKs from from Google for for building agents. So, should we just trust the model to do the planning, right? This bitter lesson approach that you talked about, you know, now now you've told us that it's this is going to take years, and so presumably the bet is on the bitter lesson and watching the models. I mean, where do you stand? I I go back to this idea of like being tactical about it, not being flexible and tactical about it. Not ideological, not ideological. And it'll ebb and flow. So, uh, agent in beta had a much more complicated harness than, uh, agent one. That's because that was there was a that was a bitter lesson moment where 3.7 got a lot better at agentic work, and then 4.0, like Anthropic. Um, and so we were able to trim down a lot of the harness and like led the model for example, that's something we did early on, a lot of companies and now sort of catching on to it. In software engineering, it is better to let the model use the shell and do grep and do find commands in the shell rather than doing a RAG over code. And so that's ripping out a system, ripping and RAG systems can be, um, complicated. Ripping out the RAG system and telling the model, well, you know, if you want to find a file, go use the shell.
Yeah. Right. And that way it got simpler. But I will say since agent one, we we got back, now it's more complicated because agent three does a lot more. So we for the same set of features, we were able to shed quite a bit of complexity, but now we're accruing more features, and so we're accruing more complexities, and then at some point the models will get better, and we can shed the the testing, uh, framework and model that we built.
Yeah. Uh, and so it'll ebb and flow, and you know, and so you need to be very, um, zen about it, and you need not have an ego about it, and you need to change how you make software, um, and you need to be able to absorb what's happening. And by the way, you can't have a roadmap. You can't have an amazing roadmap. I mean, that that's a big change. Like we have a roadmap for enterprise features, like we want to be HIPAA compliant, like June or whatever. We can make these promises to enterprises, but when they ask us about capabilities, we can, uh, we can roughly tell you what's going to happen in like two or three months, but we can't exactly tell you what's going to happen. And internally, we'll be working on a set of projects. A new model comes out, we need to drop everything and like figure out what this model does and eval really quickly and be very agile around that. Like Replit was not the best at design. Um, and overnight we became the best in the world because we figured out how to prompt Gemini correctly and get the best designs out of it.
In your ter in terms of launching that design product, what was it last month?
Yeah, last month. Yeah, Replit Design. And so it's the best.
So can we can we move on to the cost equation? Um, you know, Mike Clark from Google, director of of product, talks about these massive reasoning chains driving up token costs. I mean, others have talked about this as well. Um, so you have this second cost, which is infrastructure, right? Every agent needs a container to run and test the code. You you've been on the frontier of actually having this happen. So you guys have been bearing this extra cost. And maybe going back to remember where we met at at Humanex, um, this event early in the year. I think you were worried about you and your set being kind of negative. I I don't know if you actually said this, but you were concerned about negative cash flow, right? The cost of of delivering these things for the for the time being was pretty expensive per the immense competition. So you couldn't charge as much. How is that going now in terms of the economics? Now I guess the question is how do you make the unit economics work when you have to pay for both the the brain, you know, the tokens, and then obviously the the container that you're using? In the Web 2.0 era, building internet businesses, we had this like magical formula, which is the marginal user cost is about zero. Yeah. Right. And you can pack thousands, if not tens of thousands of users on a single virtual machine. Now we're in an era where, like a token is a token. You can't share a token.
Yeah.
In the same way you can share a virtual machine, right? So there's like this one component. Just like being honest about it, it's like it's a new era of the internet. It's a new era of technology. It's a new era of software. The margins might not be the traditional software margins we're used to, but those are going away because they're all getting disrupted in many ways by AI.
Right?
However, the pie is a lot bigger. So, you go like the entire SaaS market is half a trillion dollars. What AI agents, the the TAM for AI agents is not software, although it is that too. It is labor, and that's in the trillions of dollars. You get into 10 trillion dollars of of potential TAM, right? And so yes, that you know, we might, uh, not get to like 90% margins. 50% is is possible. But the reason we we go from 3 million to 250 million in a little over a year in revenue is because the ROI is so great for companies and for individuals, uh, and it's a labor substitution. We have companies like Jason, who we talked about from the anecdote earlier. He told us he was using so many agencies and he was going to hire developers.
This is the guy who lost his database and he became a rough super fan afterwards after solving his problems.
You apologized to him, I think publicly.
Yes. And and and he just, he he uses Replit so much and he doesn't need to hire people because he's using Replit. And that's how you increase productivity. And so again, uh, the opportunity is a lot bigger, but the economics is different.
Would you, are you an economics positive for you now, or do you see it coming?
Yes. I mean, I I think it's important for us to to grow right now, but but, uh, but yeah, we we got into financially, we're we're doing really well. You know, I tweeted like a couple months ago, we haven't even touched the round we raised in 2023, right? And so we're still in the money burning regime, but like not as much.
Yeah. That's amazing. So that essentially means you have 250 million in cash that you can draw down.
Uh, yeah.
At least. But but but you know, you know, uh, sometimes we're going to like launch experiments or things like that that are going to be more expensive, and we're going to eat the cost. And and so the important thing is to recognize that the opportunity here is so so massive. We know we got to a point where the unit economics works. We know it works, but the important thing right now is.
Congratulations. That's amazing. So, okay, let's move to the future. Um, so open versus closed ecosystems. Uh, you know, you're building a vertical stack. It's clear to me, and I've tried tried out the app. Um, everything you're doing is is contained, although you make it extensible if you want to take code somewhere. But you use models from different providers. So do you believe the future of enterprise agents belongs to these vertically integrated, slightly more closed systems, you know, Apple, Replit, or a fragmented open-source ecosystem?
Yeah. So in the early days of, uh, open source, there was this, uh, debate with the open source luminaries. Uh, they talked about that, uh, the cathedral versus the bazaar. So, the bazaar is open and messy, and it's like a free market, and everyone's doing what they want, but there's like a lot of beautiful gems that you can find. There's a lot of crap as well. There's crime, you know, things like that. The cathedral is like a top-down thing that's designed to be beautiful and all that. And so, this is like the Apple, this is like Linux, right? Uh, or the open source infrastructure in general. You know, I don't like dichotomies. I like to harmonize, right? And when I, when we started building Replit, I, uh, articulated this idea of like a cathedral made of bazaars.
[laughter]
So can you build like a beautiful structure, but the individual components, uh, of it are open source and open standards?
Okay. So if you go, uh, you know, to your Replit environment, you can open the shell that's running Linux, that's running Nix, uh, all the libraries it's installing is Python, JavaScript, all the code it's writing is based on open source languages, the database it's using, it's Postgres, the deployment environment is using containers, everything is, you can SSH into your environment, you can connect Cursor to Replit or VS Code or what have you. It is all open standards in open, uh, that source. And so enterprises that are adopting Replit don't have to worry about locking. They can take their code and walk. But what we're trying to do is create experiences and automations on top of these and apply taste. We'll make decisions for you. We use only Postgres. We don't need to use MySQL. Right now, you can use MySQL. You can tell the agent, I really want to use MySQL, and I'm bringing my database with me. Sure, you could do it. But we just apply some taste on top of these open, open standards. So I would say it's not Apple-esque. It's also not Linux. Uh, it's not Microsoft either. It's something new, I would say.
Yeah. A fascinating analogy. Going to the hardcore enterprise. How do you view the future of Agentic Foundations? Right. Are you siding with the model? You've talked about being a harmonizer, but are you siding with the model purists who think these frameworks will disappear relatively soon?
Like you're talking about the LangChains?
LangChains, the SDKs, right, which where there's some questions about how, right? Even within Google, I was hearing, you know, DeepMind,
Um, there's these factions, right, where,
the the model purists are saying, we just, you know, these SDKs are ugly. They don't work. It should be the model that rules, or, you know, or or is Replit building the the Spring Boot, right? The Java Spring Boot for for agents, right? The boring, reliable infrastructure layer that makes the agent actually deployable. I don't think this is a new debate. This is an old debate. I came up in the web JavaScript development community, and it's always this idea of like framework versus library. If you use a framework like AngularJS, it's going to lock you into a certain way of doing things, like dependency injection is very opinionated about that. Or do you use like something like like React that is like only opinionated at the at the at the component level, or even jQuery, less less opinionated. There's trade-offs, and the trade-offs are frameworks are really easy to get started with. They get you there really far. But once you start having domain-specific problems, you're gonna start fighting them. There are problems that these frameworks can't anticipate. And so over time, they'll lock you in a certain way of doing things that are inefficient and not really working for you very well. And then every company ends up growing its own homegrown framework.
Which is what you're seeing right now.
Agent in beta was using LangChain. Right now, our harness is just like this big complicated codebase that we built over time. Right now, when you go make an agent with with Replit, we use Mo, which is another one of those frameworks. Uh, it's a great framework. Uh, but a lot of the agents that are people building Replit are fairly simple. But I think as these things get more complicated, you need as general of a harness as possible.
That's true of Anthropic's agent SDK or Google's or what have you. Now, to the question, this is a new thing, which is the model taking on more of the capabilities. Let's say you're having, uh, a certain set of capabilities and you need to use frameworks and other things to get there, but then those capabilities get absorbed by the model, but you, your eyes are on the next set of capabilities. Now, there's always this push and pull between what the model can do and what you need to build on top of the model to to add value. And I think that's going to be the case forever until we get to a point where models are not making any more progress. And then and then everything is just like built on top of the models, and maybe the frameworks mature and all of that stuff. Um, but I keep coming back to this idea of like just being flexible, being zen about it, being truly in touch with the truth in reality, not being tied to code, like being able to throw away code and starting over. Um, so it's, it's unfortunately not, I can't give you a neat answer. It is very circumstance, uh, specific.
Yeah. So no winner takes all for now. In agents, they're they're essentially, I mean, in frameworks, they're essentially hacks to allow you to get to the next step. All right, so five-year horizon, moving to the close here. If we fast forward five years, does the junior developer role exist at an enterprise company, or has that entire tier of labor been absorbed by the Replit agent?
You know, I'm not, I'm not sure. Um, we talk to customers that say, we don't have to hire anymore. So in many ways, it's, it's happening. Those are small customers. Those are SMBs and startups. Like I, I was with someone today that that like already million-plus ARR in Replit, who single-handedly doesn't know how to code, never coded, and like getting to a million ARR used to be like a Series A thing, right? So in many ways, that's that's already happening. Now, we talked to other people that are saying, I'm getting so much productivity and I'm saving so much money, I'm making so much incremental money that I'm hiring more people that can use these tools and I can train them on these tools to to give me more more productivity. Uh, and so, you know, there's a company we published a story on our site that like saved like a million dollars and like poured all that million dollars to get more people, uh, that are like journalist type.
What sector was that in? Do you remember?
Uh, I think it was marketing.
Marketing. Okay. It was marketing. So they want more creative people that can be journalists and use use Replit to to do more things like that.
And so it's a, it's a complicated question. I think if, if you're not holding me to to a timeline, I would say that the population of professional developers, developers that went to computer science and trained as developers will shrink over time. Uh, and a population of vibe coders that not necessarily are just very parochially focused on coding and computer science, but have other skills, but can solve problems with software with agents, that will grow tremendously over time.
Yeah. And so the bet is, I'm assuming that, or the assumption is, is that there's just going to be this thousand flowers blooming across the economy, or or millions, or potentially billions, right, of of new jobs created because of this phenomena.
Here's an analogy. Um, think about, uh, assembly developers.
Uh, they never went away, right?
There's still developers out there. You still need someone to go hyper-optimize the code path that gets run like a trillion times at like Google Search. I'm sure there's like blocks of inline assembly that someone like handwritten, right? But for the most part, C developers, like shrank the population of of assembly developers. And then the population of Python or Java developers ate into the population of of of C developers. But at every step of those, uh, the net, uh, number of developers increased. They, the absolute number of developers increased because you just opened the opportunity to to more people. You still are dependent on those people down the stack because you still need that expertise. You need you need that that sort of, um, real understanding of what's going on at the register level, for example.
Right. Well, that's a good note to end on, some hope. And again, this has been a great conversation. Congratulations for all the traction you've had.
Thank you. Thank you. Thank you for having me. This was a lot of fun.
And special thanks to our presenting sponsor Outshift by Cisco. You can learn more about their work on the internet of agents via their Linux Foundation project, agentcy.org. This open-source project enables agents to work at scale across any vendor or framework with trusted and secure discovery, identity, access, and observability. For more stories about the AI revolution, like and subscribe to the podcast, and check out venturebeat.com to sign up for our newsletters.