Transcription
Nobody understands their own code anymore. There is code running in production now at companies we use every day that nobody can really explain. Not the engineer who shipped it, not the team that owns the service, not the CTO. The code works. It passes tests. And no human on the payroll fully understands what it does, why it does it, or what would happen if it stopped doing it.
The industry is starting to call this dark code behavior in production that nobody can really trace end to end. It's not buggy code. It's not spaghetti code and it's not technical debt. Dark code is code that was never understood by anyone at any point because it was made by AI. It was generated. It passed automated checks and it shipped. The comprehension step didn't happen. Not because someone was careless, but because the process no longer requires it to ship.
Now, this is often portrayed as a security issue. It's often portrayed as an engineering quality issue. I think we need to look past both of those. It's partly those things, but it's really an organizational capability problem. It's got regulatory exposure elements. It's got business liability elements. If you are building software right now, there is a fundamental shift in what it means to be good at your job. And at the heart of that shift is grappling with dark code, which is only going to 10x from here. You think it's a problem now, it's going to be 10x more next year. So, in this video, I'm going to lay out what's actually happening, why the obvious responses aren't as effective as they seem, and what actually works.
So, first, let's dig into the problem. Anyone who's used AI tools, this is going to feel very intuitive. We're not going to spend very long on it. We have two reasons why dark code is multiplying and they interplay to make things worse. The first reason is structural. AI generating code means that it is harder for you to understand the code because the AI wrote it in the first place and you didn't bang it out with your fingers. That is just a structural reason why it is harder to understand the code unless you're really, really disciplined with your nonfunctional requirements. Hint, hint, you should be.
The other reason why dark code is tough is because we are moving so fast. AI enables us to move fast. The pressure is to move fast. That is part of the reason we're making these trillion-dollar bets on AI as an industry is because we want to move fast. When you combine velocity with a structural reason not to understand code, comprehension starts to decouple from authorship unless you take really, really clear measures to prevent that. So that's the problem. We've all lived it. Even if all we've done is type in "please make this in lovable" and it makes something fast, that is dark code.
Why do the obvious responses to dark code not work very well? Fundamentally, the obvious responses to dark code are rooted in the idea that this is a tooling problem, and it's not a tooling problem. Let me give you some examples.
Number one, there's often a response to dark code that says we should make dark code observable. If we instrument every service in the stack, we're going to understand what's happening and that's going to give us the ability to respond rapidly and effectively. I would respond and say, you know what, you should be observable. I love telemetry, but that doesn't mean the same thing as comprehension, right? That doesn't solve your dark code problem. It just means you can measure what dark code is breaking for you in production, which is great. You should do that. It's still dark code. It doesn't solve the problem.
The second way people solve this that I think is also incorrect is people have an instinct and and here I'm especially going to call out my colleagues in engineering there's an instinct to say we can harness the agents so the dark code by definition is fine. That is partly true. It is certainly good to have guardrails, right? It is certainly good to have excellent orchestration platforms. I've made videos about it. It's super important. Having an excellent pipeline reduces your risk in enterprise production systems 100%. It's really important if you're building with an agent in 2026. But if you're adding layers to your agent pipeline, that is also not actually solving your dark code problem. That is just adding a layer. And when you try and figure out what went wrong with the dark code in production, now you have to troubleshoot multiple things. So that doesn't solve it either. It's still important. You still have to do it, just like the telemetry, but it doesn't solve the problem.
Answer number three is to say it's okay to have dark code and we're fine with it. Factory.ai is a famous example of this. We're going to see this thesis tested. I think it is unlikely that we will survive for very long with code in production if nobody understands it at all. I think there will be problems if you don't understand the code thoroughly. And I want to give Factory.ai credit here because I don't think it's that they don't understand the code. I think it's that they're extending their understanding with a hypothesis that extraordinary testing and extraordinary discipline at the eval layer proxies for that understanding in a way that is useful and allows them to learn from their code. That is a genuine hypothesis that not a lot of people are trying. And most of the people who are yoloing their code into production aren't as disciplined as Factory.ai.
So regardless of how you do it, maybe you're super disciplined, maybe you're good at your non-functional emails and that's how you're extending this. Or maybe like most orgs that I've seen, you just yolo the dark code and the PM puts stuff out there and the engineering team puts stuff out there and the marketing team puts stuff out there and you gain it a little bit, right? You know, the marketing team has to manage the website and the PM can only vibe code up to a certain stage and then engineering has to take it. But the problem with all of this yoloing is that nobody owns the sustained total package of code in production. You have problems with ownership because you have distributed authorship. And distributed authorship is a strength we want to encourage. So it's not as intuitive as saying you got to shut it off. And I know IT departments who are like, you know what, we're shutting it all off. Nobody can vibe code. Only engineers can write code. They're in trouble too. They're in trouble because they can't ship fast. So the answer is not as simple as "don't do it." You have to let people code and start to build. That's actually super important. But you have to think through what accountability looks like when everybody is now able to code. And that is a hard problem.
I think AI's strengths really mask its weaknesses here. So we've talked about some of these different responses that organizations make, right? Some of them are like, observation matters, telemetry matters, that's how we solve it. Some of them are like, agentic pipelines matter, that's how we solve it. And those are both important things. And some of them are like, you know, yolo, like for one way or another, we're going to accept that there's dark code and we'll deal with it. The issue here is that it looks more okay to yolo stuff. It looks more okay to say we'll just observe it in production. As AI gets stronger. And so, in a sense, AI's strengths really mask its weaknesses because the stronger an AI model becomes, the easier it becomes to say, you know what, it's okay. We're not going to go there. The AI will know what its code is and does, and the AI will be the one that fixes the code, and we're just going to let it be. Right?
I think the challenge with that is that it is hard to know when the AI is overconfident and when it is not. And so this is a situation where if you're like, well, how do AI-native orgs solve this problem? How does Anthropic solve this problem? How does OpenAI solve this problem? It is sort of like an amalgam of multiple approaches that I've described here without an assumption that AI is magical. Yes, the AI-native companies do not have an assumption that AI is magical. So they're going to invest heavily in eval, sort of like Factory.ai does. They're going to invest a lot in understanding their agentic pipelines. We saw that with a Claude code leak. They're going to invest a ton in telemetry and understanding how their systems work in production. And all of those don't mask the fact that they are also having individual engineers still commit PRs and still have reviews of those code and still understand the code that's going out. And in that sense, as much as AI is doing a lot of the code writing, they are still demanding a degree of legibility and comprehension from engineering teams that reflects the sophistication of the work that they're doing.
Ironically, right now, the moves that the industry is making are compounding this problem for the worse. Dark code gets worse when we lay off people and expect the others to do even more, and they don't have time to understand the code. We are in a sense creating more of a dark code problem for ourselves, the more we lay off engineers across the industry. And if you think this is just an engineering team problem, this is a company entity, board-level problem, right? Questions about SOC 2, questions about encryption at risk, things that companies become liable for are things that dark code touches. And if we don't have good answers for those answers we are held accountable to, we are going to be in trouble.
So you get the idea. We have tried different things. We've tried observability. We've tried telemetry. We've seen hints of how this works as the labs have opened up the kimono and showed us a little bit of how they work internally through interviews and newspaper pieces and and comments on X. But what we really need is a comprehensive approach to dark code that treats it as what it really is: an organizational capability problem. And that's what I want to lay out here in the second part of this video.
So what actually helps? If dark code is an organizational problem, what do we need to do to fix it as a team? I think there are three layers to this solution, and I want to talk about each of them separately.
Layer one is simple: Force understanding before the code exists. There are two ways people fail at this. Number one, there are a lot of teams in 2026 that persist in 2010s era of product development where they over-document before they go, and they do it for the sake of human process. I've argued against this strenuously. I've said agents are becoming more and more the heart of the development process so we can speed up. I'm going to say it again: As long as you understand what you want to build and can write it out clearly, go. But for people increasingly in 2026 who just want to go and who don't want to take time to understand, this principle is also a break. Stop. Understand what you want to build, then go make the code exist. Do you see what I mean? I'm not giving you a blank check to just go vibe code. I'm also not telling you slow down for lots of process. I'm saying do just enough to understand what you want to build in a degree of detail you can write down. That's it. That is such an organizational discipline because so many people want to fall off on one side or the other. They want to jump into process and say, "Give me a linear process. I want to have like 16 artifacts I have to make." Not because I want them, but because that gives me an assurance that I've thought about it and I don't have to get in trouble if something goes wrong. No, no, no. It's your code. You own it. You're liable. You're accountable. Make sure you understand what you want to build. That's all you have to do. That's it. That's the principle. Force understanding as a cultural primary artifact before the code exists. I call this spec-driven development. You basically write the spec out, and that makes it really clear. And here's a detail worth noting: After the December outage, Amazon actually rebuilt their coding tool Kira with exactly this concept. The tool now leads with spec-driven development, turning prompts into requirements, tasks, and task lists before the code gets generated. They learned very expensively that AI coding tools need to force comprehension before generation. When the company that learned this lesson the hardest bakes it into the product, maybe we should all learn that lesson. And by the way, if you're wondering how do specs relate to evals, I'll give you a hint: The spec becomes the eval. It's actually not that hard. If you can write out a clear spec, that is how you get an eval that you can then set the agent again. Because really, all you're doing with an eval is you're saying, "Here is the test. The test is the spec. The agent is going to keep trying to pass the test until it does." So writing a clear spec helps you to get good code and also code you can understand.
Layer two, right? Make your systems inherently self-describing. So I'm not talking about an agent self-reporting. That's not what I mean here. I dismissed that earlier in this video. I'm talking about something more structural. Context engineering is the practice of restructuring your codebase so comprehension is embedded in the code itself. That's part of what we do with context engineering. Some people think it's just getting the docs together. It's also the code. You don't want the code understanding locked in the heads of people. You want it easily and immediately legible to humans and agents alike. So think about this in three layers. I know this is a three-layer and a three-layer. Stick with me. We're not going to go too far down the rabbit hole.
If you're making a system self-describing, part one is make sure you have structural context that answers the question: Where? Every module should have a manifest that describes what it does, what it depends on, and what depends on it. Where does this code go to? Right?
Part two, have semantic context. Semantic context needs to answer the question: What? In other words, when an AI reads an interface, it needs to read semantic context that gives it the rules of engagement, not just the shape of the data. What I mean by rules of engagement are things like performance expectations, failure modes, retry semantics, behavioral contracts. Basically, how the interface is expected to behave. This is a cousin to the idea that we should have API contracts, right? In this case, what I'm saying is all interfaces need a degree of semantic context, not just API interfaces.
Layer three, you're trying to catch what the first two layers miss. What I'm suggesting is we talk about eval-driven development as having a test, and then once you pass functional and non-functional tests, the code is reviewed by a senior engineer and it ships. Now, we're running into a situation where there's so much code generation where engineers have trouble making that legible. My goal is to put a comprehension gate in front of the code as a senior engineer is trying to read it and keep an eye on the PRs that are coming across his or her desk. And I want that comprehension gate to make key questions that a senior engineer would ask immediately and obviously legible. And yes, if you're wondering, can AI do this and help? It can. Can AI discover things through that comprehension check that we pass back to the evals for better code? 100%. This is a flywheel. Ask questions that a good senior engineer is going to ask. A good senior engineer is going to ask you, "Why did you call that dependency here? Why did you structure the code so that it caches in a location unreadable by other services? How are you thinking about separation of concerns? If you're making this monolithic, you want to think through the kinds of questions your leading engineers ask and use them to create a comprehension filter that acts as a way of making the code legible." And again, it does two things. One, it helps to make the code readable and therefore accountable by people in the organization. And two, it becomes something that feeds your flywheel and makes your evals better and improves the quality of your code over time. Can you imagine improving speed and quality of code at the same time? It's the dream. We can do it. We just have to set ourselves up so we don't have dark code.
If you've got this far, I hope you realize this is not a security team's problem. This is not an observability problem. This is not an agent pipeline problem. This is an all-of-us problem. This is a problem for everybody who's building in the software space. And if you think you're not, when was the last time you vibe coded?
If you are leading an engineering org right now, the question you should ask yourself is not "How's our observability? How's our agentic pipelines?" Those are table stakes questions. The question you need to ask yourself is, "Do I have mechanisms that enable me to make the dark code that I'm producing legible so that I know where I'm driving?" Because otherwise, you really are driving with your headlights off, and you are liable to run into the ditch and generate problems that you are not aware you're causing from a risk perspective. And that should make you stay up at night.
If you're a founder, this is both a competitive mode and also a risk. If you're like most founders, and I'll be honest, I've seen a lot of them, they vibe code so fast and get to market so fast, and they listen to only the speed part of YC teachings, they try and sell you on something that is really a thesis and a trench code. And maybe the code isn't there, and maybe it's not high quality, and maybe it's not what you need to depend on. Honestly, that's a liability for the founder. That's a liability for the company. You can stand out so easily as a founder if you just know the code in this day and age. Just get to know it. Make sure you don't have dark code. Make it legible. Explain your trade-offs. Be transparent. You're going to stand out and like be so easy to do business with and build so much trust.
And by the way, if you're a vendor, ask for that, right? Ask for transparent code. Ask about dark code. Ask how much, if they're shipping a lot, they understand what they're shipping. That is a question vendors should be asking. And if you're just getting started in this field, if you are just out of college, if you are just getting into vibe coding and then kind of leaning into the technical track, take this as an opportunity. This whole idea of a comprehension gate, you can set up, and I've given one and set one up for you. You can set up a skill that allows you to look at a piece of code and start to understand those questions that senior engineers ask, the principal engineers ask. And that accelerates your comprehension. That makes the code less dark.
And then one last note, if you are a senior engineer, if you're a principal and you're used to reviewing by hand, I know this is a massive adjustment. I know trusting AI to give you a comprehension gate is a big step. You absolutely should feel comfortable adjusting and embroidering the prompts and the skills, the stuff I've packaged together with this video so that it feels good to you. It's a tool that needs to feel good in your hand. But I don't think you can avoid using AI to help you understand code because the expectation for volume is not going away. And so you need to figure out how you can remain accountable for the code while putting your eyes across more stuff. And I think that you need essentially lenses on the code that help you to see more, farther, clearer, so that the code is not dark to you. Because the alternative is just looking at those emailed PR reviews and autofixes from Codeex or Claude Code and saying, "Ah, it's probably fine." You don't want to be that person. That's too risky.
Look, nobody's going to slow down. Shipping before you fully understand what you've built is a problem we can fix, but it's not a problem we can fix organizationally by just choosing to go back to the way things work. This is a river we've crossed. AI is helping us speed up. No one wants to slow down. No one should slow down. And I'm not arguing that that is a good idea. I think we need to speed up. I think on average, most orgs still go way too slow. But we have to be honest that going fast imposes new kinds of requirements for understanding the code that we are shipping at speed. And that is a different class of problem than the old way of understanding code because the old way of understanding code was human-mediated, right? And one of the larger theses I've been exploring in 2026 through this video series has really been saying a lot of our human-mediated touchpoints are breaking under the speed that AI is imposing on the business. How do we start to think differently about those mechanisms? And dark code takes that kind of thinking because otherwise, it is going to balloon and become an absolute security nightmare. Don't tolerate dark code. It is an organizational choice, and you can fight it. Choose to do so.