Transcription
OpenAI dropped a release that feels like somebody finally caught an AI midthought, and I mean that literally. You can trace a decision down to a handful of tiny components, like you are following a circuit on a motherboard. That is the vibe of this whole thing.
The project is called Circuit Sparsity, and it comes from a paper with a pretty blunt title: "Weight Sparse Transformers Have Interpretable Circuits." OpenAI also shipped an actual model on Hugging Face called `openai/circuitsparity`, plus a toolkit on GitHub called `openai/circuit_sparsity`. So this is one of those moments where it is research, and it is also tooling you can touch.
Here is the punch line up front. They trained a GPT-2 style transformer on Python code, and they forced it to learn with almost all of its wires cut. The wild part is they did not do it later. They enforced sparsity during optimization, during training, every step, the whole time.
So let us zoom into what "weight sparse transformer" means here, because the details are the story. Normally, a language model is built like a massive, tangled web. Every part talks to every other part. Millions, sometimes billions of connections firing at once. That is why people call them black boxes. Even if the answer is correct, nobody can really say which internal parts mattered and which ones were just noise.
OpenAI decided to do something that sounds almost reckless at first. They trained a model while deliberately cutting most of those connections away. Not after training. During training, step by step, every time the model updated itself, OpenAI forced it to keep only the strongest connections and delete the rest. Not weaken, not ignore, but fully zero out.
The result is extreme. In the most aggressive version, only about one out of every 1,000 connections survives. That means over 99.9% of the internal wiring is gone. And they did not stop there. They also limited how many internal parts are allowed to activate at all. Roughly only one out of every four internal signals is allowed to light up at any moment. So fewer connections, fewer active parts, much less internal chaos.
At this point, most people would expect the model to fall apart. And that is the trick. It does not. The reason it survives is how OpenAI trains it. Early on, the model starts out normal and flexible. Then, slowly over time, the allowed number of connections gets smaller and smaller. The model is forced to compress what it learned into fewer and fewer internal pieces. Whatever survives that process ends up being the most essential logic.
That setup lets OpenAI do something very revealing. They can keep performance the same while shrinking the internal machinery. And when they compare these sparse models to normal dense ones, they see something striking. For the same level of accuracy, the internal thinking machinery inside the sparse models is about 16 times smaller. In simple terms, the same behavior fits inside a much simpler internal program.
And this is where the idea of circuits comes in.
Before we jump deeper into the story, there's something I keep seeing in the comments. People asking how we managed to produce so much content so fast. Look, in 2025 alone, this channel pulled in 32 million views. That's not luck. That's not grinding harder. It's because every time a new AI breakthrough drops, we plug it straight into our workflow. Most people watch AI news and move on. We use it immediately.
So, we decided to release something we've never shared before: The 2026 AI Playbook. 1,000 prompts to dominate the AI era. This is how you go from just consuming AI content to actually using AI to build real unfair advantages for yourself. Get your proposals done in 20 minutes instead of 4 hours. Launch that side business you keep putting off. Become the person in your company who gets twice as much done in half the time. Founding member access opens soon. Join the waitlist in the description.
All right, back to the video. Instead of talking about vague features or hidden states, OpenAI defines everything very concretely. A circuit is just a small group of internal units and the exact connections between them. Each unit is tiny: one neuron, one attention channel, one read or write slot in memory. And each connection is literally a single surviving weight.
So now the question becomes, can we find the smallest possible internal circuit that still solves a task? To test that, they created 20 very simple coding challenges. Each one forces the model to choose between two possible next tokens. No open-ended answers, no creativity, just pick option A or option B.
Some examples feel almost trivial on the surface. Close a string with a single quote or a double quote. Decide whether to output a closing square bracket or a double closing square bracket based on how deeply nested a list is. Track whether a variable was created as a set or a string so the model knows whether to use `.add` or `+=` later.
Then they do something clever. They start removing internal parts of the model until performance drops. The goal is to find the smallest internal mechanism that still solves the task well enough. They do not guess. They optimize for it directly. Anything removed gets frozen to an average value, so it cannot secretly help anymore. What is left at the end is not a visualization. It is a stripped-down internal machine that actually does the job.
And this is where the title of the video starts to earn its weight. For the quote-closing task, the final circuit has 12 internal units and nine connections. That is it. Inside that tiny circuit, two units appear almost immediately. One activates whenever the model sees a quote, any quote at all. The other carries a simple signal that tells the difference between single and double quotes. Later on, another internal component takes that signal and copies it to the end of the sequence, right where the closing quote needs to appear. So the model is not guessing. It is not pattern matching loosely. It is running a tiny internal routine: detect, then classify, then copy, then output. You can follow it step by step.
The bracket-counting task looks different but just as clean. When the model sees an opening bracket, it triggers a few internal detectors. Another component looks across the whole sequence and averages those signals, which effectively turns into a sense of nesting depth. Later, one more component checks that depth and decides whether a single closing bracket is enough or whether a double one is required. That is counting, plain and simple.
Then there is the variable-type task, and this one is especially interesting. When the variable `current` is first created, the model stores a tiny internal marker that says what type it is. Later on, when the model has to choose how to modify it, another internal component retrieves that marker and uses it to pick the correct operation. So the model remembers not vaguely, not statistically. It stores something, then retrieves it later when it matters.
These circuits are small enough that you can actually read them, and that is the moment where it stops feeling abstract. You are no longer talking about outputs. You are watching internal decisions form.
Then OpenAI adds one more layer that makes this even more powerful. They introduce something called bridges. Think of bridges as translators. They let information flow between a clean, readable, sparse model and a normal dense model. You can take a specific internal signal from the sparse model, tweak it, and inject that change into a dense model. So instead of saying, "This behavior exists in a toy model," you can say, "This feature exists, and here is how it affects a full-scale system." That is a big shift. It means interpretable features do not have to stay trapped in research demos. They can be mapped onto real models.
And this is not just theory. OpenAI released an actual model. It is called `openai/circuit_sparsity`. It has 0.4 billion parameters and it is available on Hugging Face under an Apache 2.0 license. They also released the full toolkit on GitHub, including the tasks and a visual interface for exploring circuits. You can load it, run it on Python code, and know that inside, almost everything is zeroed out. What remains is the minimum machinery needed to function.
That is why this release feels different. It is not about making AI stronger. It is about making AI legible. You are not just seeing what the model says. You are seeing how it arrives there, using a handful of internal parts you can point to and name. And that is why OpenAI just caught an AI thinking, not because the model is conscious, but because for the first time at this scale, the internal process stopped being a blur and started looking like a sequence of actual decisions you can follow.
Around the same time, OpenAI dropped this Circuit Sparsity work, Axios published a piece with a headline that quietly says a lot: "OpenAI is not too big to fail. It is bigger." And that framing matters. The company sits at the center of the AI economy in a way no other lab really does. When OpenAI shifts direction, investors feel it almost instantly.
According to Axios, Sam Altman is dealing with pressure from multiple sides at once. Competition from Google is intense. Lawsuits from families continue, and there is over $1 trillion dollars in long-term spending commitments tied to infrastructure, chips, and data centers. The AI economy has wrapped itself around OpenAI's trajectory, which makes even small signals ripple outward. Axios points out that a simple suggestion of delays in Oracle-built data centers for OpenAI was enough to move tech stocks.
That is how sensitive venture capitalist and MIT research fellow Paul Kadrski described it bluntly: "OpenAI's individual role might look small at first glance, then that impression collapses once you look at how interconnected everything has become. A serious stumble would freeze parts of the ecosystem in place."
Dip Singh, former deputy national security adviser and now head of global macro research at PGIM, took it a step further. He warned that if OpenAI falters, "the foundations of the AI sector weaken fast with effects cascading through chip demand, capital spending, and financial markets." He pointed to a specific pressure point: chips. Microsoft and Meta have been buying aggressively to avoid falling behind. If OpenAI's momentum slows, that urgency fades. A drop in chip orders would hit billions in capital expenditures that have been propping up growth. Singh even floated a rough figure, suggesting that as much as half of that growth could stall. Those chips also sit behind loans as collateral, which means shifts in demand ripple into credit markets, too.
Inside OpenAI, leadership has pushed back on the idea of government backstops or special treatment. Altman has said openly that failure should remain possible. OpenAI's public messaging emphasizes confidence, strong investors, and continued progress. Still, the scrutiny remains because the stakes are real.
At the same time, OpenAI is preparing consumer-facing changes that make internal decision-making matter even more. TechCrunch reports that an "adult mode" for ChatGPT is planned for early 2026, confirmed by Fiji Simo. Access would depend on an age-prediction system that infers user age from behavior and context rather than a simple checkbox. The system is already being tested in a few countries.
This move goes beyond adult content. It opens conversations around topics currently filtered for being too sensitive, including relationships, sexuality, and mental health. That kind of feature brings legal, regulatory, and trust questions with it. Governments worldwide are tightening rules around age verification, and OpenAI clearly wants a solution that scales across regions while keeping users engaged, possibly through premium tiers.
And this is where Circuit Sparsity quietly connects back in. OpenAI is building systems where internal decisions carry real external consequences: decisions about code behavior, decisions about content boundaries, decisions that regulators, users, and investors care deeply about. In that environment, having models with clearer internal mechanisms, fewer hidden interactions, and traceable decision paths stops being academic curiosity. It starts looking like infrastructure.
Circuit Sparsity fits into that picture as a way to turn internal behavior into something compact, readable, and steerable; a way to shrink complex behavior into small machines you can point at, test, and move with intent. That context gives this release weight far beyond a single research paper.
Does readable AI move us closer to real control, or does it accelerate power in ways we still underestimate? Drop your take in the comments. I want to see where you land on this. If this breakdown helped, hit like and subscribe for more deep dives like this. Thanks for watching, and catch you in the next one.