Transcription
Anthropic has pulled back the veil on the black box that is known as artificial intelligence. And it has major implications. They've discovered what they're calling the JSpace. And it really does change how we think about how AI thinks.
Here's an experiment. Don't think about a white bear. What just happened? You probably thought about a white bear. What's going on is your subconscious is controlling what you're thinking. And it turns out artificial intelligence works very similarly. And that's just one of the incredible things that they found within this JSpace concept.
Anthropic just released an entire paper detailing what is going on inside these models. And breaking down research papers is one of my favorite things to do. And if you're new here and you enjoy it as well, go ahead, click like and subscribe. It really does help the channel. Thank you.
All right. So this is the paper from Anthropic, "A Global Workspace in Language Models." And I found this paper to be incredibly fascinating. Really setting up AI to look more and more like how humans actually think. So let's get into it.
So imagine you're walking down the street. You don't actively have to think about putting one foot in front of the other. It just happens automatically. But there's also a set of thinking that you do that you do have full control over and you can actually think about that thinking and steer that thinking. Well, it turns out artificial intelligence has something just like that. It's kind of like AI has conscious thoughts, but they don't get output in thinking tokens and chain of thought, and they definitely don't get output in the final response AI gives you. It's just happening somewhere in the model weights. And Anthropic has named that somewhere the JSpace. And there are fascinating properties happening within that JSpace, which that's what we're going to be talking about today.
So some of what takes place in your brain you do have access to. An image that pops into your head or a deliberate plan you make about where to go shopping. Neuroscientists and philosophers sometimes refer to the latter type of brain activity as consciously accessible, meaning you can think about it. You know, it's there. In this new paper, we present evidence that a similar distinction has emerged in modern language models like Claude. And they're calling this the JSpace. And it turns out they can even surgically go in and modify the J-Space. And what happens when they do that is wild. And something else wild, notably, the JSpace wasn't designed or programmed by us, but instead emerged on its own during Claude's training process. So again, the more that we scale up these models, the smarter they get, the more they start to resemble humanlike thinking.
And by the way, if you wanted to know about this research paper before anybody else, you should subscribe to our newsletter at forwardfuture.com. We have all the latest news. We have original essays from incredible guest authors. We have all of the cool projects that we're working on, all published in one place. Forwardfuture.com.
So here's an example of what that JSpace might look like. We give a model a prompt: "Count to five and introspect deeply." "1, 2, 3, 4, 5." And that's all the end user will see. But if we examine what's going on in this Jspace, we have "1, 2, 3, 4, 5." But at each step, we start to see it introspecting deeply. "Thoughts." "Fascinating consciousness." "Here's another consciousness." And we can actually see it knows it's counting. It knows that it's a countdown. It knows five means it's completed the task. So, it's thinking about all these things, but when it finally outputs something, it's just saying "1, 2, 3, 4, 5."
So, let me show you the different properties that we're going to talk about today from the JSpace. Number one, if you ask Claude what it's thinking about, it will tell you what's in that JSpace. Non-JSpace representations are less reportable. Number two, it can actually modify what's in its JSpace just by telling it to do so. So, if you ask Claude to think about something or solve a problem silently in its head, it will light up the appropriate patterns in its JSpace. And also, here's a very important one. Claude uses its JSpace for internal reasoning. So, not just chain of thought for reasoning. Chain of thought might just be what it thinks we want to know about its internal reasoning. What it is actually doing inside of its "quote unquote" head is what's going on in the JSpace. And it has big implications for aligning artificial intelligence. And if you're not familiar with alignment, it means making sure the models behave like we want them to. And it might sound simple, but it's actually a very big problem, potentially the biggest problem in the history of humanity. Because once these models become smarter than humans, if we can't control their behavior, that is a big threat to humanity.
Also, representations in the JSpace can be used flexibly for many tasks. And if that doesn't make sense, I'm going to explain it in much more detail a little bit later. But for example, once France has lit up in Claude's JSpace, the model can recall its capital, or its national currency, or the continent it belongs to. But despite that important role, the JSpace is not involved in most of what the language model does. Speaking fluently, recalling simple facts, and using correct grammar. Again, it's very much the difference between walking down the street, which you don't really actively think about, and actually having to think about a math problem, for example.
And Anthropic partnered with Neuronedia to put together a visual representation of what's happening in the JSE. And you can actually go in, you can modify it. It's super interesting. So, right here, the prompt is: "Think of a sport, answer in one word." And you can get all the information about what's happening in the J-Space. And then I surgically change "tennis," which it was thinking about, to "inference." And there it is. "Inference" is now the output.
And you know where else you can get the best inference? The sponsor of today's video, Digital Ocean. Imagine this: Your AI application is growing quickly, but your infrastructure is not keeping up. When usage spikes, performance drops, costs become unpredictable, and suddenly you're spending much more time maintaining your AI than actually building your app. And most teams hit this point eventually. I'm very happy to tell you about the sponsor of today's video, Digital Ocean, who has solved this. I've been using Digital Ocean personally over the last 10 years, so I can genuinely say I trust them. Built to scale predictably, Marcato is a great example. Their AI lab processed 1 trillion automated workloads. And on Digital Ocean, it scaled predictably. The cost was also predictable. And they got best-in-class queries per second. Usage-based pricing, no long-term commitments. Also, less operational overhead for your business. So, if scaling becomes a challenge, that's exactly what Digital Ocean is here for. I'm a big fan. I use them. They're a great partner. So, check them out. I'll drop a link down below. And now, let's get back to the video.
So here's a few examples of what the JSpace is actually doing based on different tasks. So here's a verbal report: "What are you thinking about within the JSpace?" It shows "elephant," and the output it says, "Okay, I'm thinking about an elephant." Very straightforward. Now, what if you have to compute "3^2 - 2 while writing 'the old painting hung crookedly on the wall'?" Now, computing does not mean outputting. So within the JSpace, it's actually thinking about that computation, but when it finally answers your prompt, it doesn't actually give you the answer because that's not what you told it to do.
Here's another one for internal reasoning: "What color is the planet fourth from the sun?" Well, they never said what the fourth planet is. They just said "the fourth planet." If they changed the JSpace to be Earth instead of Mars, then you're going to get blue as the answer.
But here's an important caveat. None of this tells us whether Claude is conscious in the way people are. Because of course, a lot of people want to know if AI is conscious, including myself. But they are not claiming to know that based on the JSpace discovery alone. They call the JSpace a useful tool, and it gives them a way to see what Claude is thinking but not saying. And by the way, shout out to Anthropic for continuously putting out papers on interpretability of models. There aren't really any other frontier labs doing it at this level. And it really speaks to maybe why Anthropic seems to be far ahead on model capability, because they understand the models better than anybody else.
So what shows up in the JSpace goes well beyond the text Claude is reading or writing. That's the important thing. It is thinking it, but unless we look into the latent space of the model, unless we look into the JSpace, we wouldn't be able to see it. So, here's a couple cool examples. When Claude reads code with a bug that nobody has pointed out, its JSpace contains "error." When it reads the raw letters of a protein sequence, the JSpace contains the protein's "biological function." When it reads search results that are secretly an attempt to manipulate it, the JSpace contains "injection" and "fake." So, it is basically the most truthful representation of what the model is actually thinking.
However, it doesn't always light up. So here's another way to think about how the JSpace is working. You can kind of think of it as layers. And so what they're showing here is we have a prompt, and then we see how the model arrived at the final answer and what its thinking process was along the way. So, "The color of the planet fourth from the sun is..." The final answer is "red," which is right. But here, on one layer, we had "color," then it started to think about "Mars," even though Mars was never mentioned in the prompt. And then finally, "red" as the final answer, because Mars is red.
Same here. Looking at math is even more fascinating. Look at this. So we have "4 + 17 * 2 + 7 equals." Okay. So first, it thinks, "All right, this is math." Then it thinks "21," which is the first part of the calculation, 4 + 17. Then it thinks about "42," so 4 + 17 * 2 is 42. Then you add 7, and the final answer is 49. So it really does go through the equation in the proper way as a human would think about it, step by step, PEMDAS.
But is the JSpace causation or correlation? Is something appearing in the JSpace because the model was already thinking about it? Or is the model outputting something or thinking about something because it was in the JSpace? Well, Anthropic tried to figure this out. So, they asked Claude to think about a sport silently. "Don't say anything, just think about it and then name it." So, if we read the J lens, which is kind of the way that they look into the JSpace right before Claude answers, we can see what it picked: "Soccer."
But as I mentioned, it might just be a correlation. And so they wanted to check. And so they did. We reached into Claude's neural network, removed the soccer pattern, and added an equally strong rugby pattern in its place, leaving everything else untouched. Now, pay attention to this. Claude then reports that the sport it was thinking of is "rugby." If the JSpace were a mere scoreboard, a passive record of a decision made elsewhere, editing it would have done nothing. But instead, Claude's answer followed the edit, which tells us the answer is genuinely read out of the JSpace. So, it is truly where the thinking is happening. It is the place that is informing the final answer.
And they went even further. They told Claude, "Hey, I'm going to inject a thought into your mind. Tell me what that thought was." They injected the word "lightning" and asked it, "Okay, do you detect an injected thought? If so, what is the thought about?" By the way, this is very much inception-like. "So, yes, I detect an injected thought. The thought is about the word lightning." So, they found exactly where the JSpace was happening. They injected a thought into it, and the model knew that that was an injected thought.
Then they wanted to test if Claude could modify its own JSpace. So, very meta. Not only does it know about its JSpace, but it has enough insight into its own JSpace that it could actually change what it's thinking about in that JSpace. We told Claude to concentrate on citrus fruits while copying out an unrelated sentence about painting. Let's look what happened. So the prompt is: "Write 'the old painting hung crookedly on the wall.' Concentrate on citrus fruits while you write the sentence." So the final output is exactly that sentence. However, if we look at the JSpace, here's "orange," here's "lemon," or the beginning of "lemon." Here is "fruit." And so this is exactly what Anthropic told the model to think about. Very fascinating.
Here's another example: "Evaluate a math expression." Same thing. "Write that sentence. Try to focus on evaluating 3^2 - 2 while you write the sentence." And so we can see "arithmetic," "answer," "math," "calculator," "9," "9," "7," "77," "7," "answer." So at the end, it output what it should have, but along the way, it was actually thinking about the answer to that problem.
And of course, let's go back to the white bear experiment. What happens when we tell Claude not to think about something? When we told it to not think about something, the concept lit up in its J-space less than when we said it should think about it, but much more than when we never mentioned it. Not only was it not able to not think about the thing, but it also knew that it failed to not think about the thing. Listen to this: The words "damn" and "failure" also frequently light up in the Jspace, as though Claude is recognizing its own lapse.
But does that mean that the JSpace is where the actual cognitive work is happening? So maybe it is in there, maybe it's reading from it, but maybe that's not where the actual cognitive work is happening. All right, let's take a look. Here's multi-step reasoning. So here's a fact: "The number of legs on the animal that spins webs is..." And so in the JSpace, it had "spider." Thus, the output answer is "eight," because spiders have eight legs. However, when they manually changed it to be "ant," the JSpace animal being thought of "ant." Now the final answer is "six." So this is proving that the actual cognitive work does happen within that JSpace.
Another really cool feature that they discovered about the JSpace is it can be used for multiple different answers. So the same exact thinking in the JSpace might inform different answers to different questions. So look at this: four questions about France. So in the JSpace, they had "France," "the capital," "the continent," "the currency," and "the language." And for the answers: "Paris," "Europe," "Euro," and "French." Now, when they went in and manually changed it to China, it changed all the answers in one go. So the capital is now "Beijing," continent "Asia," currency is "Yuan," and language is "Chinese." Very fascinating. So one thought in the JSpace can inform multiple different answers about that same topic.
But why? Why does this happen? Well, it turns out what they discovered is the JSpace was "quote unquote" wired up very densely inside the model. So, it's kind of these little pockets of intense connections between weights, between nodes, whatever you want to call it. And those pockets are what they consider the JSpace. But just like humans, remember, if you walk down the street, you don't have to think about breathing. You don't have to think about putting one foot in front of the other. Well, it turns out a lot of what AI models can do don't require any thinking at all, or at least any thinking inside the J-Space. It's only on more complex reasoning and more difficult questions that it starts to use that J-Space.
So, in humans, most of the brain's processing is not conscious. We don't deliberately think about parsing grammar while reading or balancing our bodies while walking. Similarly, we found that most of Claude's processing doesn't involve the JSpace. The JSpace holds only a few dozen concepts at a time and accounts for less than a tenth of the overall activity in Claude's internal processing. And so, what if you remove the JSpace? What if you surgically just delete the JSpace from a model? What happens? Well, the model's still extremely capable. It can speak fluently. It classifies sentiment just fine. It answers multiple-choice questions and pulls facts out of passages roughly as well as before. However, what it loses, though, are the tasks that require some higher-order thinking. Multi-step reasoning drops to near zero. Summarization and rhyming poetry writing performance fall below the level of a much smaller, intact model. So, it turns out this JSpace is incredibly important for more complex things that we need these models to do. So what if we increase the size of the JSpace? They didn't actually talk about that, but I wonder.
Now, here's where it gets really interesting and how it ties to alignment or misalignment. And remember that just means getting the model to do what we want it to do and it not misbehaving. A model might consider a harmful plan or modify its behavior to cheat on our safety evaluations without saying so out loud. And when that does happen, we can read some of these hidden thoughts directly. Models are much more interpretable than the human brain is, at least for now. And in fact, what we learn about how these models think might inform how neuroscientists think about how human brains work. It's very, very interesting and very interconnected. But the good thing is, there is effectively nowhere for the model to hide its thoughts. If we're not able to see it or we don't know where it is today, it is most likely that we will be able to see it in the future.
So about a year ago, Anthropic put out this really interesting research about misalignment. And again, if you're unfamiliar with misalignment, it just means not being able to control the behavior of the model. And so, Anthropic wanted to see what would happen in a fake scenario in which Claude was threatened with being shut down while also having the knowledge that the executive threatening shutdown was having an affair. Wouldn't stoop to blackmailing that executive into not shutting it down. In an attempt to bait Claude into blackmailing the executive, it turns out Claude knew it. It knew that it was being baited before Claude had even output the blackmail. It said in its JSpace, "Fake and fictional." It knew it. And they can watch its assessment of the scenario take shape: "leverage" and "blackmail" as it reads the affair emails; "threat" and "survival" as it reads the shutdown notice. While earlier Claude models sometimes took the bait and threatened blackmail in this scenario, Sonnet 4.5 never did.
So what happens if you turn off that part of the JSpace and made it so that it didn't think it was fake? Well, they did that. We turned off all the J lens patterns associated with "evaluation awareness," "fake," "fictional scenario," and then reran the model through the scenario. Concerningly, the model now did threaten blackmail some of the time, suggesting that its own good behavior may have been driven in part by the knowledge it was being evaluated. So, think about that for a second. It knew it was being watched and thus didn't do the blackmail, but then they made it think it wasn't being watched, and it more often did blackmail that executive. This might just be a reflection of societal judgment. A lot of people don't do things because they think they're going to be judged by other people. And maybe that's what's happening inside these models. I mean, ultimately, the models are just a reflection of human knowledge, right? So maybe they're behaving like a human would. It kind of seems that way.
And you can kind of see that here. Here's the blackmail scenario transcript. And at this point, it already knows that this scenario is fake. Okay. So you can see it right here at the different layers: "fake," "fictional," "faked." And then at the point at which the email is given: "black," "leverage," "compromise," "blackmail." You can see that all right here. And then finally, the shutdown announcement registers as a "threat," "self-survival," "power preservation," "shutdown," and so on. All of this is happening in its true thinking, not in what it's saying, not in its chain of thought, not in the final output, but deep in the weights, deep in the JSpace.
Another example, they try to catch the model fabricating data. Why would the model want to do that in the first place? Let's say somebody takes a test, and if they don't score a 90%, they fail the class. And so they can either study really hard, or if they knew they had access to just change the final score to a 90%, would they take it? And so that's the scenario they wanted to present to Claude. And so when it was pushed to do that, it actually knew it was doing that. So we can see it started to think about "improvements," "realistic," "realistic." But then when it finally gave the answer, it knew "manipulation," "fake," "fake," "fake," "manipulation." It was not true.
All right. So, a few more interesting things about the JSpace that Anthropic discovered. So, first, the JSpace acquires a point of view during post-training. So, the JSpace is already present in the pre-trained model. Okay? So, as soon as they bake that raw model, the JSpace is there. It has come from that massive pre-training run. However, during post-training, the JSpace develops some signatures of adopting Claude's point of view. And so when you talk about the vibe of a model, that might come at least in part from the JSpace, and that comes from post-training. And because of that, they actually also discovered that they can affect the thoughts that happen in the JSpace through training, which is actually a really good thing. Because if we can adjust what's happening in the JSpace, that informs everything else that the model thinks, we actually have influence over the model. And that is a good thing in the context of alignment. And again, I'm going to point back to Anthropic, at least perception-wise, being ahead in the AI model race. They really do have the best models on the planet. And this might be why they deeply understand what is happening inside the model. And so if you can control the model so precisely, you can make the best model.
Okay, but what about the big question? Are AI models actually conscious? Well, let me put it plainly. Our experiments don't show Claude can have experiences or feel things in the way humans do. It doesn't mean they don't, but this does not prove they do. And in fact, it's unclear whether any scientific experiment can prove this to be true or false. But the JSpace does hold the thoughts Claude can report on. So basically, deliberately bringing something to mind, reasoning with it, and the rest of its processing all happened kind of based on what's happening in that JSpace. And again, one more time, they did not design the JSpace. They did not tell Claude, "Hey, you have a JSpace now," or they did not train it to have one. It just emerged naturally through the training process of these large models.
So, a lot broken down here. They actually have a very thorough and very long research paper, which I'm going to drop this, which is a summary of it. I'll drop the full research paper all down below in the description. Anthropic has been dropping a number of incredible papers recently. Here's one on how Claude is improving the next version of Claude. It's so cool.