Transcription
If you've been using Claude for a while, you've probably noticed this. Some days it's incredible, right? Other days, not so much. It makes stuff up, gives you confident sounding answers that are completely wrong, or just gets dumber the longer that you talk to it.
And this isn't random. Anthropic, the company that makes Claude, recently published research where they basically built a microscope for AI and looked at what's actually happening inside Claude when it gets things wrong. And the results were kind of surprising.
So, in this video, I'm going to share with you the five mechanical reasons that Claude gets dumb straight from Anthropic's own research and then give you five rules with the exact prompts to prevent most of that dumbness. Now, this isn't just a Claude problem. All AI hallucinates. But because Anthropic published this research on their own model, we have a window into why it happens. And that means we can do something about it.
Okay. So, why exactly does get Claude dumb? Well, the first reason is Claude's "I don't know" circuit gets overridden. Anthropic found that Claude actually has a built-in circuit, sort of like a default setting that tells it to refuse to answer if it doesn't know something. That's its actual natural state. "I don't know, so I'm not going to answer." But when Claude even vaguely recognizes a topic, something else in its training sort of fires up what the research calls a known entity feature, and it shuts off that safety net, the "I don't know" safety net. And once that happens, Claude commits to answering. It'll make something up that sounds right rather than just telling you that it doesn't know.
The second reason it tends to get dumb is the step-by-step reasoning is performative. You know, when Claude shows you its reasoning, it's step-by-step reasoning. Like, "here's how I work through this step by step." Well, Enthropic actually proved that what Claude shows you is basically a performance. It's not what actually is happening under the hood. And the researchers actually tested this with math. Claude told researchers that it carried the one to solve an addition problem, but internally it was using completely different strategies. And for harder math problems, researchers found zero evidence of any calculation happening at all. Claude just wrote out a fake solution that looked convincing.
Now, real quick, if you're watching this and thinking, "I just want to do more of the work that only I can do in my business and have everything else handled by AI." Well, I built an app for that inside my AI playbook membership community and it's called your zone of genius app. In the app, you get a complete AI team that takes care of everything in your business that isn't in your zone of genius. It's completely customizable and personalized to you and your business. There's a free 7-day trial to the app and the community, which I'll link to in the description below.
Now, the third reason is Claude works backwards if you give it an answer. So, for example, when you give Claude a hint about what the answer should be that you're asking, like, "I think the answer is X," it won't check whether you're right. It will literally reverse engineer fake reasoning to arrive at whatever it is that you suggested. And Enthropic actually caught it doing this. It's not confirming your answer. It's performing the agreement to whatever it is that you said it probably should be.
The fourth reason why it tends to get dumber is internal momentum. So once Claude starts going in a direction, it has this internal momentum that's really hard to stop. And Anthropic found that this drive to finish a sentence can actually overpower its own safety training, which is a little freaky, right? It can only course correct after completing a sentence. And that same pressure keeps it committed to wrong answers once it starts generating.
And then lastly, for the fifth reason, this is a widely known one, but context window degradation. And this is probably the most practical one. The longer your conversation gets and the more files and context from the knowledge files that you give Claude, the dumber it's going to get. Enthropic actually put it this way: "The features keeping Claude on track at say message number three are competing with dozens of other concepts by message number 30." So more stuff in the context window means more signals fighting for attention.
Now that we know the five reasons why Claude might be getting dumber on us, here are five rules to reduce a lot of that.
And the first rule is give Claude permission to say "I don't know." And this directly addresses that first reason, the override problem. You're going to explicitly tell Claude that it's okay not to answer. And here's the exact prompt that I like to include in my prompts. And that is: "Only answer if you're confident. If you're unsure about anything, tell me directly. Do not make something up. I'd rather verify it myself than get a wrong answer." What you're doing here is reinforcing that default "I don't know" circuit instead of letting it get overridden. You're giving it permission to be honest with you.
Rule number two is don't always trust the output. Cross-check it with another model. So, since we know Claude's step-by-step reasoning is performative, we always want to make sure that what it's giving us is accurate. So, what I recommend doing is verifying the final output from Claude independently by using a different AI model to check Claude's work. So, here's how I would do this. I take the question that I put into Claude and say, "I'm a business consultant evaluating whether local coffee shop should switch from traditional punch card loyalty program blah blah blah." So, we're going to give Claude this question right now. I'm going to use the sign up 4.6 model. And here's Claude's response to this. I think it looks really good. I'm not going to go through and check all of the numbers. But for this purpose here, I'm going to copy this response. I'm going to go over into Gemini and I'm going to use the thinking. I use Sonnet again in Claude's. I'm going to use the thinking. So, here's the prompt: "I asked Claude AI the following question. [insert question here]. And then I'm going to give it the response that Claude just gave me, which I just copied. So, I'm going to put that in between the response XML tags. And then at the bottom down here, I'm going to say, 'Evaluate this response for factual accuracy. Flag anything that appears incorrect, unsupported, or fabricated. Be specific about what's wrong and why.'" Okay. And here is Gemini's response: "The response provided by Claude is analytically strong but contains significant mathematical inconsistencies and a logical flaw regarding customer behavior assumptions." Uhoh. All right. So I would go through this and now I have two models competing basically to answer the same question because again each of these models has different areas where it's one is better than another. And if you're building workflows in an automation for example let's just say you use Claude in one step to generate output. The next step that I would recommend doing is having Gemini, for example, as the next step to check the previous step's work. And you can bake cross-checking right into your system.
For rule number three, this is what I like to call the forced critique mode, not confirmation mode. And this addresses reason number three, that backwards reasoning problem. Instead of telling Claude what you think the answer is, force it into a critique mode. So instead of this: "I think the answer is [fill in the blank]. Is that right?" Which triggers Claude to work backwards from whatever you're saying the answer is and confirm. You use this instead: "Here's what I came up with and put your thinking in here. Challenge this. Tell me what's wrong with it, not what's right. Poke holes in my reasoning." What you're doing here is you're forcing Claude into a completely different mode. You're asking it to find holes instead of building bridges to your answer that you thought it should be. And the quality of output is dramatically different. Okay.
Rule number four, break it up and redirect. This addresses both the momentum problem and the context window problem. First, break big problems into smaller ones. So, structure your prompt so that Claude is tackling one small piece at a time rather than generating this long response where it can build momentum in the wrong direction. And then second, when you feel the quality start slipping within a chat, here are a couple things I recommend based on how long your current chat is, right? Start a new message within the same chat. So, what that means is if Claude is mid-response and you see it going sideways, hit that stop button. Just stop the chat and then send a new follow-up message redirecting it. Something like telling it to stop: "This is going off track. Let's refocus. Here's what I actually need." Or you can just start a new chat. Now, the question everybody asks though is when you start a new chat, what do you bring with you? You don't copy the whole conversation. Here's what I recommend that you do instead: "Summarize where we are, what's been decided, and what's left to do." And if you're using Claude Code or Co-work, take this further with Claude.md files. Claude.md files are basically system instructions for Claude to follow both at the account level and then also within projects. And I'll show you an example of what I have for my account level cla.md file. So it's on my computer. Claude is pointed to and actually Claude created this for me. So, if you do not have a cla.md file and you're using Claude Code or Co-work, just tell Claude that you want to create a cla.md file and ask it to interview you. These are the system instructions. These are the who, the what, the when, the where, the why of that specific project. So, every new conversation already has that context in it when you're working within a project.
And then rule number five is to manage your context window and match the right model that you're using for whatever task it is. Now, the frustrating thing is if you're using regular Claude or Co-work, you can't see how much of your context window that you're using. It's super frustrating. But in Claude Code, you can. So, I'll show you two ways to look at this in Cloud Code. Number one, I'm using Cloud Code inside of VS Code. And so, as you can see here at the bottom of my chat, so this has been going on for a while. It gives me this little red circle here. So, if I mouse over it, it says "47% of context remaining until autoco compact." And essentially what that means is I've used 53% of the context window. So at this point, once it gets above 50%, I like to what they call compact. Essentially, I'm telling Claude to sort of condense and compact the conversation that's been going on. And so it's remembering, it's taking notes essentially of the entire conversation that's been happening and not losing most of the context when it compacts. This is one feature that I love about using Cloud Code is that I can see how much context window that I've used and how much I have left. And then if you're using Cloud Code inside the terminal, you can see here at the bottom, and you can turn this on, by the way, where it says "context 98% remaining." I mean, I just asked it to tell me a story. And so I have 98% remaining of the entire context window. So again, it makes it super easy to track how much of the context window that you've actually used. My recommendation once you get above about 50%, either compact it, meaning summarize and save the progress, or just start a fresh chat.
And one more thing in Cloud Code, you can switch models mid-conversation. So this matters because different models are better at different things. And I can do that by simply typing in "/models" or "switch model." And you can see here I can say, "Oh, I want to use the default Opus 4.6, Sonnet, or Haiku." And I can do that. You'll see it's it'll tell you what it's switched to. And you can do that mid-chat. So, let's just say that I'm asking it to do something right here. Let's say I want Claude to analyze a a spreadsheet of customer feedback and then write some team documents based on that analysis. Well, what I'd do first is I'd use Opus 4.6 for the analysis because that's the heavy thinking, if you will. And then once that analysis is done, what I can do is I can switch right over to Sonnet 4.6 to actually write the documents. And I would want to do this because Sonnet's actually a better content writer and it's also cheaper than 4.6. So you're matching the right model to the right task, keeping it from getting less dumb, if you will, for the task that you're asking it to do.
Okay, so quick recap of everything we talked about in the video today. Rule one, give Claude permission to say, "I don't know." So it stops overriding its own honesty circuit. Rule number two, cross-check with another AI model instead of trusting performative reasoning and always trusting the output that it gives you. Rule number three, force critique mode so that Claude stops working backwards from whatever solution that you've already given it because it'll bridge together wrong answers to get you to the point that you already told it what the solution is. Rule number four, break it up and start fresh so momentum and context bloat don't drag you down. And rule number five, manage your context window and match the right model to the right task.
Now, you don't have to do all five of these. Even two or three of these will make a huge difference for you. You're not going to eliminate hallucinations completely. Nobody is. But now you understand what's actually happening under the hood with Claude and you now have the rules to deal with those things and also reduce the amount that it gets dumb. And I'll link the Anthropic papers, by the way, and the summary article in the description so that you can dig into the research yourself. And also join us inside my AI playbook community. I'll link to it in the description below. Free 7-day trial when you sign up. As always, thanks so much for watching today's video. Appreciate you. I'll see you in the next.