Transcription
ChatBT starts out smart, and then somewhere around message 15 or 20, it gets dumb. It forgets your instructions. It contradicts itself, and it even makes stuff up. But here's what nobody tells you: There's a pattern. The same thing happens every time, right before your AI falls apart. I'm going to show you the warning signs to look for, as well as the four tactics that can stop this from happening. Let's get into it.
So, what do I mean when I talk about the AI getting dumber? Well, we can think about it as a whiteboard. Imagine the AI's memory, its brain, is the size of a whiteboard. And every piece of information we add to that whiteboard, as it fills up, the AI gets dumber. So, here we have maybe five messages going back and forth between you and the AI, and 25% of that whiteboard is filled. Now, when I say messages, it includes what you give the AI, what the AI gives back to you, any files you attach to it, and also the AI's thinking process. All of that is information that's being added to this whiteboard. And as the whiteboard gets fuller, so say we get above the 25% mark, around the 60% mark, the AI's intelligence is still effective, and it will still follow your instructions, but it's slowly starting to degrade. And once you get past the 60% mark, and we're getting around the 95% mark, the AI's ability to be effective and follow your instructions degrades rapidly.
Now, the size of this whiteboard and the AI's ability to use information on it varies by the model you're using. And most of you are using likely one of three options: Either you're using ChatBT, you're using Claude, or you're using Gemini. I'm also assuming that you're using most of these tools in the UI itself. And each of them have different size memories, brains, and/or, in our analogy, whiteboards. So, either it's going to have 60,000 tokens, 200,000 tokens, or a million tokens. So, what is a token? A token, simply put, is part of a word. And a good equivalent to understand how big these actually are is: 60,000 tokens equates to around half of a novel, say 120 pages. 200,000 tokens equates to around a novel and a half. And then a million tokens equates to 7 to eight novels. And with ChatBT, they do say that they actually have a 400,000 token limit, which would be around 3ish novels. But what you get access to inside of the UI itself, that's going to be around 60k. At least that's what users say they're getting right now.
So, this is the size of the whiteboard. Quick pause in your regular programming. This video is brought to you by me, as always. So, two quick things. First off, below is a 30-day AI Insight Series, completely free. You'll get 30 insights in your inbox so you can apply AI to your business and your work. The second thing is, if you'd like to work with me, below are a series of offerings to see if there's a good fit between the two of us, such as a private AI community of business owners and leaders, or one-on-one coaching. With that being said, let's get back into the video.
And again, the goal of this entire thing is to utilize as much intelligence as possible before it starts to degrade. And that's what this chart represents. So here on the vertical axis, we have the performance of the AI. On the horizontal axis, we have the context usage. So that's basically the brain space that's getting filled up with the AI. And then these lines here show the intelligence degrading over time. So, if you do nothing and you just use the AI as is, the cut-off point is going to be probably around 50% of the AI's memory space, and the intelligence will start to degrade after that point. So, the AI gets dumber. And if you use the tactics that I'm going to walk you through today, there's a chance that we can extend the ability of that AI to follow your instructions and keep the intelligence over a longer period of time.
And before we actually start using those tactics, we need to first understand the warning signs of when we're approaching the degradation of the AI's intelligence and its ability to follow your instructions. So, there are a few different warning signs you can see here that I'll walk you through. We first have the instructions getting ignored. The AI contradicts itself. It starts to forget facts you initially gave it. And then also, there's an automatic compaction that occurs for different types of models, which I'll show you later. We'll walk through each one of these.
Now, the first one here is when the instructions start to fade in the AI's memory. And in this case, you'll eventually have to start repeating yourself over and over to get it to follow those instructions. As an example, say in this first message you give to the AI, you say, "I'm going to ask you a series of questions, and I want you to respond in under 200 words, so I don't have to read really long responses." And this will work for one back and forth, maybe five or 10 or 15 back and forth with the AI. Well, you start to notice maybe around 16 or 20 messages, the AI is giving you back 600-word essays instead of the maximum count of 200 words. This is happening because we've filled up the AI's memory and it's forgotten the initial instructions we gave it. That's the first signal.
The next signal is when the AI contradicts itself. So, maybe at the beginning of a conversation, we've asked the AI how to specifically structure a presentation for a potential client when we're presenting our offering to them, and it initially stated that we should lead with the ROI. So, what's the return on investment for the offering we're giving to this potential client? But then we do a bunch of back and forth with this AI, having a long-running conversation, and we ask a similar question, or we have it summarize the presentation and create the slides for us, and it comes back to us and says it's actually important that we put the ROI at the end of the presentation and instead we build context first. Now, we know because we just had the conversation with the AI that this actually contradicts what it initially stated. And this again is a sign that we've filled the AI's memory and it's starting to get dumber. In this case, that's our second warning.
Our third warning is when facts start to disappear. So, this is when you're having a long-running conversation with an AI and you give it a series of statistics or facts to remember in the beginning, but over time it starts to either hallucinate those facts or completely ignore them in their entirety. As an example, say we're having a conversation with an AI about a project, and at the very beginning of the conversation, we state that we want the price, we want to price this project at $9,000. But near the end of the conversation, when the AI is drafting that proposal, it changes the price to $6,500. If nothing happened in between that should have changed that price, it likely what happened is it just forgot the fact that you wanted to price it at 9K in the beginning. This is the third warning sign.
And the fourth one is very explicit and obvious to see. So, if you're using Claude specifically, it's going to have an automatic compact happen in the conversation. What do I mean by compact? Well, what happens is, in the background, Claude is automatically assessing how long the conversation is. Once the conversation gets to around 90% or 95%, the AI is going to automatically summarize this entire conversation. It's going to insert a new AI in that same thread. It's going to pass it that summary, and it's going to continue that conversation with you. You'll know that this is happening if you read inside the section where the AI is thinking, it says, "I'm organizing my thoughts," or it'll say, "I'm compacting the conversation." It'll make it very clear. Now, this can be useful for some people that are using Claude. But the issue I have with this is we have no say in what's being summarized above. It could be summarizing things generically, and the conversation gets worse as we go on. But note, this is just for Claude today, but I guarantee that ChatBT and Gemini will follow suit and add this feature as well, where they have an automatic summarization occurring once you get to a certain part of filling up the AI's memory. And that's our fourth warning sign.
Now, one tactic I want to share with you before we actually get into the ways that we can mitigate this is knowing actually how many tokens you're using. Sadly, all the tools that you use today within the interface of the UI, such as ChatBT, Claude, and Gemini, they don't explicitly show you how much of the memory is being filled. This feature already does exist in coding tools, which means I'm sure this will be ported over to the UIs that we're all using today eventually. But in the meantime, what you can do, and this is something I commonly do with my clients and myself, is if I have a long-running conversation or a large file, I'll upload it to Google's AI Studio. Let me actually show you what that looks like.
Okay, so what we have here is Google's AI Studio. So all you have to do is go to studio.google.com. You want to then click on the playground here on the left-hand side. And then you'll see something like this. All you need to do is drag and drop a file into the chat box down here, or you can select the button here to upload the file directly. I'll drag and drop a file to show you the context window that's being shown. So I drag and drop it here. You can see that box pops up. Now, this file, what's going to happen is it's loading the file into it. And you can see immediately it shows me that this is 5,735 tokens. Now, this is useful to know because remember in the slide that I showed you previously, we have a good idea of how many tokens each AI can take. So, if I know that I'm giving the AI 5,000 tokens, any of these can handle that easily. So, as a rule of thumb, I recommend to use Google's AI Studio to best understand how many tokens you're using, either in the entire conversation you've had already. So, you can copy and paste the entire conversation into AI Studio to understand how many tokens you've used so far and how much of the memory of the AI you've already filled. Or, if you just have a large file like a PDF or something else and you want to see how big it is, you can drop that in there to see how many tokens it is. So that's just a good little trick I recommend everybody take advantage of when trying to mitigate the AI getting dumber too quickly.
Now onto our four primary tactics of how we can actually extend the intelligence of these AIs. So, the first primary tactic is going to be the handoff approach. So, this is basically taking the previous point that I made around Claude and how it does an automatic summary. We're going to take that idea, we're going to apply it to our own situation so we can make very custom-tailored handoffs. And this is what the flow looks like. So, say that you're having a conversation with an AI over here. And in that conversation, you start to either see two things: One is you've copy-pasted the entire conversation into AI Studio and you realize that we're getting around that 60% point and it's time to move on to a new, fresh conversation. Or, you start to see the AI's intelligence degrading and getting dumber. Either way, we know it's time to start a new conversation.
So, the first thing you do is you open up a new tab inside of your browser and you start a fresh conversation with a new AI. You then go back to your other conversation that's starting to degrade. You want to ask the AI to summarize the entire conversation, answering these four questions: What have we covered so far? The most important points. What have we, what big decisions have we made throughout this conversation? Where have we left off based off of all the to-dos we have here? So, we probably have done some things. We still have a lot of things left. I want you to let us know where we are sitting in that to-do list. And then finally, I want you to give a very specific ask to the next AI on what it needs to do next. By giving it this ask, either through a dictation or typing it out, the AI will give you a very specific, tailored summary for the things that you care about in that context. You then copy and paste this summary into that new chat that you created in that new tab and you hit enter. That next AI will pick up where the other AI dropped off. So, what we're doing here is we're basically refreshing the AI's memory and getting a fresh start with a new AI and a new memory. And this is our first tactic: the handoff process.
The next tactic is really understanding the files that you're giving the AI. So, some files are bigger than others. And also, when you give a big file to an AI, sometimes it doesn't necessarily need that entire file. Maybe you can slice off a certain part of it just for that specific task you're giving to the AI. So, we want to be strategic with the files that we choose. So, first off, we need to understand the nature and the size of different files. So, when you're giving a Word document, a text file, or something else to the AI, these are cheap because they're smaller. And it's okay to assume that these files are smaller and they're not going to fill the AI's memory so much. Next, we have PDFs. So, this will depend on the complexity of the PDF, meaning are there a bunch of diagrams, images, and things like that, and/or the size of the PDF. If they're overly complex or overly large, we can assume that this is going to fill the AI's memory much faster. But if not, then it's okay to drop those in. I would say if the PDF itself is over 120 pages, you probably want to drop it in AI Studio just to see how many tokens it takes. But if it's anything below that, most AIs can handle that pretty easily. Next, we have images. Now, we can just assume straight off that images are going to fill up the memory faster just because of the nature of the file. And after that, we have Excel sheets. Now, this is varied by the Excel sheet. So, if it's a really complex Excel sheet that has many different tabs, many columns and rows, AI tends to, right now at least, struggle with being able to do complex calculations and models on these specific Excel sheets. So, there's one caveat here: If you have the Claude plugin for Excel set up, it may be able to do some more complex actions because of how that's built. But that's only if you have that plugin for Excel right now. And then finally, we have video files. So, if you're dropping a video into an AI, we can automatically assume that's going to fill the AI's memory very quickly. So, if you have a video that's longer than 2 to 5 minutes, it's going to fill up the entire thing, and you need to probably use Gemini since it has the biggest memory right now. So, that's understanding the token cost or how much memory we fill per file type.
Now, after we understand this, there's a few things that we can do with the files we're sharing with our AI. First, as I mentioned, we can always just check how many tokens this file fills by using Google's AI Studio. After that, I recommend, depending on the file and the task you have at hand, instead of giving the entire file to the AI, maybe you cut it up into chunks and you only give the AI what matters for that specific task. And this is beneficial for two reasons: One, we're not filling up the memory, but also, we're not distracting it with irrelevant information. And then finally, is if you have a massive Excel file and you want the AI to do something with this, there's likely one tab in this file that matters. As long as all the tabs aren't interconnected in some convoluted way, we can export that specific tab as a CSV file and upload that into the AI, and it'll likely do what you needed to do with that specific tab. And those are the three tactics of how you can kind of mitigate the file type.
Now, on to our third tactic, which is experimentation. This is all around building intuition, specifically understanding how complex of a task AI can deal with today. And the way that we're going to do this is through the learning cycle. So, the first thing is we're going to drop a task into AI. And these tasks can be complex because we want to push the capabilities of what AI can do today. So, we drop this in. We're going to observe the results. One of two things happen: Either the AI surprises us and it achieves it flawlessly. So, that then updates our intuition of understanding what AI can do today based off the advancements these labs are pushing. And the second thing is the AI fails. If it fails, then what we need to do next is we need to adjust, adjust the complexity of the task. So, we need to take that task we initially dropped in there. We need to break it into smaller tasks and have multiple AIs take on different elements of that task. So, maybe we open up three tabs with ChatBT, and we have one tab working on one part of the task, the second tab on the second part of the task, and the third tab on the third part of the task. And running through this cycle, we're building intuition of what's possible with AI today. And we're going to then update our understanding of the types of tasks we can give off to AI right now. Now, that's our third tactic: experimentation.
Our fourth tactic is strategic summaries. Now, there's two important caveats for this. So, the first caveat is this is mainly for people using ChatBT and Gemini. The second caveat is this is for the people that are unwilling to use the handoff strategy. So, if you for some reason feel like you have to stay in the same conversation thread and you're unwilling to summarize it and go to a new conversation, this tactic's for you. So, what we're going to do in this tactic is we're going to do the same thing that Claude does reactively. We're going to do it proactively. So, all these dots represent an ongoing conversation with AI. And what we're going to do is every 5 to 10 exchanges with the AI, we're going to proactively ask the AI to summarize what we've done so far and then continue working in that conversation. So, if we decide to do this every five exchanges, I'm probably assuming that you're having really long exchanges back and forth with the AI, meaning you give it a file, it gives you a long response, you give it a file, it gives you a long response. We're filling up the AI's memory much quicker, which means we need to do this refresh sooner. If we're doing it every 10 to 15 exchanges, that probably means that you're having short back and forth with the AI. Gives you a short question, you give it a short answer. So, the AI's memory is filling up slower. And it's important to note when we do this refresh here, we want to ask the AI the same things we asked it in the handoff. We need to ask: What are the primary things we've achieved so far? What are the key decisions we've made throughout this conversation? Where are we leaving off based off of all the to-dos we have for this overall task? And what specific steps are next after this? And then I would also ask for it to make sure that that summary is very brief because we don't want to unnecessarily fill up the memory with a bunch of jargon that's not needed. Once we have this summary, we're then going to ask the AI to continue the conversation. We then do it again after 5 to 10 exchanges, maybe 15. Then we keep doing this. And the reason we keep doing this is we're going to extend the runway. Without this tactic, the AI's intelligence would drop off much faster. With this tactic, we can extend its intelligence. It'll eventually still drop off, but we can extend its intelligence over time. And again, this is only for the people that insist that they stay in the same thread.
And those are four tactics. So, as a quick recap, the first one is the handoff summary. So, summarizing that conversation, opening up a new tab with a new AI with a new memory, and giving in that new handoff summary. We're starting fresh. The second one is strategic file choice and parts of files. So, first, we need to understand which files are cheap and which ones are expensive. And then we need to know if the entire file is needed or if we can cut off part of the file and give it to the AI so we're not filling its memory with irrelevant information. Then we want to work on experimentation so we can build our intuition and our understanding of how complex tasks AI can take on. And if the task is too complex for the AI to do, we want to break it up into subtasks and have different AIs take it on in different tabs. And then finally, we have strategic summaries. So, this is for people that are using Chat and Gemini and insist on using the same conversation without starting a new one. And that's when we can do, we can extend kind of that runway over time by having the AI write brief summaries throughout the overall thread to refresh its memory on what it needs to refer to and understand. And that's it.
So, as a reminder, two quick things. First off, below is a 30-day AI Insight Series, completely free. You'll get 30 insights in your inbox so you can apply AI to your business and your work. The second thing is, if you'd like to work with me, below are a series of offerings to see if there's a good fit between the two of us. Now, you know how to stop AI from getting dumb. But here's a much bigger problem: Your smartest employee, everything that makes them great is stuck in their head. And if they leave, that knowledge leaves with them. I made a video that shows you how to pull all of that knowledge out of their head and spread it across your entire team using AI. It's right here. So, go ahead and check out that video. Click that video, and I'll see you next time.