Transcription
You keep running out of Claude or Codex or or Chatg GPT or Kimmy or whatever you want, and you don't do anything unreasonable to run out of tokens. You asked a handful of questions, and it told you to come back in 3 hours or 5 hours or next week. On one working day, my tracker recorded 3.77 billion tokens moving through my Codeex workspace. Of that, 3.59 billion were reused input, almost 96%. And look, I pushed these tools really hard. That was 143 separate codeex threads in one day. But I did not type 3.77 billion tokens, and neither did you. So if the answer fails, the retry carries most of all of that a second time, and a third time, and a fourth time, while you're telling Claude or you're telling Codeex, you got to fix this.
So this is the core idea that every single rule that I'm about to give you comes back to. The message you typed is the tiniest part of the overall call. I'm Nate B. Jones. I'm here to help you build the life you want with AI. That's what I'm going to show you how to do today. We're going to go through all 15. And by the way, if you're a beginner, this is for you. You're going to go through things you can do initially. I built a skill so you don't have to remember all of it. And if you're a little bit more technical, we are going to get to a really cool multi-agent automated solution at the end. So stay tuned for that.
Your 10th message, your 20th message, your 100th message in the chat cost many more times than your first. And nobody ever raised the price on you. It's just that every time you hit enter, the way LLMs work, the entire conversation gets wrapped up in a bow and sent again from the top. That's how LLMs pretend to have memory. Right now, watch how fast that compounds. Your first message costs exactly what you typed. That's the easy part. Your second costs what you typed plus the answer plus what you originally typed. Your 10th costs what you typed plus all the nine previous conversations you already paid for. And by message 30, the thing you actually wrote just now is a tiny, tiny rounding error. That old material, it has a name. It's called reused input. The part of every request the model has already seen.
So two things to call out. One, this is not getting fixed by the labs. The labs are not going to magically fix this tomorrow. It is up to us. The labs aren't going to fix it because frankly, they have an incentive to get us using the product, and if we're using the product, that's great up to a point, right? If they run out of compute, they're going to sort of rein it back, but fundamentally the labs want us to use their tokens. So it's up to us to make the most of the limits they give us. AI, it's like this desk, and there's a belief going around that this is going to fix itself right now, and that our desks can clean themselves, and the models are going to get bigger context windows, and they're going to run longer, and have more agents getting more tools, and soon you can point the thing at your work, and it's just going to run, right? We, in other words, we believe we're going to be able to have a messy desk, and it's going to be fine. But if I take all the stuff off of my shelves and put it on the desk, I'm not going to be very productive. I'm going to be overloaded and stressed, and it's not going to work. It just doesn't work that way. I have to pick one Lego set for my desk. The more capable the tool, the more tools you hand it, the more material it puts in, and the faster you hit the wall. So, ironically, more capability may end up giving you more cleanup problems. And I talk about all three in this video, but none of it really takes the responsibility of organizing your AI desk for you. And reducing your token consumption is not what any of these companies is graded on. It's your desk. You have to own it. Is it fun to clean your desk? It can be kind of boring, right? It's almost always the boring stuff though that gives you the superpowers.
So, level one is for everybody. It's how you keep your desk clean. Nine habits. Nothing that you need to install will work on any AI. And some of it is advice we've known for a while. And I'll tell you when it is, but guess what? Old advice may still be something you need to hear if it's not something you're following right now. Then we're going to get to level two. Level two is kind of like hiring someone to keep your desk clean. I built a skill. It's called token saver. It installs into codec. It installs into cloud code, and it does most of level one for you while you keep working normally, which is kind of handy. Level three is stopping the mess before it ever gets to your desk. And that's a software piece that I've been building around my Ringer multi-agent framework. It's the most powerful option by far. And I will tell you exactly where the edges are and how it works when you are ready at the end of the video.
Level one, clean your own desk. Rule number one, you've got to edit your mistakes. So, if you're writing something in AI and you're like, "Oh man, I didn't mean to write that. I had a typo. I asked the wrong thing." Do not say that was wrong in the next chat. Just edit it, which you can do, and resend the message. And you may have heard of this, but almost nobody does it. The model will give you something wrong because your request was unclear, and your instinct is to just say, "No, that's not it." Well, instead hit that little edit button and make sure that you are actually correcting the unclear request you had before and get it right.
Rule number two, ask related questions together and say how you want the answer to appear. Not new advice again, but absolutely worth a minute for you to realize. If you have multiple questions from the same document set, put them all into one query and then specify what you want at the end. What does it look like? Is it a one-pager you want? Is it 150 words? Is it just give me the bullets? Is it give me the headline? Name it and say it because that way you are reducing the ambiguity that the AI is going to spend tokens on to give you answers on. So just name what your questions are all at once. Don't just string them along as multiple questions, and name the answer you want in advance.
Rule number three, start a clean task when the job changes. And this is, this is a huge win when people seem to resist this. I know, in fact, a lot of people who believe they are in romantic relationships with AI because they didn't bother to do this. Long conversations are really good while you're focused on the same problem. But they're really terrible for trying to deal with token usage when you're trying to get specific questions answered. And we get away with this more now because models are smarter, and so you can disambiguate more in that long context window, but it's really token-heavy, and you hit your usage limits faster. And this drove the biggest measured change when I was testing all of these out on my own codeex and claude installs. When you are carrying a working conversation, you carry so many reused tokens with you. Like you can carry 50,000, 100,000, a million, or as I've been saying, if you're getting into the billions, it's hundreds of millions of tokens with you. Start from scratch. Now, a clean task doesn't actually report zero every time because, of course, Codeex and Claude will still send their own starting instructions, but what it does is it stops the old conversation from writing along. And you're not throwing the old task away. You can keep it. Just stop making the next job that you, that you want it to do, carry that task.
Rule four is related. Carry the answer that you care about, not the argument. Let's say you have a multi-step process. You have a research piece. You want to have a document writing piece. This was big for me. Make sure that you separate out the stages of the job. And that when you have an artifact that's produced after a stage, like a research report, that's what you carry forward into the next step. So that you only have that to give to your AI to start. You don't have 10 million, a 100 million tokens of research that you're handing, much of which is irrelevant because you had to guide the research along the way. Be precise. You don't need to stack your bad first draft and your three rounds of criticism and the sources you rejected and the models reasoning all into the next step. You can actually just take the result you got and move on to the next. Keep your desk clean.
Rule five, ask for only the answer you need. Input gets a lot of attention because it gets so big. But output costs you twice. It costs you once when it's written very expensively, and again when it becomes part of the input on the next turn, and again when there's a turn after that, and again when there's a turn after that. You keep getting billed on it. So remember when, when we first got like the capability to write a lot from AI, I saw a lot of people who would go tell deep research to write them 50-page papers. But what I see a lot more of now is people saying, "Can I get it in 50 words?" "Can you please write it precisely in a way that I can understand it and give me what matters?" We need more of that because when you do that, you're saving not just on the output tokens, you're saving on every single response that comes after that. And so ask exactly for what you need. If you need a paragraph, ask for that. If you need JSON, ask for that. If you need five bullets, get five bullets.
Rule six is really simple, and it's not something people understand a lot. You want to search the file yourself whenever you can, and don't make the model search the file. Yes, the model can search the file now. It's a great convenience. It's also a massive token burner. You want to be in a position to say, "Hey, I searched it. I found these things. This is what you should focus on. This is what I'm including in this snippet for you. You don't need to go read this entire file."
Rule seven is related. You want to send the lightest useful form of the source. In other words, if the words matter and the layout doesn't, just convert it to text. I've talked about this before. You don't have to send a PDF just because the source came to you in a PDF. Convert it to markdown. Convert it to text and just paste the text in. It's so much more efficient. Don't just be lazy and say, "I can throw a PDF and 18 screenshots in and it's all fine and it will sort it." Because as tempting as it is, because it does sort it out, it will just eat your token bill so, so fast. Make sure that you take the time to actually get your sources in order. Clean your desk. It's a common theme.
Rule nine ties into those of you who have OpenBrain. Keep your answers around somewhere you can find them. And like, if you have OpenBrain, it may be in your OpenBrain database, or you may have your own database that you use. But really, really important, if you are working on a particular problem set and you're working around the edges of it, you have multiple conversations, the more you keep information about that in a place you can look it up easily, the more the AI is going to be able to look it up and not have to go recreate it and not have to go dig for it in the sources and recalculate it. It's just going to be able to get that particular piece of data out of the database and be done with it. And that is so, so much more token efficient. So, if you're not using OpenBrain, you can check it out. I have a lot of videos on it. I can link it here. It's super easy to get started on. But make sure that you have a system that allows you to go and retrieve data for stuff that you would look up multiple times otherwise, because the cheaper you can make that, the more you're saving tokens.
Let's say you're like, "Nate, I love this. I'm lazy. I don't do this. I don't remember to do this. Please help me." That's what I built a skill for. I built a skill called token saver. One command will install it. You can get it into codecs. You can get it to claude code, wherever you do your skills, and then it will keep working with you the way you already work. And so you can just say, "Use the token saver skill for this job," and it will handle a lot of the tedious parts of level one for you. It'll search before opening large sources. It will send selected passages instead of whole files. It will run exact work as code wherever it can. It saves the version you accepted and builds your next request from that result plus your change. It keeps answers to the length that you asked for, and it stops pointless retries over and over. So, there's a lot of good things about it, and I want you to use it and find ways to continue to improve it. And that's something that I love about our community in the Slack is that we find ways to improve each other's skills. So, I'm launching this. I've tested it. My team's tested it. We love it. I'll also call out that this helps you with the next three rules I'm about to give you that are really difficult to do by hand. And so I want you to like listen to these, understand them, but realize the skill is going to help you get there.
So rule 10, load only the tools the job can use. Now, this is something where I, quite frankly, in six or eight months, expect the models to be good enough to solve this problem, but they're not reliably good enough at it today. Every tool you connect carries a description right now. What it does, when to use it, what arguments it takes. That description is model input before the model does anything at all. Now, Enthropic has published some work that makes this whole problem space very, very real. A typical setup with several tool servers connected, like say, GitHub and Slack and Sentry and Grafana, it burns roughly 55,000 tokens in tool definition before Claude does anything.
Now, we're starting on more advanced rules. This is stuff where the skill can be supported, but you also have to use your head. Sometimes starting clean on a thread, like I recommended in sort of the beginner section, it's not very practical. Let's say you're debugging a particular system, and the model needs a decision from you, and you can't restart it because it's in the middle of the task. This is an area where compaction and context editing become really important. OpenAI supports compaction for long-running work, and it carries forward the state, and it turns out it needs fewer tokens as a result, but you're depending on their native compaction capabilities. Enthropic supports context editing, which clears old tool results out, and, uh, clears thinking blocks before the next request, and that's also super helpful. And so the skill helps a bit with that. But I also need to be honest with you that this is just something you, as an advanced AI user, should be aware of. You should know where your context window is, and you should recognize that when you clear out old material, you are depending on an approximated version of the initial prompt, approximated versions of the initial responses that the model makers are using to enable you to continue work on a long-running thread. That is not perfect, but it's a whole lot better than just hitting a wall, which is what we used to do. So the takeaway for you is use the skill, understand where your context window is, and then make sure that you actually anticipate in your work the consequences of hitting that wall. Put the signal early.
The skill also helps when you want to figure out what model to use because sometimes a smaller model helps you get the job done cheaper, but you have to trade off the context, etc. So what this skill does is it looks at the, at the question, the problem you're tackling, and it comes up with at least an initial opinion on whether the model you're using is correct or not. You can obviously disagree, you can move on, you can say, "No, I want to use this model," but at least you have a first blush approximation at what a token-efficient model solution is. I, I like to say, like, if you're doing a serious task, use the absolute dumbest model that will still get the work done for you. And the more you work with AI, the more you have a feel for what that line of dumbest model is. And so the skill just kind of helps you get there. Helps you take a guess at that if you're new at that.
I hear a lot about prompt caching. That's what we're talking about with this rule. Prompt caching is really, really important if you're doing repeated work. It's especially important with API work. It's not something that I would recommend for people who are doing just initial desk work. It's not a clear your desk feature if you're a regular knowledge worker. It's something where it's an API feature. You're going to be in a position where you can cache a prompt or cache part of what you're sending. And that makes it much, much more efficient to send because you're not sending the whole message back and forth. And that's really all you need to know. And if you're someone who's already diving into this, you're like, "Yeah, Nate, I do prompt caching or at least I know what prompt caching is." And you're off to the races. And if you're someone who's like, "What is an API and what is prompt caching?" Well, by definition, you don't need it. And you can focus on all the other good habits that I just talked about to keep your desk clean and make sure that you're not running into your AI limits.
And this is where the Ringer multi-agent framework comes in. Everything so far shares a single ceiling. And that ceiling is that a skill cannot make the call it is inside of any smaller. Like, if you're in the middle of a conversation and you're using the skill, it's going to do its very best, but it's going to sort of work on things going forward. By the time the model reads the skill, the request initially has already been sent, right? The conversation, the standing instructions, the tool definitions, the hidden setup I talked about, a lot of that is in the envelope before the skill ever gets invoked. And so what I want to do is think about that initial request and how we hook into that and make that cleaner. And that's the problem I'm solving with Ringer and Level 3. And yes, it is absolutely a bit more of an advanced solution. Don't, don't be scared of it. You can absolutely do it. I'm just telling you honestly what it's actually going to take. Ringer runs locally between your AI and the model provider, and before the request goes up to the model provider, it can return an answer without a model call in some instances. It can run a fixed local recipe with no model call in some instances. It can select only the useful passages to send through in some instances, or it can forward a small request under hard limits, or you can even stop it entirely. It is not another chat window. You don't go somewhere else to use it. Instead, it is effectively an intermediary that hooks in and it helps to constrain the size of what you're sending with all of these tricks built in.
Rule number 14, the, the second to last one, you want to make sure that you are able to enforce hard limits if you're serious about token usage. So, if you want to say, "I only want to send packets of a certain size out," or "I only want to get back packets of a certain size," or "I want to make sure I have a hard limit on my call, so I'm never sending 10 million tokens," that's something you can enforce with an in-between intermediary like Ringer. It's not something you can really do without that. The other thing that's important is that Ringer allows you to take advantage of what I talked about with OpenBrain. So, OpenBrain has, oh, if it's got an answer, we can go get it. Well, Ringer can go hit OpenBrain and come back, or your database of choice, and come back and say, "We've already had an accepted answer here. We already talked about this last week. We've got this response. Is this what you mean?" And that saves you the call, right? It saves you the entire 100% of the call.
We have our desk here. The nine things that we're talking about initially are you picking up your pen and your paper and your Legos and keeping the desk clean. And then the skill is kind of like someone who comes in and cleans your desk for you every night. And then Ringer is really a magical system that keeps your desk clean for you before the desk ever gets messy. I'm always telling you to think big on this channel, and I don't want limits to be the thing that holds you back. And so this video is all about making sure that you can literally 10x the value of your tokens and get where you want to go. I'm saying 10x for a reason. And I was able to audit down all of the tokens that I've been using and say, "Where are they actually going? What am I wasting? And how do I make sure that I'm putting my tokens toward their maximum value so I'm not wasting the dollars I'm spending on subscriptions?"
So, better tools are coming, and they will get better, but you're still going to have to keep your desk clean. And everything is linked below. The 15 rules are written up in full. I have the skill for you over on Substack, and Ringer is there. And if you want a whole introduction to Ringer, I have a whole video on that. And if you turned out to be the kind of person that wants Ringer, you can get that started. If you've tried any of the tools I mentioned, if you've tried the skill, if you've tried Ringer, let me know below. If you're just trying it for the first time, let me know below, too. Let me know what your experience is. And if you know someone who's running out of AI, share this video with them. We don't want them to run out of AI.