Transcription
I'm running Claude code right now to build a full AI application, and I haven't hit a single rate limit. While everyone else is getting throttled by API caps, I'm running unlimited coding sessions on my own hardware using Claude code router.
Today, we're building an AI PDF chat application to prove that this actually works. Upload any PDF, ask questions, and get instant answers, which is all going to be running locally. And here's the meta part: the same AI model helping me code this app will be the engine powering it. The AI is basically going to build its own interface. By the end of this video, you'll learn exactly what's possible when you're not constrained by rate limits.
I've got my local model running. Claude code router is already connected. So, let's get coding. Today, we're heading straight into reaction. So, I'm going to go ahead and type `CCR code` to launch Claude code, but routing it through my local AI model that's running here in LM Studio. And I'm actually going to be using Coin 3 for both coding the application and just using AI in my application to just be able to ask questions about PDFs, right? So that's very cool, and you can see here that it's reachable, you know, on this local URL. So that's going to be very nice, and we're going to be starting here by passing the initial prompt to Claude code.
Now, this initial prompt is something that I prepared beforehand to just explain to the AI the project that I want to build and some of the technical specifications that are requirements in my eyes. So, it's always important to do this because no matter how powerful your AI system is, you do need to give it some hints as to what you actually want to build, right? So, we're going to go ahead and ask Claude code to read this file and then create a spec file with to-dos. So, you can work out this MVP. And then what I'm going to do is use Shift-Tab to switch into plan mode. And now you can actually see that my GPU is fully utilized as it's going to be generating the answer.
Now, while it's generating the answer, I want to talk a little bit about the environment that I'm using. So, you can see here that it's actually using bash commands like `ls`, but I'm on Windows, right? Well, not entirely. I'm actually using Windows Subsystem for Linux and running this specific environment in Ubuntu so that I get access to a proper Ubuntu bash shell. And this is important because these AI code agents are much more used to running commands in bash opposed to something like PowerShell. So, I would always recommend to do that for this reason. But there's another reason for this: I'm going to be using a more unleashed version of Claude code which allows me to run any command without having to approve it, which will speed up my coding process. And doing this is much safer when you're in an isolated environment like with Windows Subsystem for Linux.
You can see that now it has actually written that specification file, and you can see here that it just creates a folder structure that it's going to be working on. You know, there's different components that we need for this integration. We need an API route, and all of this looks pretty good initially. So, I'm going to be very curious to see how it's going to get started. It's very nice. And I'm going to go ahead and just allow it to create that specification file. You saw there that I already had to give approval to run that command. Um, but in this case, you know, I'm using a local AI model on an isolated environment. So, I don't really want to always give all of these different approvals, right?
So, what we're actually going to be doing is it's asking me if it wants to proceed with the application implementation. But let's go ahead and switch that more unleashed version of Claude code so it can actually speed up and not have to approve every single command that it wants to run. I'm just going to get rid of the initial prompt because the specification file is probably leading in this case now. And I'm going to go ahead and run `CCR code` but I'm going to pass `dangerously skip permissions`. And we will now get a warning. It actually warns us that it is going to run every command that it wants, but I'm okay with that because I'm in a relatively safe environment. So, it can just read that specification file and get started with the first step.
You know, what's happening now is it actually just creates the `pdf-local` folder on its own, which is now in here. And it actually just creates it, you know, a Node project from scratch. Um, which is fine. I mean, you know, I don't really mind too much. It should be fine as long as I can figure out how to actually create a proper Next.js on its own. It's a bit of an outdated version, but this obviously happens a lot with these AI agents. They might use outdated packages, and that's why as an engineer, you know, you add the value to these AI agents, right? Because I'm going to be taking this code later, then updating dependencies, making sure that everything is up to date, and I'm just using this to scaffold the initial implementation.
So, now I can see that it's finished this AI integration component, which is pretty nice. You can see here that it's doing a fetch request to that exact same endpoint that I was talking about before, right? Because this is where my AI model is running. So, that's pretty cool, and it seems to be pretty effective at getting the initial files created. So, looking good.
So, after a little while, it did create a new `package.json` file. You can see that it did add a dev command. What is a little bit weird is that this is not formatted at all, but that's not a big deal because if I save this file again in Visual Studio Code, it will auto-format it for me. So, now it seems like it's getting pretty close, and it's actually trying to build the application here inside of Claude code. But I'm actually just going to stop it there because I think at this point, I want to just take over and see how far the project actually is and whether it runs when I try to run it myself locally.
So, I'm going to go ahead and actually stop the execution here. I'm going to interrupt it by pressing Escape. Let's run the project as is locally. So, I'm going to open a new terminal, and then we are in a `pdf-local` folder. That's correct. I'm going to run `npm run dev`. Uh, and then let's see what happens. And then actually, we start to run into a 404, which could be accurate. It depends a bit on how our pages are structured. So, what I'm going to do is I'm going to just put this on the top left here. And let's see here what our project is actually containing because I don't know if we even have a root page. This is sort of our root page, it seems. So, it does seem like we have a root page, but for some reason, we cannot access it at the root URL. And I think this is an interesting one for Claude code to try and figure out.
We're just going to be running Claude code right next to our `npm run dev`. So, what I'm going to do is I'm going to say, "Next dev works, but the page localhost:3000 leads to a 404. Investigate the routing." From then, I'm going to go and drag this page in there and figure out what is going wrong.
Okay, so I can see it's taking a while. So, I wanted to take this time while the AI model is thinking to talk to you about my AI Native Engineering community. Because what you're learning today is just a tip of the iceberg of what you can learn in our community. Whether you are just getting started with your career, you're a senior engineer already, or for example, an entrepreneur trying to build AI systems, you will be able to accelerate yourself with real AI skills. So, definitely check us out. Link in the description below. And I'll grab a cup of tea and wait until all this is done.
Okay, this is an honest limitation here. I've been trying to get it to understand how to add that route file, but it just keeps getting into a loop. And this is where the reality of local AI coding comes in. When you hit limitations, you have to recognize that we're limited here. We need a powerful cloud model to actually take care of making sure the application actually works end-to-end. Because if I go ahead and actually just paste the exact same prompt that this one had, I'm just going to go ahead and do that now. Just going to go and copy this and then paste it in here, you will see that the actual cloud model that's running in the cloud will be much better at actually solving this issue. It's much faster, first of all, at reading all of my different files, and it's actually not getting confused at all immediately. It's like, bang, you know, Next.js 13 plus requires this kind of routing for the homepage, this kind of routing for API routes, and I'm immediately able to just create a directory to go ahead and fix all of that.
Okay, so I will say it took a little while, 15 to 20 minutes of playing with Claude to actually fix up the intern's work, but that's totally okay, right? That's what real AI human collaboration looks like. And now we actually have a PDF local reader application running locally, and it's just loading this Pro Git book in by default. And I can just ask questions about any page. So, for example, I can go to page two and actually see the authors on this page. I can, for example, ask, you know, who wrote this book? And automatically, the text content from this page is injected into the AI model context. And you can see, hey, you know, according to the content, this book was written by Scott and Ben. And apparently, this page is about unstaging a staged file with `git restore`. Sounds pretty complex to me. You know, I'm recording this video at the same time as reading this, so I don't have time to understand what this all actually means. So, I'm going to ask the AI assistant to summarize this page like I know nothing about Git. Let's see what it comes up with now.
So, now it says, "Hey, this page explains how to undo changes in Git, especially focusing on the new `git restore` command introduced in version 2.23.0, which is very nice." And in fact, you know, if the context allows for it, I'm able to just inject the entire document into the memory of the AI system and ask a question like, "Where can I find information about the command `git status`?" Let's see how this works. Oh, we get an error message. Well, why is that the case? If I check LM Studio, I do actually see when I scroll to the developer logs that we get a very clear error. So, it seems like we have quite a lot of tokens in this request. To be specific, we have over 200,000 tokens, but we only support around 50,000 with the current Coin 3 deployment that I have loaded in. So, clearly, that's not going to work.
All right. So, I'm going to load in this Coin 7 billion parameter model and I'm going to set the context length to 250,000 because then I can actually fit the entire book in my memory. So, now I'm going to ask the same question again: "Where in the book can I find information on `git status`?" And this time, you can actually see that the request is being accepted by LM Studio, and it's generating a response. If we check out our task manager, you can indeed see that our GPU usage is increasing substantially, but it takes a longer time to generate the response.
Okay, so this took really long. And now I finally get a response. And this response basically tells me to go to page 28. Um, if I go to page 28 here though, and I scroll down, I know this is a small UI tweaking need to make. This is actually page 22 in the book itself. So, yeah, of course, the way that the book tracks pages versus the actual page count of the PDF differs a little bit there. So, we actually probably need to go to page 28 according to, you know, the number here on the footer. So, that means we have to go a couple pages in advance here, and yeah. Oh, yeah, there we go. So, page 28, this is where `git status` is being explained. So, indeed, the answer here is quite correct, and we can even continue to page 31 to find more advanced usages of this command.
The problem here is that loading entire books into your GPU memory is just not feasible when they're hundreds of pages long. You need to cut up your books, create vector embeddings out of it. I've explained a lot of that in other videos that I created. But if you want to set up this kind of great local AI coding environment for yourself, you should check out the links in the description because first, there's a masterclass video that will walk you through the entire setup. But a more effective way of learning is to join our AI Native Engineering community where you can learn how to actually accelerate yourself with AI today. So, I hope to see you.