Transcription
What if we could take a language model and turn it into an expert on something it has never seen before? Not by teaching it, but by giving it fast access to documents, like a serious investigation into whether Lord Elron is secretly Agent Smith. And that's exactly what we will do today.
So, in this video, we're going to take a powerful language model, and we will give it access to witness reports, testimonies, and forensic evidence, coding a proper REGG pipeline from start to finish. We will turn our model into a brilliant investigator fully familiar with the case. We will learn about chunking, embeddings, vector databases, and retrieval using 10 fanfiction documents I created for this project. We will plug in Olama, Lenchain, Feice, and Tiny Quen, giving you a solid foundation for building your own expert systems. By the end of this video, you will fully understand the right process and how to make hilarious projects with your new set of skills.
And finally, this video is brought to you by Hopspot. We'll talk about them more shortly, but in the meantime, let's roll.
So, first things first, let's download our project files from GitHub. We have 10 PDF documents full of evidence. Some defending Eland and some accusing him. And if there's one document you want to read, make sure it's Sauron's testimony. I had a lot of fun writing this one. Okay, now let's save them in a project folder named RAG on our WSL file system. If you don't have WSL yet, check out this video and also make sure our documents are stored in an additional folder named data just like on GitHub.
Then we will open a WSL terminal and we will install a very important tool named Olama. It runs language models directly on our computer. It does it for free and it gives us full control. Just copy this Linux command, okay, from olama.com. There you go. This one. And paste it inside your WSL terminal.
Now, once we have Olama installed, we will download two very small models to our system. We will start with my personal favorite, Quen. This is the model we will turn into an expert. So, let's download it with Lama Pool Quen, specifically in version 2.5 and with 1.5 billion parameters. We will also need a second model for something called embeddings. We will cover it shortly, but it basically organizes the text in our documents and makes it easy to search. So in our case, we will pull a model named bge-m 3.
Now once we have both models on our system, we will take care of the working environment. So let's cond-ash n. We will call this environment rag env and we will install python 3.12 in it. We will of course activate it and we will install a whole bunch of dependencies inside it. You can just copy them from the description where lench chain helps us talk to python. Fice stores our embeddings, PIP PDF reads our PDFs and Jupiter this is where we write our code. Now even though we will use CPU for this workflow at the end of the video I will show you how to do the same on GPU. So don't go anywhere. Now let's give it a quick run. Then finally we will navigate to our project folder with cd rag where our data folder lives. And here we will open our Jupiter notebook with Jupiter lab.
Now let's start by copying the imports from the description so we don't have to worry about them later. And let's quickly run this cell with ct controll enter. Now when you get this warning don't worry about it. It just means that some of your packages are not maintained. It doesn't mean that they don't work. It just means that they won't be updated. Okay. So, if we rerun the cell, the warning is gone.
Then we will move on with the first stage of rag which is loading the documents for our expert system. For this, we will type os.list dear as in list all the files in a directory named data. And this will pull all the file names from it. Now we can also sort them alphabetically with the sorted function. So it will just encapsulate it in the function. And now our files are printed in the correct order. Now once we are happy with the order, let's save it to a variable name. Let's call it file names. And then right below we will load them file by file with file in file names. And for each of them we will create a file loader with loader equals pi pdf loader into which we will pass the name of the file as well as the name of the folder where our file lives. In our case that would be data. So we basically pass the path to our file right over here. Now since each file has many pages in it, we will go ahead and extract them with pages equals loader.load. How do we know if it worked? Well, we can just display the first page in each document with display pages in the index of zero. And then we will add some kind of a separator. Let's print a whole bunch of hashtags. Okay, there you go. But actually, let's only focus on the page content. So, let's copy this attribute name. And then we will display pages in the index of zero dot page content just so it's easier to double check that everything is a perfect match.
Now, at this point, each page from every document is loaded separately. But for reggg we only need one collection of pages that we can process together. So we get page one, page two, page three from the first document and only then we move on with the next one. For this we will create an empty list above our for loop. We will call it all pages. And then into this list inside our for loop we will save the pages with all pages.extend passing it pages. And then we can just get rid of our next two lines of code because we already verified that. So if everything worked properly on your end when we print the length of all pages then we will get 49 pages. Boom. And great we successfully loaded the investigation files into Python. But we haven't involved AI in the process just yet. Right now these are still just documents sitting in memory. The real magic happens when we give AI access to this information.
But once you understand how to give AI custom abilities, the next thing most people want to learn is agents, systems that can use these abilities to actually automate your work. If that's something you're curious about, I highly recommend taking a look at HubSpot's free AI agents cheat sheet. I included the link in the description, and let me show you quickly some of the cool parts. It is a practical guide that breaks down seven popular AI agents, what they're best at, their setup time, cost, and when you should actually use them. What I really like is that it doesn't just throw a bunch of tools at you. It gives you helpful resources like starter prompts, real use cases, and even reasons why you might want to skip it. It covers anything from simple agents that just collect information from websites to automation tools like N8N and all the way to coding agents that help you build software faster. My favorite section is probably the comparison chart. If you're anything like me, you've probably heard of OpenClaw, Manus, Zapier, and many other tools, but it's hard to tell which one is actually worth learning. So, this chart gives you a quick overview that helps you figure out where to start. This guide is completely free. Grab it right now using the link in the description. Thanks so much to Hopspot for sponsoring this video. And now let's go back to implementing Rag.
Our next step is called chunking. We are splitting the pages into smaller units of text and we make sure that they overlap a bit. For example, if we split the sentence, this is not my first rodeo into two chunks, we get this is not my as well as not my first rodeo. So, we have an overlap of two words or more accurately six characters. Think of it as a safety net. We make sure that neighboring chunks share a little bit of information. So, let's do the same with our documents. For this, we will copy the recursive character text splitter from our imports and we will initialize it in a brand new cell assigning it to splitter. Next, we will apply the splitter on our documents with splitter dotsplit documents passing it our all pages variable. And this is where we get our chunks. So let's call it chunks. So let's see exactly how many chunks we got by printing the length of chunks. And okay, we still get 49, which is the same as our number of pages. Not good. But let's quickly see why. When we open our PDFs, or at least one of them, we see very large fonts and lots and lots of spacing. So technically, every page in our document is already smaller than the default chunk size, which is 2,000 characters. Our splitter has no reason to break it apart. But here's what we can do to change that. Okay, so back in our notebook, we will go ahead and add a chunk size argument inside our splitter and we will assign it to a maximum of 500 characters. Additionally, we will set the chunk overlap to something like 150 characters. And now when we rerun this cell, we get 147 chunks. So we are in full control of all these numbers.
And it is finally time for embeddings where we take a language model that is an expert in organizing words. It analyzes our chunks and makes our searching mechanism much much faster and easier. We cannot chat with it, but it can help us find order in chaos. For example, imagine we are building a castle from Legos. The embeddings model will take our chaotic box of Legos and will sort them based on color. So when it is time to assemble them, it will be much easier for the chat model. Same goes for words. The embeddings model groups similar ideas together, making them much easier to find later. So, let's quickly load it with lama embeddings, passing it the model name of bge-3, the one we downloaded earlier, and we will assign it to embeddings.
Next, we will need somewhere to store our process chunks because we don't just put our sorted Legos in the old box. We need special boxes for each color. So in rag, we use something called a vector database. If you'd like a dedicated video on it, let me know in the comments. Now there are different kinds of vector databases. But in this project, we will use fice. So down below, let's initialize it with fice in all caps dot from documents. and we will pass it our chunks followed by the embeddings model. So we basically tell it take all these chunks pass them to the model and store whatever it returns inside you. We then give it a variable name of vector DB and as soon as we are done we will immediately save it with vector DB dots save local and in my case I will call it Eland investigation. So now when we give it a run this might take a few moments but from now on we can skip the entire workflow up until this point. I will show you how to load it later.
Okay, but how do we know if it worked? Well, first we need something called a retriever. We basically give it a question and it searches the vector database for chunks that relate to it. When it finds them, it returns them. So let's create this retriever with vector db as retriever and we will call it retriever. And then right below we will call retriever.invoke invoke passing it some kind of a question. In my case, why is Eland under investigation and then we will assign this expression to retrieved chunks with no typos. Okay. And then at the very end, let's go ahead and print these retrieved chunks like so. And okay, we get a whole bunch of information in return. It looks like a bunch of documents, but it's kind of hard to read through it. I'm not even sure how many documents there are, you know, in those chunks. So, instead of our super simple print statement, we will quickly convert it into a for loop. And now, let's give it another run. And okay, now we can clearly see that we are dealing with one, two, three, four chunks. But we don't really have to. We can actually control the number of chunks we retrieve by setting a search keyword argument inside our retriever. So we will type search underscore kw arcs assigning it to a dictionary mapping k to some kind of a number in my case five. And if we rerun this cell then we get one, two, three, four, five chunks instead of four. So once again we are in full control of all these numbers and you can tailor them to your specific data.
Now there is one last modification that we need here for a proper reggg process. Right now we have a list of retrieved chunks. Obviously, we can iterate over them, but if we want to feed them into a language model, we need them as a very long string where each chunk is separated from the other. For this, we will first create an empty string named context. And then right below for chunk in retrieved chunks we will call context plus equals chunk specifically focusing on its page content only. And then very importantly at the very end of this expression we will add a double new line operator. Okay. So back slashn back slashn and this will separate between one chunk to the next one. Okay. So now this lovely context string is exactly what we pass into our chat model in the next section.
Finally, we reach the fun part. Passing all this information into our chat model and asking it about the investigation. So first let's load our model with chat lama passing it the model name of coen 2.5 with 1.5 billion parameters and then we will assign it to llm. Now just to demonstrate our starting point let's see what happens without rag. We will go ahead and call llm.invoke passing it the same question from earlier. Let's just copy it from right over here. Okay. But now when we run this cell, we will never actually get an answer because sometimes it will refuse to talk about politics. Okay. Or other times it will just make something up about blockchain or money laundering or you know either way it will make things up. So when we complete the rag process the difference will be obvious.
Now the first thing we need is a function that we can use anytime we ask a question. So let's go ahead and call it ask and let's pass it a question parameter. And then into this function we will copy our invoke command replacing our hard-coded question with the question parameter. And then we will assign this expression to response. And then at the end of the function, we will go ahead and return the response. Why? Because the content is the only thing we actually care about. This is the actual answer from the model. The rest of these arguments, they're not really important. Then right below, we will make sure to call this function, you know, with ask, passing it the original question from earlier. Let's just copy it once again. Okay. Okay. And now if we run this cell, we get the same silly type of answer as earlier, much longer this time. But this is nothing new. If we want to implement rag, we need to plug in our retriever. So back in our embedding cell, we will go ahead and copy our retrieved chunks along with the context in our for loop. And we will go ahead and paste it at the very top of our question of our function. Fixing the indentation, of course. like so. And then once again we will replace the hard-coded question with the question parameter. And this is where the rag process actually happens. Instead of sending only the question to our model, we send it both the question and the context. So when we invoke the llm, we will go ahead and pass it a multi-line fing where we specify the context using the context variable from above. Additionally, we specify the question using the question parameter also from above. And now we can officially rerun this code, but this time we will get a proper answer. Okay. So, why is Eland under investigation? Well, that's because he has a possible affiliation with an extradimensional entity known as Agent Smith. Perfect. Our rag pipeline is complete and Quen is officially an expert on the case.
But before we celebrate, let me show you a few improvements that will make it even better. So, in terms of best practices, you can find all of them on my GitHub. But let's quickly go over the most important ones. Right now, our AI detective can answer a single question at a time. But real systems work as conversations. The moment we start asking multiple questions, we need to store the chat history. There is no automatic way to do it. It is up to us to ensure the model remembers not only the current question, but also everything that was discussed before. It is also important to give our model an identity. Is Quen an AI detective, a journalist, a lawyer, or maybe you wanted to be the real Agent Smith? A simple instruction can dramatically change the way your model behaves. Another improvement is loading the vector database directly from disk, not rebuilding it every single time. We already saved it earlier, so we can now just load it in our notebook every time it restarts.
Now in real systems we are usually not running the workflow on CPU. We are using GPUs instead. So if you have a CUDA based GPU on your system the only change in your workflow is installing fice GPU instead of FIC CPU. The code stays exactly the same. Just copy the installation command from the official FIC GitHub page and remove FI CPU from your setup. In terms of a llama, it automatically uses your GPU if it's available. Otherwise, it falls back to the CPU, so you don't need to worry about it. And finally, probably the most important detail here is selecting the right parameters. In this project, we picked some random numbers and they just happen to work. But in real systems, you test many different combinations in a process called hyperparameter tuning. I have a great video about it using scikitlearn. But rag is no different. If we don't actually optimize things like chunk size and K values, we're not really machine learning. We are machine guessing.
And congratulations, you officially reached the end of this crash course. I hope you enjoyed it and learned plenty of interesting things. Let me know in the comments what you like the most and what kind of projects you'll be building with your new set of skills. And as for the investigation, Eland is clearly guilty. Not only is he Agent Smith, but he's also Red Skull Env. Our investigation doesn't really prove it, but we all know it's the same actor. And thank you so much for watching. If you found this video helpful, please share it with the world. And don't forget to leave it a huge thumbs up and all kinds of comments. Now, if you'd like to see more videos of this kind, you can always subscribe to my channel and turn on the notification bell. I'll see you soon in an awesome tutorial. So, in the meantime, bye-bye.