The Future of RAG is Agentic - Learn this Strategy NOW

Retrieval augmented generation has been the go-to way for feeding external knowledge into LLMs since the dawn of generative AI. It is the classic method for turning an LLM into an expert at something that you care about, like your favorite agent framework or your eCommerce store.

But if you've actually tried to implement RAG before, you definitely know its common pitfalls—the things that make you want to pull your hair out, like the wrong text being returned from the search and the LLM completely ignoring the extra content that you give it. The kind of things that make it so that even though RAG seems logically sound to you in theory, it just completely falls apart in practice.

You are certainly not alone if you feel this way. That's why there's so much research in the industry right now for how to do RAG better. There are a lot of strategies out there, like reranking, query expansion, and rank normalization—things that I'll cover in other videos on my channel.

But out of all of the strategies that I've researched and implemented myself, agentic RAG is the most obvious. It works the best, and it's exactly what I'm going to show you how to do in this video. I'll show you how to take standard RAG and then turn it into an agentic approach that actually delivers, so you don't feel like throwing your computer out the window because nothing is working for your agent.

In the last video, I showed you how to use Crawl for AI, an open-source LLM-friendly web crawler, to scrape entire websites super fast for RAG, and we used the Pantic AI docs as an example. Now we're going to take this much, much further because we are going to ingest all of the documentation into a database for RAG using Supabase.

Then we'll use Pantic AI to create this agentic RAG agent, leveraging our knowledge base. Finally, we'll use Streamlit to create a UI to interact with our agent very seamlessly. Through this, I'll walk through what agentic RAG is and why it solves the common pain points that we often see with RAG, all at the same time building a live example for us to see this in action.

So buckle up because I am shoving a lot of value into this video for you. I'll even talk about things like Quadrant versus Supabase for RAG, how you can expand the heck out of this setup, and also how you can try this agent for yourself right now with no setup. So with that, let's dive right in.

Now, the big question that I have to answer to really set the stage for the rest of this video, where I do a deep dive into an agentic RAG solution, is: what is agentic RAG? There are a lot of websites, articles, and diagrams out there on the internet explaining it, and the best one that I have found is this article on the We8 website, which, by the way, We8 is a vector database platform.

I'm not going to read through this entire article here, but really, it's these first two diagrams that I really appreciate to help us understand what agentic RAG is and why it is useful. The first diagram, which I'll zoom in on here, is what basic RAG is. It explains this very clearly.

Essentially, you have a knowledge base made up of a bunch of documents. You split them into chunks so that we have bite-sized information to efficiently give to the LLM. Then we turn all of these chunks into vectors—their mathematical representation—using an embedding model, and then we store that all in a vector database.

So essentially, every chunk is turned into a large string of numbers in a vector that represents all of that information. When a query comes in from the user, when you're interacting with the agent, that query is then turned into a vector representation just like the chunks were, using the same embedding model.

Then, using some mathematics under the hood—some vector math—the query is essentially matched to the most relevant documents that are then given as context to the LLM. Essentially, the prompt is expanded to include whatever relevant context was retrieved from the vector database knowledge base.

So you have something like: here's the user question, and then here's all of the relevant context for you to answer it. Essentially, what the prompt then looks like for the LLM, which then gives the final response using all that information to augment its response. That's why it's retrieval augmented generation.

You can kind of see the downside to this. It's what we call a one-shot here, where the vector database is retrieved from, we give that extra context, but then the agent can't actually reason about what it was given. It can't decide, "Oh, this wasn't enough context, so I need to look again or search in a new way." There's no opportunity for the agent to actually improve upon whatever it got for that additional context.

And so that is what agentic RAG does. Let me zoom in a little bit on this diagram here. This is agentic RAG—the second diagram in this We article here that I absolutely love. Basically, instead of just having RAG be a one-shot, "Here's some context from the vector database," we actually create RAG as tools for the agent to interact with.

This unlocks things like even being able to search through different vector databases. An agent, based on the user question, can reason about where it will actually go to find the knowledge. You can have other tools that can search the knowledge in other ways, giving the agent the ability to intelligently explore the data and not just work with what it's given in the first shot.

That is the power of agentic RAG. I don't like these tool examples in particular because these don't really relate to the knowledge base, but you can very much imagine other tools where maybe for the Pantic AI documentation, you would give it the ability to see the URLs for the Pantic AI docs and actually pick to search in a specific page based on the title of the documentation page.

That's actually something that we are going to be implementing. There's one other diagram that I really like as well, which is this one. I think this is very similar, but it explains it very nicely as well because essentially you have your large language model layer, here's your AI agent, here are the tools that it has access to, and the retrieval functions that actually go to things like the vector database.

This is just one of the options. You have all these other tool functions and action functions that can work with the knowledge base in other ways, really allowing the AI agent to reason about how it is searching for the answer for the user. That's what makes it so much more powerful.

Hopefully, this example that we're going to go through, what we'll be building in this video, will make this whole idea of agentic RAG super crystal clear for you.

All right, so there are a few moving parts for building our agentic RAG solution. I've created this simple and beautiful diagram here with the help of Claude to walk you through what we're going to be doing in this video together. I'm breaking it down step by step to make this entire process very digestible.

By the end of this video, we're going to have an awesome solution here. The first thing that I'm going to be doing with the RAG website crawler is essentially taking what we did in the last video with Crawl for AI, scraping the content of a bunch of websites, and now actually putting it into a database so that we have our RAG knowledge base.

I'll do this before we're even doing anything with the database setup because I want to get into the code right away, dive into the meat of it. Once we have this script set up, then we'll go over to Supabase, do the very simple setup there, and I'll walk you through everything.

Then we'll run our script, and then boom! After these first two steps, we have our knowledge base ready to go for an AI agent. Naturally, the next step is using Pantic AI to create the foundation of our AI agent.

We're actually going to start with just basic RAG because I want to give you clear examples of when basic RAG is enough versus when we need our agentic RAG approach. We'll take these negative examples of when RAG doesn't work and test it again once we move on to actually implementing agentic RAG.

There are a couple of new tools that we'll introduce here to make it a very robust solution. After that, I'll talk briefly about the Streamlit UI and show you how I built that so that we have our interface. The entire time, as we're doing our testing, I'll be using the Streamlit interface and then just referencing it at the end, showing the code off a little bit for that.

That is our entire process, breaking it down step by step. Let's dive right into it. First things first, if you want to try the exact agent that we're building in this video for yourself right now with no setup, you can do that.

All you have to do is head on over to the Automator Live Agent Studio, sign in, and you'll get some free tokens. I have the exact agent covering in this video hosted right here, so it's a really cool way to actually go and throw some of your questions at it, explore Pantic AI documentation, and test out what we are building right here.

It's really awesome because there are even some example questions for you to get started here. Everything—like I have not changed the code at all—everything is the same. So yeah, in this case here, I'm asking it for an example agent, specifically calling out the weather agent that I know is in the Pantic AI documentation.

Boom! Look at that! We've got the full code example right from the documentation. This agent is awesome; it saves so much time having to scrunch the documentation because the agent is doing it for you with RAG. Super neat!

All right, let's get into coding our agentic RAG solution. Everything that we're going to create together here I will have in a GitHub repository that I'll have linked to in the description. That is all the code that we are looking at right here.

First of all, to ingest our documentation into a knowledge base, we've got this script right here. Everything that we currently see is just what we already did in the last video with Crawl for AI. We'll be expanding this to actually work with a database.

Then we've got the SQL to show how to run all this later to set up our database. This is our Pantic AI agent that we'll build together from scratch, and finally, the Streamlit interface. I'll go over the code for this a little bit later as well, and then we've got a nice README in the repository to walk you through everything and how to set this up yourself as well.

Going back to that diagram that I showed earlier, I just showed you the code for every single step that we've got here. I'm going to break it up step by step, and I'm going to make this as simple as I possibly can.

In the last video on Crawl for AI, we covered this repo, went through their documentation, and some of their examples on multi-URL crawling to get it so that we could give all of the documentation pages for Pantic AI and actually get the contents of every single one of them super, super quickly by doing parallel scraping on all these pages.

Going back to the code that I've got right here, that is what this script is doing so far. Again, this is mostly just what I coded up in the last video on my channel, where we crawl in parallel all of these URLs that we fetch by getting all of the Pantic AI documentation URLs from the sitemap.

This page right here lays out the entire directory of the Pantic AI documentation, so we just process that XML to get all of our URLs, and then we crawl them all. That's it!

Now, in this function, we have a new step right here. After we crawl the page and get the markdown in the result here, we now actually want to add it to our knowledge base. I'm zooming a little bit here, focusing in on this function right here.

This is not implemented yet in the script; that is what we're going to add. After we get the content, we now want to process it—create the chunks, get it ready to insert into our knowledge base, and also define all the metadata and things like that that we will get into.

So let's go ahead and do that now. I'm creating a new function here called process_and_store_document. That's what's called in our function, and its job is to orchestrate everything that we need to get the information ready for the knowledge base.

First of all, we want to chunk our text. The reason for that is when we store our information in the database for LLM knowledge, we don't want to store a massive page just in one record for the LLM to retrieve.

If we have a documentation page that's like 50,000 words and we give that all to an LLM at the same time, that is going to overwhelm the prompt and just make the LLM super confused. So we want to split up longer pages into smaller chunks so that the LLM can retrieve the specific knowledge it needs from a piece of documentation without pulling in the entire thing.

We chunk the text, and we'll define this function later. In parallel, we want to process all of these chunks. At the exact same time, we're going to do what it takes to actually turn all these chunks into embeddings that we can put into our vector database—in this case, using Supabase.

We'll do setup for that in a little bit. The last thing that we need to do is store the chunks. We split them up, we process them, and then we store them. Super simple three-step process.

Now all we have to do is define all of these different functions for chunking, processing, and storing. So let's go ahead and do that right now. The first function that we want to define is the one to chunk our text.

Chunking can get a little complicated because there is a lot that goes into taking a single page and breaking it up into smaller bites for the large language model while still respecting things like code blocks and paragraphs.

Think about it: if we have to split a document at any given point, we don't want to split it in the middle of a paragraph or the middle of a sentence or a code example. We want it to preserve that so that one chunk has all of the information that it needs, so you don't have to look at the start of the next chunk to finish the sentence or the paragraph or something like that.

Because this is a little bit more complicated, I actually used AI quite heavily to help me create this function. You can use that as a rule of thumb—anything that's a little bit more complex that I code here, I definitely used AI to help me with, and you can do the exact same thing.

I'm not going to explain this function in a ton of detail, but just know that I'm taking a single document, turning it into bite-sized chunks, and respecting code blocks and paragraphs when I do it.

The first thing I'm going to do is just loop through all the text in the current chunk, basically just defining the boundaries for it. If we're at the end of the text, we're just going to take what's left because we're at the end; we're not going to split another time.

Otherwise, we're going to try to find a code block. Because Crawl for AI is returning markdown to us, I know that this is the syntax representing the start and end of a code block in markdown. I try to find it and actually adjust the end to include the entire code block if I find one there.

That's how I respect code blocks, and then I do the same thing for paragraphs and sentences as well. That way, at the very end of a chunk, we are not splitting in the middle of a paragraph or a sentence or something like that.

Then we just extract the chunk and clean it up. At this point, we have a final chunk that we can add to our list of chunks. When you combine all of these chunks together in this list, that creates that full document.

That's how we have basically just an array. If I go up to the top here, this function returns a list of all the chunks that together make up the entire document that we just split up. Then we'll just return all the chunks at the end.

A little bit more complex, but the idea is very simple. You can very much see we get the text, and we're given back a list of chunks so that we can insert them one at a time in the knowledge base, so the LLM doesn't get super confused pulling in the entire document at once when really it was just looking for a small portion of the knowledge in that page.

That is our function to chunk our text. At that point, we've gotten past this right here in our main function to process and store every page. The next thing we have to do is process each chunk, and you'll see in a little bit what that actually entails here.

First, we're going to start by defining our data class that actually defines what information we get in a chunk when it is fully processed. We have the URL for the Pantic AI documentation page, we have the chunk number—like maybe if we split the document into 10 different chunks, this is chunk number two out of 10—and then we're going to give a title and a summary for every single chunk.

This is going to get into agentic RAG a little bit, but if we have this extra context about what this chunk represents, that's going to help the agent reason about when to use this specific piece of knowledge. More on that later.

Then we have the actual content of the chunk, some metadata—we'll get into that as well—and then finally, we have our embedding. This is what actually allows for RAG, where we have kind of the vector database part of what we're storing here, where we can do that retrieval with vector mathematics.

That's our process chunk, and that's what we're going to use in this function right here to process our chunks. The job of this function is to take the chunk in the raw string format and turn it into this class right here with all that information I just went over.

The first thing that we want to do is actually use a large language model to extract a title and summary from this chunk. We'll define this function in a bit; I just want to lay out this entire process first.

Same thing with this next function: we want to get the embedding. This is the actual vector representation of the chunk content, which is what we're going to use for RAG. Finally, we will create the metadata for our chunk.

This includes things like the source. Really, metadata is just additional information that you attach to the record so you can do specific filtering. For example, I say that the source is Pantic AI docs, and the reason I do that is this actually makes it so I can use one database table for my knowledge base for many agents.

If I have a Pantic AI agent, and then I have a CW for AI agent, and then a LangChain agent, I can have them all use the same knowledge base. Whenever I query the knowledge base for that specific agent, I would just say, "Give me the results from your search, but only looking at the records where the source is Pantic AI docs."

I can segregate my knowledge in a single table for multiple agents using metadata, and I can also query based on time. For example, if I only want to search based on the records in the last day that were ingested in the last day, I can do that with this crawled metadata.

That's the importance of metadata. It's super key to have good metadata because that gives you a ton of options. It unlocks so much as far as additional filtering that you can do, and that's a whole advanced concept in and of itself.

I just wanted to include that here to show that you can take filtering very far with RAG. Finally, we're just going to return our processed chunk. We have the URL at this point that was given into the function, we have our chunk number that was also given in, and then we have the title and summary that we extracted with a large language model, the content, the metadata, and then the embedding that we created.

Now we can define the function to actually get the title and summary. There's a nice little system prompt that I added here to instruct the large language model, which I'm going to be using—GPT-4 mini—but we instructed on how to extract the title and summary.

We're looking for a specific JSON object, so that way what we get back can be referenced in this way, where we want the title that is extracted and then the summary that is extracted. It's very simple; we're just going to make a call with our OpenAI client that we're giving here, where we are using GPT-4 mini or whatever you define in your LLM model environment variable.

You can tweak this as well. We give it the system prompt and then also the chunk—just the first 1,000 characters. That's all it needs to then create a title and a summary, and we return a JSON object as our response format.

That is what we return. We just load it as JSON, whatever we get back from the LLM, and that is going to again include our title and our summary. Very, very simple. If we encounter any errors, we'll just return that; we'll tell that to our terminal there so we know that there was some issue trying to actually process this specific chunk.

Next, we want to define a function to get the embedding. This, again, is what we're going to be using for the actual basic RAG when we want to do that search with vector mathematics.

We're going to be using, again, the OpenAI client, just using their text embedding 3 small model, giving the text from the chunk, and then just returning the embedding. Super simple function. OpenAI makes it so easy to get our embeddings.

Very, very easily. So that is that, and then we'll just handle any errors as well, just like we did before. If we do get an error, we're just going to return a zero vector here so that the process can continue, but we at least have some value for the embedding there.

So that is it for processing a chunk. Now we can get on to our very last step here, which is actually inserting chunks. We reference this function in our main procedure here to process and store our documents. This is inserting a chunk; this is where we actually put it into Supabase.

Now, everything that we are inserting here is not just for basic RAG. You'll start to see once we get into agentic RAG how some of the things that I'm inserting, like the URL and the title and the summary, can be used to understand the knowledge in more ways than just basic RAG with the embedding.

So we have the embedding for basic RAG, but also this other information so that we can explore the knowledge in different ways. First, let's finish off this function here to actually insert this data into Supabase.

First, we're going to create an object here that has all the data that we want to insert just based on the chunk here, and then we're going to insert into the site pages table. We'll create this in a little bit here; this is the SQL to actually create this in Supabase.

But we're inserting this chunk into the table—very, very simple—and it will just handle any errors. That is everything; we have now completed this script. There's a lot that went into this.

Actually creating the AI agent itself is going to be much simpler, but now we can get on to setting up our database tables and then running the script to create our knowledge base.

The sponsor of today's video is an open-source platform called GPU Stack, and I'm super excited to bring this to you today because it is free to get started and solves a huge problem in the AI space, which is scaling local AI.

Trust me, when you really start to build an application around local AI—whether you have your machines hosted in the cloud or you built them yourself—you’re going to really need to think about how you manage your different GPUs and all of the AI inference going on on your machines.

That is what GPU Stack can help you with because it is an open-source GPU cluster manager, and it is phenomenal. It supports any hardware that you could be running on; you can dream of any large language model that's going to be accessible.

It supports single-node, multi-GPU setups as well as multi-node setups. Now, the obvious question is: why use a GPU cluster manager? The easy answer is if you have tried to use more than just one GPU for your local AI stack, you know how important and difficult it is to monitor your GPU usage and also efficiently dish out the inference to your different resources.

That is what GPU Stack makes very easy. It is so easy to monitor your GPUs in the platform, set up your clusters, add GPUs into them, and just manage your entire local AI stack. They have an open web UI-like interface, which is very, very clean, with a huge focus on doing all that monitoring and managing of your clusters.

You can set up your stacks as OpenAI-compatible endpoints, so you can really hook in your clusters into any system you could dream of. I will have a link in the description to GPU Stack. I would highly recommend checking it out because as soon as you use local AI for an agent or an AI application for anything but just yourself, you are going to need something like GPU Stack to monitor all your resources as you are scaling your application.

So we have just finished this step right here. Now it is onto the Supabase setup, and then we can run the script to create our knowledge base. Luckily, what we just went over is actually the longest and most complicated part of this entire process.

Once we actually have our knowledge base set up, creating the agent to interact with it is actually pretty simple. Let's head on over to Supabase right here. I'll assume that you already have a project created in Supabase. The rest of it, I'll walk you through everything.

This is the homepage for a Supabase project. Once you have it set up, all you're going to want to do to get things started is go on over to the SQL editor tab right here. You'll click on this, and then you'll have this blank slate right here where you can paste in all of the code in the SQL script that I have in the repository.

Going back to our code here, that is this. You're just going to want to copy this, and I have this set up specifically for Supabase, including row-level security policies as well. With this, you'll just paste it in right here and then go ahead and click on run.

This is my first time setting up for the Supabase project, so I am doing it completely from scratch with you right here. Once you have this run, you can just go to the table editor here, and then boom! There we go; we've got our site pages table with nothing in it right now.

Now it is time to actually set up our environment variables, including our Supabase URL and service key, so we'll insert that all in and then run our script to get everything in this table here.

First of all, to actually get our Supabase credentials, you want to go down to the project settings in the bottom left here and then click on API. What you're going to want to do is copy your Supabase URL and then also your service row secret. These are the two environment variables that we need to give our script so that it can actually insert the knowledge into the table—the site pages table that we just created.

Copy those two things and then head on over back to the code and open up your env.example file. You're going to want to take this, rename it to .env like I did here. I'm not going to show this file because it actually has my credentials, but you're going to want to take the Supabase URL and service key that you just copied from the Supabase UI and then paste those here.

You're also going to want to get your OpenAI API key, which I have instructions here on how to get it, just right here in the .env.example. Then define the large language model that you want to use. In my case, I'm using GPT-4 mini. This is exactly the ID that you would just set it to right here to reference it.

That's it! We now have all the credentials set up, and we can go ahead and run the script that we just created and create our knowledge base over in the terminal.

Now let's go ahead and run this. I have everything installed, including my Python libraries already. I've got instructions in the README for how to do that with the requirements.txt file. I'm in the current directory with that script that we just created, and now all I have to do is run the command Python and then the name of the script, crawl_pantic_ai.py.

This is actually going to be quite chatty as it processes all of the chunks and sets them all in the knowledge base, but it's kind of cool to see this all roll through. It's running in parallel in batches of, I think, 10 pages right now, and also batches of 10 chunks at a time.

It's cool to see it make all the calls to OpenAI to get the titles and the embeddings and the summaries, and the calls to Supabase to insert into the knowledge base. It's pretty neat! I'm going to pause and come back once this is done in just like 10 to 20 seconds.

All right, there we go! We are at the end of our script, and we have everything in our knowledge base now. Also, if you run into any rate limit issues with OpenAI or anything like that, you can also just change the batch size and also add in some delays with the time Python library.

Play around with that if you have to. I didn't run into any of that myself, but I know some people did from my last video when I did this with Crawl for AI. Let me go out of this for now, and then let's go over into our Supabase because we're going to check out all of the knowledge that we just added in.

I'll go back over to the table editor here, go to site pages. It says it's empty right now. There we go! Now refresh. We got 245 records! So out of the 42 pages for the Pantic AI documentation that we scraped from this right here, we turned it into 245 chunks.

We've got the URL, the chunk number, the title and summary that was created with GPT-4 mini for every single chunk, the content, metadata, and then the embedding as well. That's the massive vector representation that we'll actually use for basic RAG.

With that, we can now move on to our next step here—not this diagram, this one—which is creating the AI agent for basic RAG. Let's go over and do that now.

This is super meta, but we are building a Pantic AI agent to be the Pantic AI expert. If you want help building agents like what I'm about to build right here, you can literally refer to this agent to help you code your agents and also refer back to the Pantic AI documentation.

Let's dive right into creating this agent here. First of all, we're going to start with just a basic RAG example. Then we'll extend this exact same agent to do a lot of really cool stuff with agentic RAG.

The first thing we're going to do is import all the Python libraries that we need and then do some of that initial setup, like loading our environment variables, including the specific model that we want to use from OpenAI, configuring LogFire—this is the logging monitoring for Pantic AI, which I'm just going to disable for now.

I'll set this up maybe in another video on my channel and showcase that. Then we want to define the dependencies for our Pantic AI agent. The way that you typically set up your agents with Pantic AI is in three parts: you have your dependencies, which are things like the Supabase client and the OpenAI client that your agent needs access to, to use in the tools.

That's number one. Then you set up the actual agent itself—that's number two. Finally, you define the tools for your agents. We're going to walk through all three of those.

This is step number one with the dependencies. Then we're going to create our system prompt here, and this is a pretty rough draft of a system prompt. I haven't said too much in this here, starting really, really basic—just describing what kind of tools it has, what the role is of this agent, even kind of pointing to some things that we'll have for agentic RAG in a little bit here, but you can just ignore that for now.

Anyway, that's our system prompt. Then we'll create our actual agents. So step number two: we define the agent using the agent class from Pantic AI. We specify our model that we defined above with OpenAI, we give it our system prompt, we tell it the kind of dependencies that the agent will have access to, and then also there's retry logic for your LLM calls baked into Pantic AI as well, which is super neat.

We've already done step one and two with dependencies and the agent, and now we can define our first tool for the agent, and that is going to be the tool for basic RAG. Before we even define that, I'm just going to throw in this function right here to get the embedding for the user's question.

Get the embedding for the user question and then use that to check the knowledge base with RAG. This is basically just the same function that we created in this other script right here, so I'm not going to go over that again.

Now we can define our tool to retrieve relevant documentation. This is basic RAG in its finest. What we're going to start out with is a docstring here because in Pantic AI, this docstring that you have at the start of a function is how you tell the agent when and how to use this tool.

That gives it all the context it needs, like the arguments and the purpose of the function. The way that we turn this function into a tool for the agent is with this what's called a decorator right here. We put this at the start of our function with the @ symbol and then the name of our agent tool.

That immediately turns it into a tool that's accessible for our Pantic AI agent. We can continue on with the code for it here. First of all, we're going to get the embedding for the user's query, and we're also passing in that OpenAI client so that our agent has access to OpenAI to do this.

Then we are going to run this procedure here. This is what actually does RAG within our site pages table. If I go back over to Supabase here and I go to the script that we ran, I'll just scroll down right here.

We have this function that we created called match_site_pages. This is all the logic that happens in the database, where we can give it a query, the embedding for a query, and it'll find the records in the site pages table based on those embeddings that are the most similar.

What we get back from running this is going to be the top—and we default to 10—so the top 10 by default records that match the user's question the most. Back over to VS Code here, actually, I'm specifying the match count as five right here, so I'm getting the top five most relevant document chunks.

I'm also doing that metadata filter, so I'm only going to be pulling knowledge from this table here where the source is Pantic AI docs. That way, if I have other agents that are also storing knowledge in here, I'm not going to overlap and accidentally pull knowledge from those agents as well.

Obviously, right now, the only records in this table are going to have the source of Pantic AI docs, but this is just a nice little value add for you to show you how you could do something like this with metadata.

Pretty neat! If there are no results that are returned to us, we're going to tell that to the agent so that it can report back to the user. Otherwise, we're going to format all of the chunks. We're going to take the top five results, put them all together into a nice string, and then return that to the large language model.

We're giving it a nice format so it can understand that, "Oh, this is chunk number one, and this is chunk number two," and so on. Then we'll handle any errors as well. If there are any issues when it's retrieving the documents, we'll print that to our terminal and also report that back to the agent so again it can go back to the user and say, "Oh, sorry, I had an issue trying to retrieve from the knowledge base."

Obviously, you don't want that to happen, but yeah, so that is our first tool. That is very basic RAG. Now we can actually run this agent. We don't have anything that is agentic RAG at this point, but we'll get to that.

First, I want to show you when basic RAG works and when it does not. All right, so here we are in the Streamlit UI chatting with the Pantic AI expert that we just built. I'll cover Streamlit more at the end of this video, but let's dive right into testing out this agent because I want to give you a good example of when basic RAG is enough.

Then I'll show you an example where this falls apart, and then we'll get into agentic RAG to actually fix that problem. First, I'll start with a very basic question: what are the supported models?

This will do that retrieval; it will perform RAG to get me the answer here based on everything that we put in the knowledge base. There we go! Pantic AI supports, and then it lists a bunch out. It even says for specific model names, "Here are a list of some of them."

So yeah, this is a solid answer. I very much pulled this from the knowledge base. If you were to ask Claude or GPT-2, some agent that doesn't have access to this documentation, it would completely bomb this question. So that's awesome; it's working pretty well.

Now let's get to an example where just basic RAG will not work. If I go back to the Pantic AI documentation—let me actually go to that right here—and I go to the examples, they have this weather agent, which is a really, really awesome example.

I'd actually recommend checking this out if you want a good example—maybe besides my videos, of course—on how to build a Pantic AI agent. I would reference this right here. If I go back over to the interface here and I ask it, "Give me the weather agent example," let me spell that right—example from the documentation.

It's going to bomb this. I've obviously tested this out a lot before I recorded this exact segment for you. I know that it doesn't get this right. So it's giving the response here, and it looks good so far, but you can see that this is pretty bad.

It kind of started correct, but it's not defining the dependencies. I mean, this is just like a tiny script compared to what we actually have here in the weather agent. It didn't really give me the example. You can see that it obviously pulled some relevant information to get me a half-decent answer, but this still is not acceptable.

This doesn't actually give me everything that I need to go ahead and then just run this entire agent. So that is why we need to move on to creating agentic RAG. What we're going to be doing here is actually giving more tools to the agent so that not only can it perform RAG with the embedding column that we have right here, but it can actually pull a list of the URLs from the Pantic AI documentation, reason about what pages it might want to look at to get the right information, and then actually go and visit those URLs.

We can even have it, for example, use the title and summary to help it figure out which pages to visit as well and pull the content from. This is where we start to get into having the agent actually be able to reason about where it needs to look for knowledge instead of just performing basic RAG queries.

Let's get into adding those tools. So back over in our code, let's add these tools for agentic RAG. Now, before we even do that, I just want to say that you could probably tweak the chunking strategy or the retrieval strategy to make it so that it could successfully answer that weather agent question with just basic RAG.

I don't know for sure, but there are so many ways to optimize RAG and other ways besides agentic RAG. It still just gave a very clear example of how basic RAG can fall apart, especially when you have to look at bigger pieces of information to give a longer answer.

Typically, with RAG, you're just looking at a tiny segment of the entire knowledge base, but with agentic RAG, we can actually instruct it on how to visit entire pages and pull all of that or parts of the page that it needs to answer a question.

The first tool that we're going to implement here—just adding another tool—is the one to list documentation pages. This is going to go to the database. Here again, we have a docstring to tell the agent when and how to use this tool, but it's essentially just going to go to the site pages and pull all of the URLs.

We're doing the metadata filter again just to make sure that we have just the Pantic AI docs and nothing else. We're pulling this list of URLs, returning an empty list of URLs if nothing was returned. Otherwise, we're giving these back to the LLM so it now knows which URLs it can reason about potentially visiting to answer the user's question.

We'll also return an empty list if there's any errors as well. Well, I promised at the start of the video I would talk a little bit about Quadrant versus Supabase as well, so here's a really quick rundown of when to use either.

I like using Supabase a lot because it simplifies things. This is a very good example. I have RAG for this function right here and then also structured data that I have to store in a SQL database used in this function—all in the same platform because I am using Supabase.

Quadrant is just a vector database; I can only store embeddings. If I wanted to use Quadrant in my solution, I could only use it for basic RAG. When I get into agentic RAG, where I'm actually looking at structured data in a table, I would still have to use Supabase.

I'd actually be using both, but with Supabase, I can have the RAG and the structured data all in one place. That's why I generally recommend Supabase. I will say, though, that Quadrant is a lot faster.

If I wanted to optimize for speed here, I actually would have all my knowledge embeddings stored in Quadrant, have this function reference Quadrant instead of using Supabase with this match_site_pages function right here, and then I would still also use Supabase and have the table for all of the metadata and things like that, like the title and the summary and the URL—everything that I'm referencing in this function.

That's a quick little rundown there. Now let's get into the second function right here. Now that we have the ability for our LLM to get all of the URLs, we obviously need to give the ability to get the contents of a specific page that it wants as well.

Again, another docstring here telling when and how to use this function. All we're going to do is query the site pages and grab the title, content, and chunk number from a specific URL, and then we're going to return that to the LLM or maybe just say, "No content found for the URL" if it doesn't find anything.

Then we'll handle any errors as well. The last thing here, we'll also do some formatting here just to make the information nice and easy to understand for the LLM, also combining all the chunks together for that single page.

This way, the LLM can now read from an entire page, and this is going to improve the performance of some of the more complex questions because now we are implementing agentic RAG. Instead of just a basic query using embeddings, we're actually giving the agent the ability to look at the URLs and think, "Oh, if they're asking for the weather agent example and there's literally a page where it's, you know, like pantic.ai/docs/example_agent, oh, I should probably go visit that, pull all the contents from it, and then use that to answer the user question."

Instead of that little bit of knowledge I might retrieve with basic RAG that doesn't actually cover the full example. With that, let's go ahead and run this again and test it out.

So back in the Streamlit UI, let's go ahead and ask that infamous question that it botched the first time. I'll say, "Give me the weather agent example code from the documentation."

All right, so it probably will start with basic RAG still because I instructed it to do that in the system prompt, but then as a part of the reasoning here, I'll instruct it to then go on to actually check the URLs, pull the right page, and then boom! Look at this!

We are definitely getting the full example now. I can even open up my terminal and see that it did start by retrieving relevant documentation. Basic RAG figured out that it didn't give it the right answer, so it went on to list the document pages, found the right one, so it called the tool to get the page content, and now that gives us the full response here, even including, yeah, like literally everything—the main function, the tools, defining the agent.

This is the full example for us! Now we have much better results because the agent can actually reason about the knowledge base and not just do a one-shot RAG query.

Last but not least, I wanted to cover the Streamlit user interface very quickly here because I set this up as a way to use Streamlit to really chat with any Pantic AI agent. It's pretty neat, and I don't want to code this from scratch with you right now because the big focus of the video is everything with agentic RAG, and this is just the way for us to talk to the agent.

I wanted to show this off very quickly here, and this is also in the GitHub repository, so you can use this to chat with this agent or hook it into the Live Agent Studio like I've showed in other videos on my channel. You can use this to really work with any Pantic AI agent with Streamlit because essentially I have these functions here that convert the text from the Pantic AI format to what I can display in Streamlit.

Then I have my main function where I actually invoke my agent here and I pass in the dependencies that I set up on the Streamlit side. I give all of the dependencies, like the Supabase client and the OpenAI client, into my agent right here, get the response, and then I actually stream this out.

Instead of it being just like, "Boom! Here's all the text at once," it streams it out in typewriter style, like we like to see with interfaces like, you know, ChatGPT or Docomo messages in the message history.

The agent, when it calls RAG and it gets the context for a specific part of the Pantic AI documentation, that is now a part of the message history, so it can even reference that later on as well. It doesn't have to pull that page a second time, so it's pretty neat!

Overall, it's a pretty concise function here. We got the main function where we set up all the messages in the UI, get the user input, and then handle that whenever the user inputs something. We add it to the conversation history, and then we stream the response from the agent, which also adds the AI messages and the tool messages to the message history—all in the format that works for Pantic AI.

Again, I'm not going to dive super, super deep into the script, but you have that there for you. The main thing that I really wanted to focus on was this agent right here, which, by the way, if you want to expand on this, there are so many ways to even for this specific example to add a lot of new tools to make this even more robust than it already is.

For example, I'm just throwing out something really random here: you could set up a dedicated knowledge base specifically for Pantic AI examples, and then you could have a function that is using RAG but just for the Pantic AI examples.

That way, when a user asks for an example, you're going to get more accurate results actually fetching an example instead of the agent maybe accidentally looking somewhere else in the Pantic AI documentation that doesn't actually have full examples.

Maybe they're just like short little clips of code, and that's not what the user wants when they want a full example. That kind of goes back to the We8 article, where you can have different knowledge stores and have the agent actually intelligently reason about where it wants to go.

That's just one example. Or you could even have another area of the Pantic AI document—like maybe you reference the actual code in the GitHub repository—and that could be its own knowledge base. You could have listing and fetching functions for getting specific content in the GitHub repo to actually dive into the code.

There are so many ways that you could expand agentic RAG here. I'm not promising that what I have set up here is going to work perfectly because, in the end, there are so many different opportunities to make this better and better.

That is a wrap for building an agentic RAG solution that actually solves a lot of problems we typically see with RAG by giving your LLM the ability to reason about where and how to get the right information from your knowledge base. You get much more consistent and accurate results.

Of course, this is just one way to level up your RAG game. There are so many different strategies out there and also ways to build on top of agentic RAG and my solution. Like I covered earlier in this video, the main thing that I just really wanted to drive home here is the importance of giving your agent the ability to intelligently leverage your knowledge base instead of those one-shot queries that you have to get out of your system.

I'll be putting out a lot more content on RAG and AI agents on my channel going forward as well, so stay tuned for that. If you appreciated this video and you're looking forward to more things RAG and AI agents, I would really appreciate a like and a subscribe.

With that, I will see you in the next video!