Transcription
If you're interested in building AI applications, then stick around because I've got a great tutorial for you. In this video, I'm going to be showing you how to build a multi-AI agent application in just a few minutes using a fantastic tool called Langflow.
Now, we're going to integrate this into a basic front end. I'm going to explain step by step exactly how to build this out, and by the end of the video, you're going to learn about how to implement RAG (Retrieval Augmented Generation), how to look up data from a database using an AI, and how to combine multiple AI agents together in a system that actually works and can solve real-world business use cases. This is super cool, I promise you, you're going to learn a ton. So let's get into a quick demo so you can see what we're going to build and the value you're going to get from this video.
So, I'm on the computer now, and I'm going to demo to you what we'll be building, which is a better customer support agent. Something that can access your orders, it can see if it's been shipped or cancelled, it can find information about the pricing, the status, it can go look up different products in the database and return that to you. And it can also do things like utilize RAG to answer frequently asked questions. Of course, it can do a lot more than that, but those are the core examples.
By the end of this video, you're going to see how to build an advanced AI system that you can extend and can actually solve real-world business use cases. That's the important thing here. This isn't just a toy, you could actually use this for a business, and maybe you're running right now. Very cool, and you can even take these type of flows and make money from them, which is what I find super interesting.
Now, on the left side of my screen here, I have the chat interface. This is a really simple Streamlit application, very basic Python front end that'll show you how to set up even if you don't have a lot of experience in Python. And on the right side of my screen, I have what's called the Langflow editor, where I've built out this AI flow. Now, we're going to dive in and talk about exactly what that means, but you can see that this is kind of a drag-and-drop editor that allows us to combine all different types of components to build these advanced AI systems. So we'll obviously build this from scratch, but I just want to show you this is kind of what it looks like, and then we can call this from an API and use this in any type of application that we want, which is what I'm doing on the left side of my screen.
Now, Langflow is pretty new. Personally, I love using it, and it comes from the sponsor of today's video, which is DataStax. Now, don't worry, this is completely free, you don't need to pay for any of this, and I've worked with DataStax for many months now to build some awesome AI apps. Anyways, let's look at what this can actually do, because obviously you want to see what the agent is capable of.
So, first things first, I asked this previously, "Can you help me with my order number 12?" Now, this is able to go into a real-world database, look up the order number, and then find this information. It's not using a custom SQL query or anything like that, like the LLM is actually doing the querying of the database. Then, it's also aggregating this with a table that contains all of the products that this user ordered and displaying that on screen. So already, that's an interesting capability.
Now, we can also ask this things related to frequently asked questions. So I can say, "What are, you know, the standard shipping times?" And this agent actually has access to a PDF that contains a bunch of questions and their common answers. So rather than just making up some random response, it can actually go to that PDF, which can act as something like the company database or internal information. It can find relevant replies and then give that to us. So here you can see it says, "Domestic orders take 3 to 5 business days, international orders take 7 to 14 business days." And this isn't just made up, this is actually what's contained in that frequently asked questions document, which we gave to the AI. So this is using something called RAG, which we'll talk about later.
I can also say, "Can I cancel order number 14?" And same thing, now it's going to go look that up, and it will also look at the cancellation policy and see if we're able to cancel that. So you can see that it was able to find that. It says the product name, the description, and then gives us some more textual data. This is really just the start of what's capable with this type of system, but I want to show you how you build out this functionality so that you can extend it and make it really cool.
Anyways, let's get into it and let's build this app. Okay, so let's go ahead and get started. Now, the first thing we need to do is create a free account with Langflow. So you can do this from the link in the description by pressing on this button. You also can just Google Langflow. It is free, it's actually open source as well, you can check out the GitHub here, and you can run all of this locally on your own computer, so you don't need to do it on the web, but it's just a bit easier.
By the way, if you guys want to help support me and see more videos like this, do me a favor and just go to their GitHub and give it a star. It really does help support the project and show that you guys are actually getting value from these videos. So if you are, give it a star, and that really does does help me out. That's all I ask of you.
Anyways, I'm going to press on "Get Started for Free," and it's going to bring me to this sign-in or sign-up page. Now, again, free account, just make one of these here so we can get to the online editor. I'm going to go to sign in, and because I already have an account and I believe I'm signed in, it's going to bring me to the Langflow homepage.
Okay, so we're in the Langflow page here, and what we're going to do is start by creating a flow. Now, a flow is like the Langflow project, it's where we can build out our AI system, and then we can call the flow by either running it locally on our own computer or by using the API that Langflow provides. So let's go to "New Flow" here, and you'll notice that there's a bunch of different templates. So if you just want to check this out for yourself and maybe you don't want to follow along with this video, you can look at examples of how to do RAG, how to build a simple agent, classification, coding, content generation. They have a bunch, and they're adding more and more, and you'll probably see this page change quite a bit over the next year or so because this project really is getting a ton of growth and a ton of work on it.
Anyways, I'm going to go to "Blank Flow" because I just want to show you how we run this completely from scratch, and I'm going to start by showing you how we make a very simple AI agent. Now, first thing I like to do in my flows is just give them a name. So if you look up here where it says "Untitled Document," we can go to "Flow Settings" and we can just give this a name. So let's just call this "Customer Support" or something, uh, because this is going to be a customer support agent, of course. We can give it a description, and for the endpoint, let's just call it "customer." Now, the endpoint will simply be kind of like that slash path that we go to when we want to use this different flow. So just give it a name that's unique. If you've used Langflow before, if not, you can name it anything that you want, and we'll use that much later in the video.
Okay, so let's save that and go back into our editor. Now, from here, I'm going to show you how we build out some different flows, and we're going to start by building some simple functionality, specifically around RAG. I'll explain what that is in a minute, and then we'll start making this agent more and more complicated, and I'll explain it step by step.
Now, something important to know here is that you can import and export different flows. So if you actually press on this button here, you can press on "Import," and you can take the JSON file that I'm going to leave linked in the description in my GitHub repo, and you can just read in the entire flow. You'll have to change a few credentials and keys, but pretty much everything will just be the exact same. And then same thing, if you want to save this, you obviously can just save it online here, or you can press on "Export," and you can save the flow as a JSON file with or without your different API key. So it's very useful in terms of giving the flow to other people to mess around with, and I just want to make you aware of that, that all this will be available from the link in the description. There's also some other resources like some sample data that will insert in the database and a PDF, and you can find that again from that GitHub linked below.
Anyways, let's start building something though, I don't want to bore you too much. When you use a flow, you always start with some type of inputs. You can see on the left side, we've got all these different types of components. There's a lot of stuff here, and don't worry if it looks a little bit different because again, sometimes things change, it's pretty intuitive. And anyways, when you start, you're always going to use a chat or text input. Now, we're going to use a chat input to begin, and this allows the user just to type something in in the Langflow playground. Now, there's a playground here, and this allows us to actually test our flow in real time. So every time we build something new, we can go here, we can test it out, and we can get the response.
So we always start with some kind of input, and then at the other side of our flow, we always have some kind of output. Okay, so if you wanted something like an Echo app, you would just connect the message to this text, and what would happen is as soon as I type something in, it would just get displayed as a chat output in our playground. So let me quickly show that to you. If I go to playground and I say "Hey," and I hit enter, you'll see that it just gives us back "Hey." And the reason for that is my flow just took some input, passed it to the output, and then that was that. I'm just going to delete this by pressing on it and click delete, but I'm trying to show you that we start with an input and then we end with some kind of output, and everything in between is kind of our AI system. So that's how you get some kind of results. Make sure you have an input and you have an output, and you'll see that we connect different components by just dragging in these different color bubbles to the corresponding kind of inputs and outputs.
Okay, so what I want to do is start with a RAG example. So we're going to bring in an agent. We're going to go to "Agents" here in the components. You also can search for it and just drag one of these in. Now, for the agent, we have a lot of different options. Obviously, we need a model or an LLM, which is going to run the agent. We need to choose the model name, and then if we're using something like OpenAI, we need our OpenAI API key.
Now, if you press on this little globe here, this this is going to allow you to view variables, and you'll see that I have an "OpenAI API Key" variable. Now, if you don't have that, you can make a new one and you can set it equal to your key, or you can use any other provider that you want. Again, keep in mind you can run this locally on your computer too, so you can use something that's open source.
Okay, anyways, if you want your OpenAI API key, you can go to this link right here, which I'll leave in the description. Press on "Create New Secret Key," copy that, and bring that into Langflow. I believe you do need to have a credit card now on, uh, open, but it's very cheap, maybe a few cents to use this. Otherwise, you can use one of the open source providers.
Okay, so what we're going to do now is we're going to drag our chat input as the input to our agent. So the idea is that whatever the user asks, maybe they ask some kind of question, because this will be like a customer support agent, we'll pass that as the input. But then we also need to give some instructions to our agent. So I'm going to create a prompt. To make a prompt, you can grab the prompt from this kind of component on the left hand side, and we'll just drag the prompt and connect that to "Instructions." From here, I can press on this little square and I can write a simple prompt that I want to provide to the agent. So let me copy one in, and then I'll talk about how that works.
Okay, so I just copied in a really simple prompt. You can type this out, or again, you can just find it from that flow link in the description. But it says, "Your primary goal is to accurately answer user questions by utilizing information from the company database. Use Retrieval Augmented Generation to fetch and present the most relevant data." And then we have a question, and this is a prompt variable. So if you want to have something dynamic that you pass into your prompt, like the user's question, for example, then you can do that. So we're going to pass the user's question. It says, "In your response, incorporate the following relevant data points," and then we have these results. Now, these results are going to be the results from our database, which contain kind of relevant information to what the user asked. I'll show you how we get those in a second. But the point is, if you want to have variables, you can embed them inside of braces. Notice it shows you right here. And now we have two variables that we can kind of pass into this prompt, which we will then pass into the model.
So I'm going to pass this as the agent instructions, and I'm going to take my chat input, so the output of my chat input, and I'm going to pass that as the question to my prompt.
Okay, now also at this point, I'm just going to change the names of a few things. So to do that, you can just double-click on the name, and I'm just going to call this "FAQ" just so that we know this is handling kind of the FAQ, what do you call it, agent components. I'm also going to take the response and I'm going to pass that to my chat output. So at this point in time, I actually already have an AI system, but we're missing the important component, which is the RAG component.
Now, for any of you that aren't aware, RAG stands for Retrieval Augmented Generation, and it simply means that we're going to go search inside of a database and find relevant information based on what the user asked, and then feed that into the prompt, and in this case, into our agent. The idea is we can get contextually relevant responses or data, and we can then allow the LLM to reason based off of that. So it gives us things related to, for example, our company or the frequently asked questions that we have.
So let me show you an example of what I mean to make this a little bit more clear. I'm just going to pop this open, uh, let me find it here, which is my frequently asked questions document. So imagine, you know, this is quite simple, I just have some questions and some answers. But if you're a company, you probably have a bunch of documents like this, things related to promotions going on, different sales, policies, terms and conditions, whatever it might be, right? You probably have all of this type of data. Now, ideally, you want your AI system to be able to use this data and not just make stuff up, but actually answer based on things that you know are true, right? That are in your kind of company database or, you know, company internal information, whatever, right?
So what we're going to do is we're going to upload PDFs like these. In this case, I'm just going to use one, and you can find this exact document from the GitHub linked in the description. You could upload as many as you want, and we're going to instruct the AI to retrieve information from this and then answer based on that. So ideally, when I ask it, you know, "How long does it take for me to get my shipment?" it will use this content that we have inside of the document. That's really the kind of high-level overview of Retrieval Augmented Generation, and it uses something known as vector databases, which DataStax provides.
So what we're going to do now is we're going to drag in another component, which is called our Vector Store. Now, we're going to bring in the Astra Vector Store, but you can use whatever you want, and it should automatically populate for you here, the Astra application token. If it doesn't do that, you can go to Astra DB, which I'll show you in a second, and generate one yourself, but it should populate that for you.
Okay, now this database here is a very fast, vector-enabled database. What that means is that we can actually search for information based on vectors rather than traditional SQL queries, and we can pull out contextually relevant responses very, very quickly. So what we're going to do here is create a new database, we're going to create a new collection in that database, and the idea is we're going to first search in this database based on whatever the user asked, we're going to take that, pass that into our prompt, and then this FAQ agent will be able to use information that it got from the database, from our frequently asked question document, to give us some response.
Okay, so in order to do this, we'll make a new database. So I'm going to go to "Add New Database" here, and I'm just going to call this something like "customer." You can call this anything that you want, and for the provider, I'm going to go with Amazon Web Services and then US East 2. You can choose anything you want here, and again, this is completely free, you don't need to pay for this. We're going to press on "Create Database," and this is going to take a second.
Now, while it's creating the database, what we're going to do is go over to the DataStax page where we can manage all of our database instances, and we're going to create a new vector-enabled collection. You have to do this from DataStax, just to make sure that we get everything enabled correctly. So where it says "Langflow" up here at the top, kind of beside my email, you'll press on Astra DB. Now, this is going to bring you to a different page, which is going to allow you to view the different database instances you have. Again, don't have to pay for this. Now, you'll see that you have some kind of databases. In my case, I have an "orders" database, which I was using previously, and I have this "customer" database, which is being created. So I'm going to click into this "customer" database.
Okay, you also see it on the side. We're going to wait for it to finish initializing, and then we're going to make a new collection, and I'll show you how to enable the vector search feature.
All right, so our database has been created, and what we're going to do now is go to the "Data Explorer." Now, from the Data Explorer, we're going to create a new collection. So we're going to go to "Create Collection" here, and we are going to give this a name. Now, I'm going to call this "FAQ," and it's going to be a vector-enabled collection. We're going to do that to ensure that we're able to perform vector search, and for the dimensions and everything else, we're just going to leave this the same and press on "Create Collection."
All right, so now that the collection is active, we're going to go back to Langflow and we're going to connect to this collection, and we're actually going to add a PDF document to it. So obviously, we need data before we're able to perform the, uh, RAG operation. So for here, we're going to select our database of "customer," we're going to select our collection of "FAQ." For our embedding model, we're going to change this to "Astra Vectorize," and I'm going to explain what this is in one second. For now, let's just fill in the details. We're going to change this to be "Nvidia" for the embedding provider. Want to ask for the model, we're going to go "EnEmbedQA," and then there's some more information here, but you can leave all of that blank. We just need what I have here, okay? Customer, FAQ, Astra Vectorize, Nvidia, and then the EnEmbedQA.
Now, this Vectorize thing that we just selected here is a relatively new feature that DataStax has, which allows it to automatically convert data into vectors when you add it to the database. Now, this is a pretty game-changing feature because previously, you would need to generate embeddings yourself. This is essentially the type of vector data that you're going to search through. You would need to make that yourself before you add the data to the database. In this case, we just enable this setting, so Astra Vectorize, and now anytime we add something to this collection, it will automatically be kind of embedded for us. And same thing when we're searching for something, it will automatically convert that embedding for us, so we can perform what's known as a vector search. It's really just a similarity search across the database.
Anyways, that's what that is, I wanted to explain that. Okay, so what we're going to do now is we're going to take our chat input, so the question, right, which is going into our prompt and going into our model, and we're going to pass that to, sorry, we're going to pass the output of that to the search input of our Astra component. So the chat output is going to three places now, but it's also going to the search input, because if we, for example, say something like, "What are the shipping times?" that's the type of thing we're going to search for in our database.
Okay, we're then going to take our search results and we're going to connect that to the variable "results" here in our prompt. But you're going to notice, and let me just move this around a little bit so it's a bit easier to view, that we're not actually able to pass the search results directly to "results" because the bubbles don't match in colors. So what we need to do instead is we actually need to parse the search results, which are going to be converted or come as JSON data, into textual data so we can pass it into our prompt. So to do that, we can search for the component that says "Parse Data." You'll notice that it will take this data as an input, and then we can just take this text and we can pass that to our results.
Okay, so we have some chat input, which is the search string in our database. We get some results, we parse the data, we pass it into our prompt, and then we pass that prompt to the FAQ agent. Now, I noticed that we're using the chat input twice because it's going to the prompt and it's going to the input. For now, that's totally fine. Later, we're going to convert this into something called a tool, which is going to kind of remove this, um, you know, added like duplicate, which you'll see in a second.
Anyways, we now have this system. Everything will actually work perfectly fine, but the issue is we don't have anything inside of our database. So before I run the flow, what I'm going to do is I'm going to add kind of a separate component here, which will allow us to add our PDF to the database. So to do that, we can go to this "Data Components" tab and we can go to "File." Now, this allows us to upload a file, right, to Langflow. Now, after file, we're going to connect this to a "Split Text" component.
Okay, so we're going to look up "Split Text." We're going to take our data, and that's going to go to the data input of our split text. Now, what we're going to do here is we're going to take a PDF file or whatever file, really, can be any type we want, and we're automatically going to split that and convert it into chunks before we add it into our database. So that's what this is doing. There's a bunch of other ones you can use as well, and you can kind of mess with the settings, but that's all this does. You know, if we have like a massive PDF that's hundreds of pages, we can't add that as one document in the database, and we don't even want to do that. We need to chunk it into smaller pieces so then we can search through it when we do our vector search.
Okay, the next thing we're going to do is we're going to go to "Vector Store" again. We're going to bring in our Astra component, and we're going to enable the same things, but this time we're going to take chunks and convert that or add that, sorry, to ingest data. So rather than searching the database, we're actually going to add information to the database, and we'll do it like this. So we're going to go to "Databases," "Customer," uh, for the collection "FAQ," again, and for the, uh, what do you call, embedding model, we're just going to change this to "Astra Vectorize." Okay, same thing. We're going to do the provider, which is Nvidia, and then we're going to select the model, which is this, and we can leave everything blank. That's all we need to do.
Now, what we can do is we can upload the file, and it should be added to the database. So let's upload the file again. You can find this from the GitHub linked in the description, in case you're wondering like, "Where is the PDF file?" or you can just upload anything that you want, it doesn't need to be this. Okay, so I'm going to upload that, and now it's uploaded here. I'm just going to press "Run" on the Astra component. When I do that, it's going to run the flow from all of the components connected to this previously.
Okay, so I'm going to run this. You're going to see that it kind of gets highlighted, and then it should get added into Astra DB, and you see "Astra DB was built successfully," which now means we have that PDF data there. Now we can go look at the PDF data, or we can just test this ourselves by running this flow that we built. So in order to test this, we're going to open up our playground, and because we have a chat input and a chat output, we should be able to run through this flow.
So I'm going to say, "Hey, can you tell me about the shipping times?" and let's spell shipping correctly, of course, and let's see what kind of reply we get. So there you go, we see we get "Domestic orders" and "International orders," and it's not just making this up, this is coming from that RAG component we just built. And if we want to look at kind of what happened here, we can see, um, kind of a, you know, look into the flow and kind of what was going on, and we can actually inspect the input that we're getting. So, for example, if we go to our prompt and we want to see what the, uh, prompt actually was, we can press on this kind of like eyeball here, and you'll see that it actually brought in the information from our PDF. Now, in this case, it just injected all of the data because we don't really have that many things in the, uh, database, but obviously if we had a lot more information, you would see that it would just get the relevant documents, bring that in here, and then the, uh, LLM, sorry, is able to reason based on this data.
Okay, so that's a very simple example of RAG, and now we have kind of one part of our customer agent built. We're able to look in the kind of, uh, what database and and find information, at least about frequently asked questions. We also know how to add information to our database by using this kind of file upload that we just built. But now what I want to do is I want to go a step further, and I want to allow the agent to actually look through customer data, so things like the orders, right? So to check the status of an order, whether it's cancelled or shipped, or to check different products. And we're going to build a kind of new system now, and then we're going to combine those two together by having an agent that can use these different agents. So we're kind of combining like, uh, you know, almost like a manager, and then two agents beneath that manager, where the manager agent will decide, "Hey, should we use the FAQ agent or should we use the order lookup agent?" and then it will go and use those tools based on what's required.
So that's what I was saying with this FAQ, what we're actually going to do is we're going to convert this into a tool that an agent's going to be able to use. So I'm going to go here, this is really cool, and we're going to toggle something called "Tool Mode." Now, when I do this, what's going to happen is you'll notice the input goes away, and so does the output. The reason for that is that now this is something that we're going to pass to another agent in its tools input, and it will be able to be used by that agent. So we've just kind of created our own custom tool.
Now, let's put the chat output over here, and I'll show you how this works. Now, first things first, whenever we make the tool, we're going to go into the controls here, and we're just going to make sure that we give the tool a name and a description so that our model knows how to use it. So if we scroll down here, we should be able to find some kind of agent description. Yes, so it says "Agent Description." We're just going to open this up, and we're just going to really briefly describe what this agent does. Say, "Looks up and answers frequently asked questions like shipping estimates, cancellation orders, and more." Uh, actually, maybe we don't want to put orders. I'll just say "cancellations and more common business questions." Okay, and let's spell "answers" correctly. So we're just giving it a description so that our other agent knows when to use this agent. Okay, there's a bunch of other stuff in here you can mess with, but for now, that's all you really need.
Okay, so now that this is in tool mode, we want to use this. So we're going to bring in another agent. So we're going to bring an agent right here. Okay, we're going to connect our chat input as the input to this agent, and then we're going to take this tool and we're going to connect it to this agent. Now, we're just going to give this agent a name, and we're just going to call this like "Manager Agent," so we know what it's doing, and we'll just select our OpenAI API key again. And then for the agent instructions, again, we kind of need to tell this what it should do. So I'm just going to click on "Agent Instructions" here, and I'm just going to directly copy these in. Again, you can find this from the link in the description, but it says, "You are a skilled customer service manager and information router. Your primary responsibility is to use the available tools to accurately address user inquiries and provide detailed, helpful responses. You can do the following: look up order numbers, access product information, and answer frequently asked questions." Now, these two we can't currently do, but we will be able to do them in one second, but this is what I'm telling kind of my main agent, like, "Hey, I'm going to give you access to these tools, use them properly to answer the question."
Okay, so now we have our manager agent connected to our FAQ agent. Now I'm just going to take the response from that and I'm going to connect that to the chat output, and that really completes the manager agent. All we've done is added another layer here where now we're only going to use this tool that we just built if it's necessary. So this agent will determine with the agent framework, "Hey, based on what the user asked from our chat input, which is over here, and we can rename this to "question" maybe, so that it's a bit more clear, should we use this tool?" If we should use this tool, it's going to go and invoke it, it's going to pass the input or whatever we need to this tool, it will then get the question, and then give us that response.
Okay, so that is cool. Let me just move this file upload down here, it's starting to get big now. But the next thing we want to do is we want to build the ability to look up orders. So in order to look up orders, we need to have some of those. So we're going to switch back to Astra DB, and I'm just going to show you how we can insert some sample data in the database related to orders and product information.
Okay, so let's go to our customer database and let's make some new collections. So we're going to go to "Data Explorer" and we're going to press on "Create Collection." And this time, I'm not going to enable the vector collection because I don't need that. So I'm just going to call this "orders," okay, and this is going to represent customer orders, and we're going to make that collection. Again, doesn't need to be vector-enabled because we're just going to look this up, like normal, we don't need to have like a vector search for this. Then we're going to make another collection, and this is going to be called "products," and same thing, it doesn't need to be vector-enabled, at least for right now.
Okay, so let's create these two, and now I'm going to show you the sample data that we're going to insert here. So you can see that I have these two CSV files, and let's kind of split the screen 50/50 so we can look at this at the same time. I have this "sample_products.csv" and I have this "sample_orders.csv." Again, this will all be available from that GitHub linked in the description.
Now, what we're going to do is we're just going to, um, copy this information pretty much, or like take these CSVs and import them as data for our orders and products collection, just so that we have something so that we can see how this works. So I'm going to go to "Products" and I'm going to go to "Load Data." Okay, I'm just going to select a JSON file, so I'm going to select that from my, uh, customer service document, whatever folder. Again, you can download it from the link in the description, and which one is this that we're in right now? This is products. Okay, so we're going to load the products. So "sample_products," we should see that it kind of gives us all of the different fields. Obviously, you can change these if you want, and then I'm going to press on "Load Data."
Okay, so let's give this a second to load, and then we should see that we have that in there. And let's go to "Orders" and let's do the same thing. This time we're going to load our "sample_orders." Now, the thing that I want you to pay attention to in our orders is that we have this array of the products that were ordered. So it contains the IDs of products in our product table here. So what we're actually going to do is allow the model to look up information in both the orders and the products database and to kind of aggregate data together, which I think is really interesting.
So we have this sample data, you can read through it, it's just like five or 10 rows, I didn't add a lot of stuff, and now I'm going to show you how we can get the model to use this. Okay, so we have it here, have it orders and products. So let's get out of this, let's go back to Langflow by changing this to say "Langflow," and let's write that new tool.
Okay, so similarly to before, we're going to bring in another agent. Now, this agent, we're going to zoom in on it, and let's call this the "Order Lookup Agent." Okay, now let's go into the agent. I'm going to enable tool mode on it, by the way, just like I did before, so make sure you click that. We're going to go to "Controls," and we're going to give this a description. So for the description, we're just going to say, "You can look up order and product, uh, information." Okay, good, sweet. So we'll close that.
And now we need to give this access to some tools. So again, the way that agents work is you can give them tools, and they will then utilize those tools to go and solve some kind of task. So first, let's do the OpenAI API key here. For the agent instructions, same thing, let's copy something in here. So let me just open this up, and let's copy this in. Okay, "You are an expert in analyzing customer orders and providing detailed and accurate information. Your primary role is to utilize and provide, uh, tools, let's use tools to efficiently look up order numbers, retrieve relevant details about the orders, and address any questions or concerns the user may have." And then it says, "Orders always contain an array of product IDs ordered. Use these IDs to look up specific products from the product lookup table and aggregate the product information with the order to provide a clear summary of the order. If the order does not exist, simply tell the user to try again, as the ID wasn't found. Only return information about orders, do not return anything else."
So what I'm telling this here is, "Hey, when you look up an order, because we're going to be able to get that in one second, it's going to contain some product IDs. I also want you to look up those products and then take that information and kind of aggregate that with your response." So this is how I'm getting it to utilize these tools, which we're going to build in one second.
So first things first, we can take this and we can pass this as a tool to our manager agent. So again, the manager agent will be able to use the FAQ agent and the order lookup agent. It's like kind of delegating the tasks between them. Then we need to make some tools that the order lookup agent can use. Now, these are actually quite simple. We're going to go to "Tools" and we're going to bring in Astra DB. Now, notice I didn't bring in the vector search Astra DB, I just brought in the normal Astra DB from tools, because I don't want to perform a vector search, I just want to do a normal lookup based on an ID, for example.
So for the tool name, the first one that we'll have will be the "order lookup." So we're going to say "order lookup," okay, and then for the description, we're going to say, "A tool used to look up an order based on its ID." Okay, now for the collection, we first need to select the database, so let's select the "customer" database, and then the collection is going to be "orders." And then what we're going to do is we're going to provide a "Tool Parameter." Now, this tool parameter is going to be the thing that the agent needs to pass to this tool in order to use it. So what we're going to do is we're going to pass exclamation point and then "order_number." Now, what this means is that I'm going to look up the "order_number" field, and the exclamation point means this is mandatory, and for the value, we just leave it empty.
So what we're saying is, "Okay, hey, you want to use this as a tool, so let's add this as a tool for our agent. If you want to use this thing, this is what it does, but in order to use it, you need to pass me an order number," and then that order number will be used to actually look up information in the orders database. We'll then pull that information and pass that to our agent. So that's kind of how this database is working. We're saying, "Look up this field, look up this order number field, and then find the, what do you call it, the like row that has this order number and then return that to me."
Now, we're going to do the exact same thing here, so bring this in again, but for products. Okay, so same thing, let's connect our tool to the "Tool Input." Okay, now for the tool name, we're going to call this "product lookup," okay, and for the description, it'll be the same thing, "Look up a product based on its ID." Look up a product based on its ID. For the database, we're going to choose "customer," for the collection, we're going to choose "products," and for the tool parameters, this time it's going to be the "product_id" with an exclamation point. So not "order_number," but exclamation point "product_id." Sweet.
So there you go, we just provided two tools now to our order lookup agent. That's really all we need. It's going to be able to now be used by our manager agent and then look up information in one of these two tables.
Now, let me just quickly run through the entire flow, and then we can test it. I understand it's small, try to zoom in and use arrows so you guys can kind of see what's going on. Okay, so first things first, we have a question, right? Like this is where our flow starts with the input. Now, what we'll do is we will go to the manager agent. The manager agent will have these agent instructions that we gave it, and it will have access to tools. It's going to have access to the order lookup agent and to the FAQ agent, and what it's going to do is use these tools, maybe one, maybe both, maybe none, depending on what we asked it, to give us some kind of response.
So if we ask it something related to shipment times, it's going to go and use the FAQ agent. If we ask it something related to our order and we give it an order number, it's going to use our order lookup agent. If we ask something related to products, it's going to use the order lookup agent. It's going to take responses from these different agents, which are really tools that it's able to use, aggregate them together using the LLM, kind of process the text, and then give us some kind of chat output.
So let's play around with this and let's see if it works. "Can you tell me about order number 1001?" By the way, the order numbers are 101, 12345, and all of the products are like 101, 102, 2011, 203. You can see them in the database, but if you just want to test it, use these IDs. So you can see we're able to find that information about the order and kind of summarize it and give it to us. Sweet.
So we can keep testing this and messing around with it, but what I want to show you now is how we run this with a custom front end. So how we actually use this from something like our Python code, how we send an API request and take what's on the web now and use it in a real application. And by the way, this is just kind of an updated version of the flow. I just rearranged it so it's a little bit easier to understand, and you can kind of see what's going on. If you download this, I'm going to add some comments to it so it'll make a little bit more sense. But now what I want to do is hop into a code editor, uh, and start writing some code.
So if you go to this API tab here, you'll see that it's going to give you a bunch of different ways to actually trigger this flow from code. So we have Python API, we have Python code. So you can actually just download this flow's JSON file and you can run it directly using Langflow. You do need to install that first. And then you have the JavaScript API as well.
Now, what I want to do is I want to use the Python API. So what I'm going to do is I'm just going to copy all of the content that's inside of this file here, just so we get all of the variables and stuff that we need. And then we'll clean this up and actually convert this into a Streamlit application, so we get a nice simple front end.
So let's open up a code editor. I'm going to go with VS Code. Uh, you can see I already got this project open, but I'm just going to open a new folder. Let's go File, Open Folder, Desktop, and let's call this, uh, you know, customer agent. Okay.
Inside of here, we're going to make a new Python file. So let's go full screen, main.py, and let's paste all of this in. Okay.
Now, there's a few things we need to do. So let's start by just cleaning this up a little bit. Now, we're going to keep all of this stuff, the base API, Langflow ID, flow ID, all of that kind of stuff. This tweaks dictionary, we don't need. This is just something that we can use to modify the flow when we call the API if we want to do that. And we're going to keep this run flow function, but for the run flow function, we're just going to take a message and nothing else. Okay. We'll leave what's in here for now, but we will kind of adjust that in a second. And then we're going to get rid of this main function. Okay. So let's get rid of main, let's get rid of the main line. And I want to focus on just working in this. Let me get rid of this comment here. This run flow function, uh, in order to actually call the API, we'll just quickly test it and then we will, um, kind of write out the simple front end.
Okay, so what we're going to do is get rid of this try catch, which we don't need. We're going to get rid of warnings, typings, and then pretty much everything else, uh, that's up here. We don't need JSON, we don't need arg pars. We just need requests for right now. Okay.
Now, our application token, let's get that quickly. If we go back to Langflow, let's open that up, you'll see that we can generate a token. So let's generate that and copy it. Let's go back to VS Code and let's make a .env file. Now, in this file, we're just going to call this app_core_token is equal to, and then we can simply paste this token. I don't want to reveal it to you, so I'm just going to paste it and then leave this file, but just make a variable app_token and make it equal to this, uh, this, what do you call it, token that you just copied. Okay.
So I've got app_tokened inside of there. So what I'm going to do now is I'm going to import OS. And where I have my application token variable, I'm going to load in that token. So to do that, I'm going to say os.environ.get and then we're going to get the app_core_token like that. Okay.
A few more changes to make here. Inside of the run flow function, we can change the endpoint to simply say endpoint in all capitals, just so that we use this variable right here. For the output type and the input type, we can change this to chat because we're just going to use the chat input and the chat output that we have there. And then we can get rid of these if statements here where it says if tweaks and if application token, because we know we're going to have an application token. Then, for our application token, we can change this variable to be the constant that we defined up here. And that's, that's pretty much it for this function. I know I went fast, but again, this code will be available from the link in the description.
So this run flow function now will allow us to actually call out to Langflow and send a request. But in order to do that, we first just need to install the Python modules we're using here. So for example, we need to install the requests module, we need to install Streamlit, and we need to install something called python-dotenv. So I'm just going to make a new file called requirements.txt and I'm just going to list these out. Uh, you don't have to do this, but I just want you to know what we need to install. So we need to install python-dotenv, Streamlit, and then requests. Okay.
Now, in order to install this, the first thing that I recommend doing is creating a virtual environment. However, here I'm not going to do that, just for simplicity and to go a little bit faster. So I'm simply going to say pip install -r and then requirements.txt. This is going to install the different Python modules that we need from this file. You also can just manually install these three modules. Okay.
So I'm going to run that. This is just going to install in my global Python installation. Again, this is not best practice, usually you use something like a virtual environment, but it's totally fine for what I'm doing and my system right now. Okay.
Now that we've got that, I'm just going to import something in here, which will allow me to load in this environment variable file. A lot of times in Python, if you try to load an environment variable file, you'll get an issue, and that's because you need to import this. You need to say from dotenv import load_dotenv. This will load any environment variable files that are in the current directory if you call this function. So we're going to say from dotenv import load_dotenv, call the load_dotenv function, and now when we use os.environ.get, it will actually look inside of that .env file that we just made. Okay.
So I want to test this before I make my simple Streamlit front end and just make sure this function works. So first of all, let's get rid of that comma and let's call run_flow. And then what we can pass to this is just a string with the question we want to ask. So I can ask something like, "What are the shipment types?" I know I'm using the same example over and over again, it's just the one that's familiar to me. Okay.
Now, to run this, we can go python main.py. I assume you're familiar with some Python code if you've made it this far. If not, you can just copy this, not a huge deal, it's not overly complex. This is just sending requests to the Langflow API, which kind of contains our flow at that customer endpoint. So now we're just going to send the request, we'll wait a few seconds, we'll get a response, and then we can print out what that response is. So actually, let's say result is equal to this and let's print it, because if we don't print it, we're not going to see anything. So we're going to say print result. And then when I run this code, it will take a few seconds because we are running through a bunch of different LLMs, and then we should get some response and we can print it to the screen.
And there you go. We do notice that it is a little bit complex, kind of hard to read because there's like a bunch of stuff here, but if you parse through this, you can kind of see that it says stuff about shipping, like three to five business days, seven to fourteen, uh, for international, and yeah, there you go. We got the response. Obviously, though, this is going to look better if it's in like a nice pretty front end and a form. So let me show you how we write that using a really great module in Python called Streamlit.
So first thing, we're going to import JSON because we're going to need that in one second, and we're going to say import streamlit as st. Okay.
Now, we're going to write a function. I'm just going to call this main, and this main function will kind of represent our Streamlit input. I'm going to make this super simple, and if you've never seen Streamlit before, you're going to see how easy it is to make a simple UI. So I'm going to say st.title and I'm going to say chat interface. Okay. You can call this whatever you want. I'm going to make a message input box by saying message is equal to st.text_area. For this, we can just give it a name, so I'm going to say message, and then I'm going to give this a placeholder, so like, you know, ask something dot dot dot. Okay.
Then whenever the user types something in, that'll get stored in the message variable. We're then going to put a button. So I'm going to say if st.button, and for the button, we'll just say run flow. Then what we're going to do is the following. We're going to say if not st, or sorry, not st.message.strip, this just means remove all of the whitespace characters, then we're going to say st.error and we're going to say please enter a message, because we don't want to just be sending spaces, we want to make sure they type something in, and we're going to return. Okay. So if we return from the function, it just means like we're not going to execute this anymore, and then we'll call the function again, allow the user to try again. Okay.
Now, so if this is the case, if we press the button, we'll try this. If we need to return, we'll return. Otherwise, down here, we're going to have a try and a catch. So we're going to say try, and then we're going to say with st.spinner, this is like a loading indicator, and we can say as the text here, running flow. Then we're going to say the response is equal to run_flow, and we can simply pass the message to this function that we already tested and we know works. Okay.
Then we can say response is equal to response. And what I'm going to do is just show you how we can grab just the textual response, because this gives us a ton of information that we don't need. So to do that, we're going to say response, outputs, index 0, and then this is going to be outputs, and then this is going to be index zero again, and then this is going to be results, and then this is going to be message, and then this is going to be text, text. Okay.
Now, this is going to be markdown text that we can simply render to the screen. So I can say st.markdown, and then we can take that response and show it on screen. And then we can take this accept, we can say accept exception as e, and we can say st.error, and we can simply display the string e as the error message. I know I went kind of fast there, I'm just trying to make this front end pretty quickly. Okay.
So this whole kind of mass of, you know, outputs, outputs zero, bra, message, text, this is just how we parse through the JSON that's returned to us to just get the text to our question, because there's a lot of information returned there. So if you want, you can look at the payload and you can see all the stuff that's in there, but I'm just giving you the shortcut to just get to the text.
So now, rather than calling result with run_flow, we're just going to say if __name__ == "__main__": then run the main function. And when we run the main function, it will run this Streamlit app. And sorry, we actually don't need the JSON module anymore. And we should be able to use our flow.
So to run the Streamlit app, we can type streamlit run and then the name of our file, which is main.py, assuming this is in the same directory. It will run this application as a Streamlit UI and then open this up in our web browser. So you can see that it just got opened to me here, and I can say now, you know, "What is the cancellation policy on orders?" Okay. And we can run flow and we can see if we get a response. Okay.
So you can see that we get our reply here. It says, "If you need to make changes or cancel your order, please contact. Change of cancellation may not be possible if the order has already been processed." Sweet. And then of course, we can continue testing this out and use it. But I just want to show you how we spin up that simple front end to work with the project.
So now we have all of our AI that we built here in Langflow. We have this simple UI that I just built in Python in like two or three minutes, and we're using the Langflow API to run this.
Now, another option we have, I'm not going to walk you through this, but I want to explain it to you, is we can just run this code locally. So if you want to, you can simply install Langflow on your own machine. You can download this flow as a JSON file, which I'm going to do in one second when I add it to the GitHub. And then you can just run the flow like this, and you can pass the input value of whatever you want. That way, you can run it all completely locally from your own computer, and you don't need to rely on the Langflow kind of like web instance or API in order for this to work.
All right, so we are almost done, but I'm just going to mention one other thing that you might find interesting. So when we do this retrieval augmented generation lookup, often times we might actually want to weigh the results. So it's possible that we want to get things that are maybe more relevant, maybe our FAQ has been updated, and we want data that was most recently posted, or we want something that is kind of based on some ranking right there. There's a lot of times where we want to weigh our responses and get some better, more contextually relevant results.
Now, in order to do that, you can actually do it in Langflow. It's a little bit more complex, and I'm going to share with you this article, "Beyond Basic RAG Retrieval Waiting." So this is something that you might want to look into if you want to get better replies. You can see here, Langflow, and it kind of talks about how this works, how you can use it, what waiting actually is, and how to make sure you're getting the best responses, uh, what do you call it, for the kind of retrieval augmented generation search that you're doing. So definitely check this out. It's going to improve the accuracy of the AI systems that you build, especially when it comes to RAG. And I will leave this link in the description.
With that said, though, that will wrap it up. If you guys enjoyed, make sure you leave a like, subscribe to the channel, and I will see you in the next one. [Music]