Transcription
Today, I would like to show you how to build a movie recommendation RB based on our Vector database, Quadrant, in NN. So, imagine the task: you are sitting in the evening with Netflix and you want to watch something specific. But you want not just a recommendation, you want a recommendation based on your positive and negative experience. For example, I want to watch something about vampires, but I don't want anything sweet like Twilight or something like that. Well, um, do you know how to build that? I know, and I'm going to show it to you today.
And we are going to use an agent, which is going to call our Vector database, Quadrant, with the data. And then, with the help of a large language model, translate this output from the vector database to a nice recommendation form for you, based on your positive and negative input. So, it's going to be two steps. The first step is going to be very easy. We just need to upload our data about movies to our database. And let's just begin.
So, what do we have for data? I decided to just use top 1,000 IMDb movies. It's a Kaggle dataset, you can find it there. And here you can see how it looks like. Um, as you can guess, we're going to convert descriptions to vector representations because it's going to be the easiest to compare users' wishes, users' descriptions to these descriptions in a vector space.
So, how can we get our data from GitHub to N10? We can directly search for GitHub and find a block, get a file here. You can connect with your own credentials. It's pretty easy. Then there is a file, there is a get operation here. You need to write your name in the GitHub minus that here, if you want to connect with me too. And here you can choose already your repo, which is going to be connected. For me, it's going to be this demo. Then we're copying the path of the file, and we already can see how it's going to work. We're going to retrieve our CSV file.
Then we want in this graph to work with a file with the fields in the file separately, because of course, we don't want to just dump them into the vector database and hope that it's going to work out somehow. So there should be an operation for extracting the data from the file, and it should be as simple as that. We get everything here.
Well, now the last step is literally uploading something to the database. So, since I am from Quadrant, we're going to work with Quadrant today. And as you can see here, I have already pre-selected credentials which are connected. But I'm not going to leave you hanging. So, to have the same, to connect them, you need to go to Quadrant Cloud, quadrant.io, and then in clusters, here you can create your own cluster. It's a free tier. I'm using a free cluster, and it's free for everybody. And then in this cluster, this endpoint is going to be the API address which you should use. And then here you can create an API key. Then you can work with this cluster in a dashboard. Here you can see collections. Collections, it's basically all the vectors that you're going to search within. If you would like to learn more about them, you can stop the video and read, or go to the documentation directly. Uh, so here you can see that I already created a collection of all 1,000 movies. But of course, I'm going to show you how I did that, and we're going to run it for a smaller example, so it would be fast on our demo.
So, let's see. We have our credentials connected. We need to insert our documents. Let's create a new collection. So, we already have one, but we want a new one. Let's call it just IMDb. And what do we need else? Um, to upload our data. Now, it's in textual format. It's a whole table, and we don't know, um, if it's going to fit in the vector store as is. And to make it fit, we need to embed our data. We need to convert it from textual to vector representation. For that, we need to add an embedding model, which you can choose any of them, including Hugging Face, Mistral, or Llama. I'm going to use OpenAI here, because, just why not.
And the second thing which you need is a function which is going to load your data and extract it field by field, because as I said, I don't want to encode all data in the vector representation. I'm interested only in descriptions. So, we're selecting a default data loader. And here we want to load specific data. So, we can even do the mapping directly. It's pretty easy. And here I can just carry this field, and it's going to be embedded as the vector. Also, I would like to save some metadata, so when we're retrieving our vectors from a database, we can understand in human readable form which films we retrieved. So, for that, I would like to save, let's say, a movie name, which we are going to call a movie name. Then I would like to say, for example, a year of release, movie release date. And one last would be nice also to get the description, which we can send to our LLM, and then it can provide, without hallucinations, data to our user. So, let's add the description.
And now the only one last thing left is to add our text splitter. What is that? Well, imagine if we have not short movie descriptions, like in our case, but big, big documents. Our model, our embedding model, always has a context window which is limited. So, if you will try to push to it a whole document, it's just going to select and transform to vector only at the beginning of it, and then it's just going to discard the information. We don't want that, because we want everything to be encoded properly. That's why we chunk our data. There are many, many different ways, and you can read in our education materials how it's done. But in general, here I can just select this simple splitter, because our model, our OpenAI model, has a very big context window of 8,000 tokens, which is around 6,000 words, and our movie descriptions are actually pretty short. Look at them. So, we should be all set up.
But since I don't want to do all 1,000 items, let me actually show you in real time how it will look on more limited data. So, for example, let's upload 30 vectors, 30 movie descriptions to our database. Let me add it directly here and connect everything. And now we should be able to see that here in our UI, we will see a new IMDb collection. So, let's test it. And you can see how they are adding in real time here. If you will go to our UI, you will see in real time how the points are added. More, you can go inside and you can check how the uploaded movies look, their dates, their descriptions, and even look at the similar points. So, if you will click, you will see that I'm on our 30 points. The closest film that we have to The Lord of the Rings is actually another Lord of the Rings, which makes a lot of sense. Moreover, if we are going to work with the whole collection, because I already uploaded all 1,000 points, it's going to be even better. If you would like to see what does mean the closeness of these movie descriptions, then you can go to visualize, and we have that feature, which is going to show you films depicted in 2D space. And then you can see how their descriptions are close and what does it mean. For example, we see these three points here, and we can see that it's Harry Potter, another Harry Potter, and another Harry Potter movie. So, it makes a lot of sense. Descriptions should be similar, and that's why these points are similar in the space.
So, as you can see, everything ran and everything works correctly, and you can do the same, but with 1,000 items, with a different embedding model, or with a different loading strategy. Now you have all of your data in your Quadrant Vector Store, and you can do the second part. The second part where you're going to retrieve the data by our recommendation and provide it to the user.
We want to do the second part where we already building a chatbot which is going to retrieve our recommendations based on the positive and negative opinion of a user of the film that they would love to watch. So, how do agents look in N1? They look like this, and it's a tools agent in our case. So, we want to call a tool, and our tool is going to be a workflow working with our database, Quadrant, and its recommendation API. Uh, and basically, we don't want anything but just to give it a system message. And this system message, I'm going to copy right here. So, it's going to be a movie recommender tool using a vector database, which will provide top three movie recommendations returned by the database, ordered by their score, but not showing the score to the user, not to make the user conflicted.
So, this agent works immediately when we send a chat message to it. So, basically, as a user, I don't see anything. I just ask, "Hey, give me a movie about vampires, but you know, nothing too sweet." As a chat model, we're going to use OpenAI once again, but as I said, it's not limited to that. Let's select something super crazy, super late, like GPT-4. And that's basically it. And here you could also select a memory. A memory is just, for example, here we're selecting window buffer memory, is to remember the context. So, the user in the chat can see the previous five messages, and also the agent can use retroactively some information which was already asked.
The last and the most important thing for our tool agent is the tool. Here you can select different tools, like some code, or some call calling some APIs, or maybe even Wikipedia. But here we want to create a workflow which is going to call our recommendation API in Quadrant Vector Store and doing some magic. We are going to receive our movie recommendation. So, let's create this workflow right inside, right on the same canvas. What do we want from that tool? So, it's going to be a movie recommender tool. And the description, which I'm also going to copy now for convenience, we just want to call this tool to get a list of recommended movies from a vector database.
So, the only thing that we want to do is to create a workflow. Here I'm going to create it in the same sheet, and you are going to see how it's going to look. So, let's copy. And this workflow is going to exactly call Quadrant, its recommendation API, provided with a positive and negative examples extracted from the chat with the user. And our recommendation API works in the way that when you provide it a positive and a negative example and a strategy to aggregate them, then it's going to return you the closest point, which is going to give you the recommendation, which is the closest to the positive and the furthest from the negative example of the user.
So, we know that it's, we are going to build it on the same sheet. And the only thing that we want to be sure is to pass the right information to our workflow. And that's can be done with specifying the input schema, schema for our tool. And we would like to do it from JSON example. So, our agent, our LLM, is going to know how to convert user's input, which might be rusty, might be anything, we don't know. Users don't define everything in JSON's way. They're just using natural language. We want our tool to take this natural language and convert it to the input which we can easily submit into our request to our recommendation API. Here I already created it, so let me copy it, and we're going to look through it. It's going to be a JSON which our LLM is going to create with a positive and a negative example in the proper properties. And a positive example is going to be an example derived from the chat of the movie recommendation that user desires. And negative example is going to be exactly what user is asking not to provide to them.
So, now we have everything defined. And the only one thing is left is to actually do this tool. So, we want to create a tool which is a workflow, the same sheet. So, let's just find our workflow trigger execution. So, it's going to be called when our agent calls it. So, we already specified for the agent what input it should be, a positive and a negative example. But for the sake of testing, let's provide a mock input data. Our mock input data is going to be considered of the agent's query, and our positive and a negative example. Here, as a positive example, I'm setting up a romantic comedy. Remember it when we are going to test our data set retrieval, our recommendation API from Quadrant. And as a negative one, we're using horror, bloody movie. Okay, we're set.
And now we can already work with our positive and a negative example. However, now they are in textual form, in the form that our user is going to provide us in the chat, and in the form that our agent is going to convert it to JSON. However, in our database, we actually have vectors, and our vectors are in a mathematical representation. So, we want to convert our positive and a negative example in the same mathematical representation, with the same model that we used here when we were inputting all of our data in our Vector Store. In N1, there is no direct way to do that. However, you can do it through API of the models. So, we are using this block. And here, of course, at the first side, you think, "Okay, that is such a pain. I need to figure out what is what." However, you can just go and check this tab. This is the API of OpenAI. And actually, a lot of them usually provide this curl. If we copy it, we can use this magical button where we just import all the data. And now we don't have to specify basically anything. The only thing left is authentication, and what is we are going to embed. And here we have positive and a negative example. So, let's embed it. We don't want any text, because it's going to change our mathematical representation. Let's test how it's going to look. And now we see that our OpenAI model actually embeds our positive romantic comedy example. Here you need to be sure that you are choosing the same model that you used when you were inputting the data to the Quadrant. Let's do the same for the negative example. Where are we? Mmm-hmm. And success.
Now we would like to throw all the data away that we don't need, so we don't pollute our pipeline, because as you remember, here we have a lot of data provided. And then we would like to combine them. However, let's start with the beauty fine. And here we just interested in the embedding field, nothing more. Let's call it positive example, positive example. And we can see that now we can work directly with the embedding. Let's do the same with the negative one. To be sure, let's check that we are mapping the right field. And just in case, even though it's not needed, I will remap it. And now we can see that we also get all the needed data without anything excessive.
So, now we want to combine them. Why? Because in our API, we send in one request both positive and a negative example. Combination is done with this block. Let's connect both of the inputs. And it's basically a join in databases, I think everybody heard about this. But here we just want to combine them without any field being based on, because we actually don't have anything to combine them on. We just want one item out of two. And then we can see that we have positive and negative example as one item together.
Well, it's time to call our API. Yet, in N1, we can't use Quadrant Vector Store recommendation API directly with that block. However, it doesn't stop you from using any features in Quadrant, including recommendation API and all others, like hybrid search, or for example, multi-model search. It's just with the help of the same favorite block of ours, which is called HTTP request. And now we can work with these points, and we can work with our API. Let's see how to approach this. Firstly, we see that it's a POST method, so we can do the same here. Secondly, we need to find the URL of our API. We are working with our particular cluster, so this is our address of the cluster that I was showing you before. And then we can look directly in how our API method is looking. So, we're firstly using the collections, and then we are providing the name of our collection. So, in our case, it's collection called IMDb 1000 OpenAI. The only one thing left is to call the exact method for this collection, and this is points query. So, now we have the right address to call. We need to authenticate, obviously, and send the right body with our positive and negative example. Let's predefine it using JSON, which we can take from our documentation here. Here we have filter. Filter works directly with metadata. However, we don't want to specifically filter any metadata. If we wanted to, we could, for example, select only films of a certain year, or with a certain traffic in the name. But let's simplify the task and just directly use our API. So, we have here query, recommendation, positive, negative example. It can be several of them, not only one, and a strategy of aggregation. We have two strategies, and I will try most certainly to provide in a post information where we describe what is the difference between these strategies. However, let me use this one. So, we want to provide our positive and negative example. Let's just drag and drop them. Since it's a vector, we need to be sure that we are passing a vector. The same here. Also, as you remember, I stated for the re agent that we are going to provide top three results of the recommendation. So, for that, we would like to limit our API to returning exactly three points. For that, I'm going to use limit three. Let's see what is our database going to return. We have three points, and we already know that it's exactly the recommendation for romantic comedy without anything bloody in it. However, we can see anything about them. We can see only that they exist. So, we need to fetch some metadata. And for that, we will have to use again our API. This is the retrieve points API. Since we can't yet do it from N10, let's do what we know how to do. Add an HTTP request block. And here we're also using a POST method. And we can already correct our long and scary URL and adjust it accordingly. Here we're working just with points, so we need to be sure that we are deleting the query postfix. Now, don't forget to authenticate and send a request. Once again, JSON. And once again, we have something to use as an example. So, here we have IDs of our points. And if I want to be only using drag and drop, I can do a little bit ugly but simple thing. Give the strings of our IDs directly like this. Pardon me, we need to select expression for that. And here we're dragging the second and the third one. Also, one little thing which is important is to add option with payload. What is payload and why? What do we need to add it? Payload, it's exactly the metadata in Quadrant. And usually, when we query points, we query them only with vectors. So, we want to add also our metadata, because that's exactly what our LLM is going to use when producing for us the result without the hallucinations. So, let's see. Now we can see actually what were the top three films recommended being a romantic comedy and not a bloody movie. And it is I Remember, and two other movies, which are also looking pretty romantic.
Okay, so basically, we have everything that we need for our agent. But I want to be a little bit picky and also throw away all the data we don't need. Plus, I promised the agent to provide it with a score of movies, so it can order them. So, here we have a score. And here, when we already requesting for the points, we kind of throw it away. So, let's retrieve it and combine all the needed information. For that, let me firstly split out the points. Here we're splitting them by the result field. And here we are splitting them by points field. Now we can combine them by ID. So, let's see. Let's map and let's merge. Now we have actually a matching field, it's called ID. And we can see that now we have both ID, score, and payload. Let's add a couple of details like cherries on the top. Remove all the not needed information. I would love to keep on this score, which is movie recommendation score. Then we want to see description. Then we want to see movie name, which doesn't want to copy. And that's okay, because that's how you know, this is not an AI generated demo. So, movie name and release year. Here we see now that we have only the things that we need. And since our agent expects everything in one field, which is called response, let's merge them back in one list. Well, we don't want anything separate. We just want to throw them in one list and call it response. Well, that's what our agent is going to see now.
The moment of the truth is coming. Let's see what is going to happen if I'm going to actually request our agent with my recommendation wishes and my anti-recommendation wishes. For example, I'm sitting here and I'm thinking that I really like fantasy, and I would like to watch something about wizards. However, I am living on this Earth already for 27 years. That means that I watched Harry Potter at least 100 times, and adding one to them is going to be kind of ungodly. So, I will ask my agent not to provide me Harry Potter. Let's see. While it's happening, we can see that our agent actually called our tool and retrieved the answer. And our answer will be in the chat. And that's Howl's Moving Castle, Stardust, and Castle in the Sky. Well, sounds pretty wizardy for me, and nothing of that is Harry Potter.
If we would like to see what exactly was behind all of that, we need to go to executions, and then we can actually see that our agent called our tool. And here we can see that our agent was so smart that it provided a positive example, a fantasy film that features wizards, and a negative example, which is a film from Harry Potter series. And then we can actually see what was the output. And the output, I think I'm looking at the wrong execution. I'm sorry. Yeah, here is the right one. Well, it's not looking worse. Here we have a magical adventure involving wizards, and here we have any movie related to Harry Potter. And here is the output, when we can see actually Howl's Moving Castle, Stardust, and Castle in the Sky. And as you can see, they're other similarity scores. So, our agent received this output, and then it combined it in a very beautiful way with our Chat GPT-4 model.
Now you have this at your usage, but you don't have to stop here. Because with recommendations, you can choose any data set, for example, for e-commerce. Moreover, you don't have to work only with textual data, because Quadrant, as any vector store, is data agnostic. Vectors can be images, videos, audios, text, whatever you want. Also, you don't have to work with one modality, you can do a multimodal recommendation. I'm wishing you to experiment more, and if you want to build something together, please come to me, and I'm happy to help.