Transcription
Hello everyone. Many who have worked with chat GPT or other LLMs would like the neural network to be excellently versed in the materials they have. These can be some corporate documents, it can be the chat history in corporate chats, where it's impossible to find anything, or even some public materials, which are simply difficult to search through. An LLM can easily summarize and provide an answer. The method that allows an LLM to search for information in your documents, summarize it, and provide it, is called RAG. I won't go into translation details, but essentially it's generation augmented by search in a document database. To create a bot that will use your data, you essentially need a database that the search will work on. Today I will tell you how to create your own chatbot that will search your documents using Nathan, a low-code system for creating agents, and the Flow service, where I have now added the ability to use a vector database right at the start. We will do this using open documents as an example. I will take a Russian chat about KTM motorcycle repair. Many who ride enduro know about them. And we will create a chatbot that will answer based on a huge database. 80 thousand messages. Users who shared how they found problems and solved problems with their favorite motorcycles. Let's try. So, first of all, we need to download the data that our agent will use. I will download the KTM repair and maintenance chat. To do this, go to the chat, click the three dots, export history. We won't download photos, the size is arbitrarily large. Format read. Okay. Export. All. We wait for the data to load. I uploaded it to Google Drive. From here I will share it. Share, open access. Everyone who has the link. We copy the link, go to it, click download. And here appears the link that we will need for downloading within Nate. Let's go create the workflow. So, we create a new workflow. The first thing we need to do is load the data into our pgvector. Click man, our first workflow, which we will simply run. What we need to do is download the data. For this, we need a node. We insert our link here. I did, however, take a link to a shortened file, just so everything loads faster for me. And we add a transformation node from file to JSON. It's called extract from file. We will extract from JSON. We run it. Excellent. It worked. Here are the messages in the chat. In fact, what do we need from this? We only need the text and the message ID. Let's make a code node that will do this. We copy this data, open bot, and write code. Write, write me code for NEN that will leave only two fields from all this. Text and message ID. Remove messages of incorrect types. We insert our JSON. Click Enter. Let's go. We wait. We copy the code. We insert it. We go here. We insert the code. [music] We insert. We run it. We look at the answer. ID text. That's what we needed. We save this. Now we need to load all this into pgvector. To do this, we add a PGVector node here. Select Document to Vector Store. We can leave everything else untouched for now. Well, maybe let's change it to 2 hours. Everything loaded for us at once. We need to select embeddings. Embeddings are a way of converting text into vectors. Essentially, vectors that are closer to each other mean more similar text. This is also done using neural networks. I won't go into details now. We need to choose. We will use embeddings from OpenAI. Unfortunately, Nate cannot use pgvector out of the box for this, so I entered my OpenAI accounts and will use it via the OpenAI key. We also need a way to transform documents to load them. Select default dataloader. And we need to add one field to the metadata. This is the message ID, so we can refer to it later. We drag the message ID. [music] We close it. And here we just need to create a table. Let's call the table KTM. We save it. We run it. So, everything has loaded for me. Now we need to make it so that our Nate can extract data, i.e., search by a message string from our chat. To do this, let's create a new trigger. Click plus on trigger and when executed by workflow. Let's set a parameter. We will, in fact, search by string, so we set QL, type string, and set mock data. For example, why isn't this starting for me? Let's delete everything unnecessary from here. We save. We save. We save. And now we need to add, in fact, the search. To do this, we add PGVector GetRank Documents. Our table is called KTM. Let's say we have four documents. Ah, we add. Yes, that's all we need here. We just need to add OpenAI again. Why? Because at the moment when Nate searches, it will first calculate this embedding vector for our query, and among all vectors, it will find the most similar one. We save, we run. Ah, I forgot to add the prompt. The prompt, of course, we will take from our workflow. We run it. So, what do we see? What kind of? When you start the bike seven times in a row, your peak current drops. In general, yes, something similar to what we need. Excellent. Now we need to create an agent that will use this information and give us an answer based on our messages. To do this, we add another trigger, on chat message. Let's add an agent here. Let's give it a nice system message right away. You are a KTM repair specialist. Give a detailed answer. Search for information in the KTM repair chat. Provide proof from the found messages with links like. And let's add any link in the KTM chat. We save. We add nodes. We will use Open Router models. Let's take, say, GPT41 Memory. Let's use simple memory here for now. It's not critical for us, although it's better to use logs. And, in fact, the tool call workflow tool. We select our workflow. We provide information here about what it is. Search in the KTM repair chat database. Query. Query. Let the neural network determine it itself. Let's give a description of what needs to be searched in the KTM message database. We save. We open the chat. Let's try. What to do if the motorcycle doesn't start? [music] I recommend doing the following: check the battery. Here is a link to the message. Here is a link to the message. It seems to be working. What improvement can we make? Now our agent only looks at one message it found. But perhaps messages in the chat have context: the message before, the message after, and it's better for us to use several. Let's fix this so that along with one found message, we find five, for example, before and five after. Let's go back to our previous workflow and make a direct selection from the database here. Let's add a postgres node. [music] execute SQL sequel or SQL, yes? This is a query language for databases. In general, it's very useful to know when you work with data. You can also write it using Claude, but since I know the syntax more or less, I'll try to write it directly. Let's write out the previous nodes. So, I'll write select star for now. This means to extract all data from commit 1. This means one row. Let's see what's there. Here I have ID. ID is some internal ID. In metadata, there is the actual message ID, text, and the entire history. And I have text information here. Accordingly, I extract meta, i.e., convert the ID to a number, because it's stored as a string between the ID that came to me minus 5 and the ID that came here. We put a plus here. Let's try. I made a syntax error. We need quotes here. So. And we don't need all the fields. We only need ID and text. And we remove the limit. Now our agent has context. We save. Let's try again. [music] Why doesn't the motorcycle start? We see it fetched various data and brought back quite a few messages. So, what have we achieved? We've created a RAG that fetches information from a chat where information on how to actually repair KTM motorcycles has been accumulated for many years. It takes information from there, processes it using a neural network, and gives it to us. Now, if something breaks, it's much easier to write there than to search through manuals and rummage through this chat. And this is not just an example. This task actually exists. My good friend asked me to do such a thing. You can use this for your corporate database, for any tasks where you search for information from specific sources, and it will give you both the correct answer and a link from where the information was extracted. Good luck. Automate your work with LLMs. Bye everyone.