Use RAG to chat with PDFs using Deepseek, Langchain and Streamlit

So this is our question, and let's wait for the [Music] answer.

So this is the answer. Notice this text inside the think tag. This is how it is showing that the model is reasoning step by step. You can easily parse this string and remove this before showing it to the user, of course. But I just wanted to show you because it's so funny how it is reasoning like a human.

But let's see the answer. The government's prohibition on peaceful assembly violates Article 20, which explicitly guarantees the right to freedom of peaceful assembly and association. And it's even saying that the government has some rights to put some restrictions based on Article 29, but it needs to be lawful and necessary.

So let's go and have a look at Article 20 and Article 29. Yeah, exactly. You can see that the answer is completely right.

Hello everyone, Naran here. Welcome to my channel. Today we want to use the deep seek reasoning model, which is called R1, to chat with PDFs. We will do this by creating a RAG using L chain, and we will also create a pretty neat UI for it using Streamlit.

Our main focus would be how powerful these reasoning models can be when they are combined with RAGs. Currently, everyone is talking about how powerful Deep Seek is. It is surely the most powerful open-source LLM model out there. But some people are even comparing it to OpenAI's OAN model, saying that this is as good as OAN or even better than that.

So we can see today how much of that is true. To run one of Deep Seek models locally, we need to go to the AMA's website. Then you can see here that I'm in the Deep Seek R1's page. It has many different models: 1.5, 7, and even like 671. Based on your computer's resources, you can choose which one you want to pull.

I tried on my laptop like 70, 32, and 14, and for this project, I'm going to use 14 because 32 and 70 were just too slow. So let's go ahead and pull this model. You can see that there are some instructions on how to run each of them. I don't want to run them; I just want to pull one of them.

I'm just going to copy this command, go to my command [Music] line, and instead of run, I will just go and write pull. As you can see, I already pulled it, but for you, it's going to take some time. This will be good enough for the usage of our project. You just need to pull one of these models.

So now let's go and have a look at our project structure. We need Streamlit for building a UI. If you are not familiar with the Streamlit basics, I would suggest you watch this video. We need a couple of Lang Chain dependencies like Lang Chain core, community, and of course, AMA, which has the integration with the AMA models.

If you also want more information and details on what is Lang Chain and what is RAG, I also explained it in this video. But in this project, we will work with it, and I will explain a little bit more.

At the end, we will need PDF Plumber to load PDFs and extract text from it. Lang Chain offers many integrations with different PDF tools. Today, I'm going to use PDF Plumber, but based on your needs, feel free to use the integration for Unstructured or F-Duckling or any other PDF tools that fit your needs.

So I will need to install all these dependencies. I will go to the command line again and say pip install -r requirements.txt, and I will hit enter. It will install all the dependencies for you.

Let's go back to the project. I already created a skeleton for our project, and I will explain it to you now.

So we basically need to create a RAG because LLMs are trained on some data, but they don't know what is the weather today, or what is the latest news today, or what is the information in a certain PDF or a certain website. So if we give them context, we can use their power—in this case, use the power of reasoning—to ask very detailed questions and get very precise answers.

So in this process, we need to do a few things. First of all, we need to upload our PDF. The user will come and upload some PDF in our system. This is what we are going to do in the upload PDF method.

Then we basically need to load this PDF and extract all the text from it. This is where we are going to use PDF Plumber and L Chain loaders to do this task for us. Then we will have a bunch of text, like each page would be a document, like a Lang Chain document.

This is great, but it's still not good enough for our processing because usually LLM models are not that great with long text. So in the next phase, I'm going to split the text into smaller chunks.

After all these pre-processing, I need to index my documents, which means I need to bring them into vector space and then store it inside the vector store. As you can see, I already created an embedding with the Deep Seek model, which is in charge of creating vectors out of text.

Then I also created a vector store. I'm using an in-memory vector store for this project, but you can use any other vector store that you want, like Elasticsearch or Chroma. Lang Chain has integration with many of these.

So then, after indexing the documents, my job for PDF processing is done. Then we would implement a chat, and the user can ask questions. These questions are like a query. We need to hit the vector store and load all the related documents to that query.

This is the job that is done inside the retrieve docs. At the end, we will create a prompt, which will contain also the user question and all the related documents that we retrieved. We will ask the LLM, "Please answer the user's question with this context." This is what is going to happen inside the answer question.

So let's waste no more time and start coding. We will start by uploading the PDF. Notice that I already created a PDF directory variable here, and the directory is here in the project.

So the user will upload the PDF, and we will store it inside this directory. This is like a very basic Python task—just writing to a file. We will say with open, and then PDF directory plus the file name, and then we will write to the file using the file's buffer.

Next, we need to load this PDF. So we are passing already the file path, and then I can just use PDF Plumber loader, which is the integration with PDF Plumber in Lang Chain, and then give the file path. This will give me a loader.

All I need to do now is call the load method on it, and this gives me a list of documents. If I'm not mistaken, each page of this PDF now is representing a document, which is just a Lang Chain class that wraps the text.

So all the extraction is happening under the hood. So that's it—very basic. I just need to return the documents.

So next, I'm going to pass these documents to be split into smaller chunks. I'm going to use Lang Chain's recursive text splitter here. So I will say, "Yeah, recursive character text splitter," and then you need to define the chunk size and everything.

I would say the chunk size 1,000 looks good. Yeah, let's actually use the default that is being suggested by Copilot: chunk size 1,000, overlap 200, and add a start index true. That is a good default.

This is my text splitter, and I will ask the text splitter to split documents. Notice that we have the documents passed in the method, and this will again give me a list of documents.

For example, if you have a PDF of 10 pages, first you have a list of 10 documents, and then you will give it to this split document method. It will split it into smaller chunks, and now probably you have a list of 100 documents, let's say.

So I can just return this and be done with it. For indexing documents, I can use our vector store and then the method which is called add documents, and then pass all these docs.

What this method will do is that it will use this embedding to move all these documents to vector space, and then it will also store it itself.

So for retrieving the docs, I'm going to use the similarity search. This method will take the user query, which in this case is a question, and it will retrieve all the related documents.

So of course, we will need to use our vector store again because all the documents are stored there. There is a method called similarity search. I will pass the query and return this. This way, we have all our related docs.

At last, when we want to answer the question, we need a prompt. So I actually copied the template from the Lang Chain website. Let me create a template like this, and let me just copy and paste it here so I can show you.

I will paste it here. Yeah, it's saying that you are an assistant for question answering. Use the following pieces of retrieved context to answer this question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.

So the question will be here. This is just the template; we will add the data later, and the context would be here, and then we will get an answer.

So I will use Lang Chain's chat prompt template, this one, and then create from this template. Now we have a prompt.

Then I need to create a chain with my prompt and my model. Notice that I already defined my AMA model here with the model dips, so I will just do the chaining. It's like prompt and then pipe and then model. This will be our chain.

One thing that I need to do is to create the context. Notice that I have separate related documents that our vector store thinks are related to this query, and here the prompt wants from me a piece of text combined together as a context.

So I'm going to just pass here the question and also the documents. This is not the context. What I can do is basically join all these documents together, maybe with two new lines to separate them.

I can join all the page content of these documents, and now I have one piece of text that I can pass to the LLM. So if I invoke this chain with the question and the context like this, in response, I will get the answer that I wanted.

So let's return this one. So that's it—that's our logic. Now let's go ahead and build our UI, which is pretty simple with Streamlit.

I'm going to use Streamlit file uploader, and as a label, I'm going to say, "Yeah, upload PDF." The type I'm going to restrict it to PDF. I'm going to also add accept multiple files as false because we want to process one PDF at a time.

This will give me an uploaded file. Let's also break this down. So I'm going to say if there was an uploaded file, first upload this PDF.

Yeah, then I need to load this PDF and make documents out of it. For the path, I'm going to say PDF directory plus uploaded file.name. Yeah, like this, and this will give us a bunch of documents.

Then I need to split these documents into smaller chunks. I will use this. So now I have all the chunk documents, which I can index easily.

Index docs, and then documents will be passed to it. So now the indexing part is done. I need to create a chat here. I will say a Streamlit chat input. I don't need any label here; I will just get the user question here.

So after the indexing is done, we are creating a chatbot so the user can ask the questions, and then we will show them the response.

So if there was a question, we need to retrieve the docs from this user query, and these are the related documents. After we have related documents, we can answer this question using the user's question and the related docs, and this will be our answer.

So the question is how to show it in Streamlit. Now here, when there is a question, I can say Streamlit chat message, and then I am writing as the user.

So I will specify the role, and then there is an answer from the bot itself. I'm going to say chat message from the assistant. The role is called assistant, and then I will write the answer.

That's it—very basic. So yeah, that's it. We can run and see how this RAG is working.

Let's go to our [Music] terminal, let's clear this, and I will say Streamlit run PDF R.P. That's it. Let's run it.

Let's go to our browser, and as you can see, we have a pretty minimalistic design. Just upload your PDF, and then we will show you the chat.

What I want to upload is the Universal Declaration of Human Rights. It has all the articles regarding human rights, and it's not that big; it takes lots of time to upload and process. It's like eight pages, and I don't want to just ask very simple questions like, "What is Article 29 saying?"

This is a reasoning model, and I want to show you how powerful it is. So I will ask questions that are regarding comparison and reasoning.

So let's upload it first. I will go here, and I will drag it. It will take some time, of course, because it's processing eight different pages. Let's wait.

And as it is done, you can see that there is a chatbot appearing here that I can ask my questions.

So let's ask some hard questions that need reasoning. I'm going to ask, "If a government forbids the right to assemble peacefully, which articles are violated and why?"

Let's ask this question. So this is our question, and let's wait for the [Music] answer.

So this is the answer. Notice this text inside the think tag. This is how it is showing that the model is reasoning step by step. You can easily parse this string and remove this before showing it to the user, of course.

But I just wanted to show you because it's so funny how it is reasoning like a human. But let's see the answer.

The government's prohibition on peaceful assembly violates Article 20, which explicitly guarantees the right to freedom of peaceful assembly and association. And it's even saying that the government has some rights to put some restrictions based on Article 29, but it needs to be lawful and necessary.

So let's go and have a look at Article 20 and Article 29. Yeah, exactly. You can see that the answer is completely right.

And here also, there are some indications that in a democratic society for public order, you can impose some restrictions.

I'm going to ask another hard question which requires reasoning. I'm going to ask, "Can you tell me about a situation where freedom of speech is in conflict with another article, and what guidance does the Declaration offer for that situation?"

Let's see. So this is our question. Okay, the answer is ready. This is just the thinking phase. Let's look at this.

So first of all, it understood that freedom of speech is under Article 19. It's saying that it can conflict with Article 2 and Article 5, and Article 29 provides guidance on that.

I guess we had a look at Article 29 in the previous question. It was saying that the government can impose some restrictions, and we can just take a look at Article 2 and Article 5.

It is saying that, for example, in Article 5, no one should be subjected to torture or to cruel treatment. So looks like the answer was more or less correct.

So yeah, I guess that's it. I hope you enjoyed watching this video. As you can see, combining RAGs with models with reasoning can be really powerful.

And then it's just not simple Q&A, but you can use even comparison, find contradictions, and stuff like this. Please like this video and also subscribe to me for more content like this.

See you around!