LangGraph + Deepseek R1: How to Build a Local AI Chatbot

Hello everyone, my name is Aruhi and welcome to my channel.

So guys, in my today's video, I will show you how you can use LangChain and DeepSpeed R1 model to build a chatbot.

First, I will show you how to download this DeepSpeed model, and then we will use that downloaded DeepSpeed R1 model with LangChain to build a chatbot.

So let's start.

So guys, if you have OLLAMA already on your PC, then don't download it again. But if you don't have OLLAMA installed on your PC, then click on this download.

When you click on this download, after that you will see the setup in your downloads folder like this.

Double click on it and follow the on-screen instructions, and then you will have OLLAMA installed.

Once you install OLLAMA, just open Command Prompt and here type OLLAMA.

And if you are seeing this kind of screen, that means OLLAMA is successfully installed.

Now the next step is to see here DeepSpeed R1. Click on it and you will be redirected to this page.

And here guys, click on this arrow and you will see the different variants of this model.

And this is a 1.5 billion parameter model, and the size of this model is 1.1 GB.

And then we have a 7 billion parameter model, and the size of this model is 4.7 GB.

In the same way, these are the different variants, and this one is a 671 billion parameter model, and the size of this model is 404 GB.

And this is the biggest model of this DeepSpeed.

But in our today's tutorial, we are going to use this 7 billion model, and the size of this model is this.

And to use this model, you will copy this from here.

Just copy it from here, go to your Command Prompt, and then paste it here, and then hit enter.

In my case, guys, I have already executed this command, so I have already downloaded this DeepSpeed R1 model.

So in my case, what will happen? It will start the chatting directly, means we can chat with this DeepSpeed R1 LLM directly.

But if in your case, if you are running this command for the first time, then it will take time to download this model first, and then you can chat with this model.

So let me run this command.

So as I told you, in my case, I have already downloaded this model, so I can directly chat with the model.

So let's write "Hi."

So when I wrote "Hi," then this is the reply from the model.

So this model is successfully working on my PC.

Now let's see the next step.

The next step is to use this model with LangChain to build a chatbot.

So whenever you want to create any project, the first thing is to install the requirements.

So that is what we are going to do now.

We will install all the requirements now.

So guys, before that, let me show you the Python 3.8.11.

The first thing is to install LangChain, LangChain Community, and LangGraph.

I have already executed this step, so in my case, the requirements will be satisfied.

So first, we are installing this LangChain, LangGraph, and LangChain Community.

The requirement is already satisfied because I have already performed this step.

Once you install LangChain, LangGraph, and LangChain Community, the next step is to install LangChain OLLAMA.

Write it like this and then hit enter.

Requirement is already satisfied, so that means we need to run our today's code.

Now we can open Jupyter Notebook and start working.

So for my today's tutorial, I have created this demo folder, and now let's work inside this folder only.

Go to this new Python 3, and you have a Jupyter Notebook.

First, let me show you LangChain and LangGraph version.

See, the LangChain version I am using is 0.3.1, and the LangGraph version is this.

Now let's write this code to get the LLM first.

So I am just importing LLM from this module, and this is how you can use DeepSpeed R1 model.

So how we are using it? We are getting the DeepSpeed R1 model from OLLAMA LLM.

So that means we have DeepSpeed R1.

Now let's write a code to develop a chatbot using LangGraph.

And first, I am importing to create the state.

State is a memory of the chatbot.

So for that, we need these two modules.

And guys, if you don't know the basics of LangGraph, then please check this video and understand what is state, what is node, what is edge, what is state graph, and then how these things work together.

Then please check this video because in my today's tutorial, I am not going to cover the basics.

So this is the next module here.

Start and end means start node, and node state is a state machine.

Using this state graph, we will create a state machine, and this we have already imported in this cell over here.

So I am not writing it here again.

So using this class, we are creating a state.

State is the memory of a chatbot where the chatbot will store all the previous messages of the user and a bot.

So this is how you create a state dictionary, and this class will give you the list of dictionary.

List of dictionary simply means whatever the user will type, that will get stored in a separate dictionary, and then both will reply to that question which the user asks.

That thing will be stored in a separate dictionary.

So like this, here you can see for each message of user and bot, we have a separate dictionary, and all these dictionaries are present in a list.

So this is what this list of dictionary means, and this is how you are going to create the memory of the chatbot.

Now the next step is to initialize calling that state graph.

I am initializing to answer the user's query.

Now let's create a function which will perform the chatbot task.

The chatbot task is it will take the user input, it will run the LLM on it, and it will give you a response.

So we will write all that task in one function, and this is the function.

See here, this is a simple function.

Function name is chatbot, and this state means this state holds all the previous messages.

And in this function, we are calling the LLM on the message which the user asked, and we are storing the response in the response variable.

Then we are appending the state messages, the messages over here.

Right, we are appending the messages with the response which chatbot gives.

The role assistant content response, the response generated by LLM, we appended this response in the messages list.

So this is how every time when the user will ask any question, then the bot will reply.

All those messages will be appended in this state messages.

Right, so this is a function.

So the next step is to see we are using LangGraph.

So in LangGraph, for every task, we use nodes, and then we use edges which tell the connection between those nodes.

So for our today's task, also we need to use those nodes and edges in order to make a graph.

Right, so how our graph will work?

So first, we will have a start node.

Start node will start the work.

Then there will be a chatbot node.

Chatbot node will perform, take the user question, and will give us a reply.

And then we have an end node which will end the task.

So this is how we can write that graph builder.

So here, this is a graph builder, and here we have a state graph.

This state graph is acting as a state machine.

State machine simply means it controls the flow of information between the nodes.

Means it controls the overall process, means after which node what should be the next node.

Means the output of which node should go to which node, like this.

So first, we are adding a node.

Which node we are adding?

The name of the node is chatbot, and the task, what task this node will perform, the task which is mentioned in this chatbot function.

Okay, so we need only one node because we have only a basic function chatbot function which will answer the user's question.

So we have only one node.

Then graph_builder.add_edge means we want to establish the connection.

Right, start node.

First of all, we will have a start node.

We are creating the connection of start node with chatbot node.

This node.

So guys, start node and end node, we can take it from here only.

So we are not creating it manually.

So start node is connected with the chatbot node, then chatbot node is connected with the end node.

So this is how you can create the graph.

Now to run the graph, you first need to compile it.

So to compile the graph, this is how you can compile: graph_builder.compile.

Now our graph is compiled.

Next step is to run this code.

Now let's see how to run this code.

Guys, this is how you can run the code.

User will ask the question, this will give you an input box where user will ask a question.

If user typed "quit" or "q," then the process will be finished there only.

If not, if user types something else than that, user input will go to stream_graph_update function.

Now let's see what this stream_graph_update function does.

We have this stream_graph_update function over here, and the parameter we are providing to this function is user input.

Whatever user typed will become the input here.

So let's say user typed "Hi."

So we have "Hi" over here, and then we are storing the user message into the state, into the chatbot's memory like this: role_user_content = user_input.

And if user typed "Hi," so we will have "Hi" over here.

Okay, and then when we updated, when the state is updated, that state will go to the graph.

So remember this graph over here, what we have in this graph.

In this graph, we have defined how the process should flow.

So first, the work will start with the start node, and then chatbot node will be used, and then end node.

So graph.stream, and whatever is there in the memory of a chatbot in the state will be the input to it.

Okay, so now user message "Hi" goes to this graph and start node, and then we have chatbot node.

And for chatbot node, this function will perform to "Hi" message.

Here LLM, we are providing that "Hi" message to LLM.

LLM generated the response for "Hi," and then the response of chatbot is getting appended in the state.

Okay, and then we are showing the response on the screen to the user.

So this is what's happening.

So let's run it.

We have this text box.

We got this text box because of this input.

Now let's write "Hi."

So when we wrote "Hi," so this is the reply from our chatbot DeepSpeed R1.

Reply: "This pack."

And then now let's ask another question: "What is GGI?"

Now this is the reply of the chatbot again.

So guys, this is how you can use LangChain with DeepSpeed, and I have given the link of this code in the description section.

You can get this code from there.

And guys, I hope this video is helpful, and if you like my content, please like, share, and subscribe to my channel.

Thank you for watching.