Transcription
Today, I'll be showing you how to make a Python AI chatbot in just a few minutes. The best part is that this chatbot will run locally, meaning we don't need to pay for a subscription or connect to something like OpenAI.
Now, the first step is to download Olama. Go to olama.com, simply press on download, install this application, and this is what we'll use to run LLMs locally on our machine.
Once you've installed Olama, the next step is to make sure the installation is working properly. To do that, open up a terminal or command prompt and type in the following command: `olama`. This should work if Olama is running on your computer. Assuming this command works, you're ready to move on to the next step.
Olama is software that allows us to download and run open-source LLMs. I have a list of them on the right-hand side of my screen, and I'll link this in the description. You can see we can access Llama 3 with 8 billion parameters, 7 billion parameters, and many other models available to us as well.
Notice that these models vary in size, and the larger the model, the more difficult it will be to run in terms of the type of hardware you need on your machine. If you scroll down, you'll see some different specifications. You should have at least 8 GB of RAM to run the 7 billion parameter models, 16 GB to run the 13 billion, and 32 GB to run the 33 billion models. So, make sure you meet that criteria before pulling these specific models.
For our purposes, we're just going to use the Llama 3 model with 8 billion parameters. To get that on our computer, we're going to type the following command: `olama pull llama 3`. This will download the model for us. This will take a bit of time depending on how fast your internet connection is. In my case, it's already downloaded, so it happened instantly, but for you, it will take a few minutes to download. Then, I'll show you how we can test it.
Now that the model is downloaded, we can test it out. To do that, we can simply type `olama run llama 3`. It's worth noting that you can have multiple models here, and you can use them by simply specifying their name. So, I'm going to run Llama 3. I'll just go with something like "Hello, world," and you can see that it actually gives us a response. If you want to quit this, you can type `/bu`, and then you will exit that prompt.
Now that we have access to this LLM, it's time to start using it from Python. Using LLMs like this in Python is surprisingly easy, and you don't need to be an AI expert to work with them. However, I always find it interesting to learn about how they work at a deeper level, and that's where our sponsor Brilliant can help.
Brilliant is where you learn by doing, with thousands of interactive lessons in math, data analysis, programming, and AI. They use a first principles approach, meaning you'll get the why behind everything. Each lesson is interactive and filled with hands-on problem solving, which is six times more effective than just watching lectures. The content is created by award-winning teachers, researchers, and pros from places like MIT, Caltech, and Google.
Brilliant focuses on improving your critical thinking skills through problem solving, not memorizing. While you're learning specific topics, you're also training your brain to think better. Learning a bit every day is super important, and Brilliant makes that easy. Their fun, bite-sized lessons fit into any schedule, helping you gain real knowledge in just minutes a day. It's a great alternative to mindless scrolling.
Brilliant even has an entire AI workshop that deep dives into how LLMs work and teaches you about the importance of training data, how to tune an LLM, and more. To try everything Brilliant has to offer for free for a full 30 days, visit brilliant.org/techwithtim or click the link in the description. You'll also get 20% off an annual premium subscription.
Now, the next step is to create a virtual environment where we'll install a few different dependencies we need for this project. I've opened a folder in Visual Studio Code, and the command I'm going to use now is `python3 -m venv chatbot`. If you're on Mac or Linux, this will be the command. If you're on Windows, you can change this to `python`, and this will create an environment for us with some isolated dependencies inside. You can see the chatbot folder has now been created.
The next step is to activate the virtual environment and then install our packages. If you're on Mac or Linux, the command to activate this environment is: `source chatbot/bin/activate`. If you did this successfully, you'll see that the name of your virtual environment will prefix your terminal.
If you are on Windows, the command can vary depending on the shell that you're using. One of the commands you can attempt is: `chatbot\Scripts\activate.bat` for Command Prompt. If you are running in PowerShell, then you can change this to `chatbot\Scripts\Activate.ps1`. Try one of these commands to activate the virtual environment on Windows, and again, make sure you have that prefix before we move forward.
The next step is to install the packages that we need. I'm just going to make this a bit smaller so we can read all of them. We're going to install the LangChain module, the LangChain Dasama module, and the Olama module. Go ahead and hit enter. This will install it inside our virtual environment, and then we are good to go to start creating this application that will use our local LLM.
The next step is to simply create a Python file. We can call this something like `main.py`, and now we can start writing our code. To interact with Llama, we're going to say: `from langchain.llms import Olama`. Now here, we can connect to Olama. We can say that our model is equal to `Olama()`, and then all we need to do is specify the model that we want to work with. In this case, it is Llama 3, but if you have a different model, you can put that here.
Now we can actually use this model. To use it, we can say `model.invoke()`, and then we can pass to this function a prompt that it will act upon. You can see that I can pass some input. Say `input = "Hello, world"`. We can store this in some kind of result and then simply print this out to make sure that it's working. So, let's print out the result and execute our code.
From my virtual environment, I'm going to type `python3 main.py`, and notice here that we're going to get some warnings. You can ignore that for now, and you can see that we get the response, which is "Hello there, it's nice to meet you." We've invoked the model with this input.
That's the basics of interacting with the model, but I'm going to show you a slightly more complicated script that explains how we can pass some context to the model and how we can make this a little more user-friendly, creating a full chat window interface to interact with it.
To do that, we're going to start by bringing in something known as a prompt template. So, I'm going to say: `from langchain.prompts import ChatPromptTemplate`. LangChain allows us to more easily interact with LLMs, and one of the things we can do is create a template that we will pass to the LLM, containing our specific query or prompt. This way, we can give it more description and instruction on what it should do.
I'm going to say `template = """Answer the question below. Here is the conversation history: {context}. Question: {question}. Answer:"""`. Whenever I want to embed a variable inside of a prompt that I'm passing to the model, I can surround that in curly braces, which I'm doing here.
Now we have our model. The next thing we're going to do is create our prompt. We'll say `prompt = ChatPromptTemplate.from_template(template)`. Now we have a prompt and a model, and we need to chain these together using LangChain.
I can say `chain = prompt | model`. This will create a chain of these two operations. The first thing we'll have is our prompt, which will have the question and context embedded inside, and then we will pass it to the model where it will be invoked automatically.
To use the chain now, we can change this slightly. Rather than `model.invoke()`, we're going to say `chain.invoke()`. This time, we need to pass the various variables that are inside our prompt. We'll say `context = ""` (we don't have any context, so we'll leave it blank), and then we'll say `question = "Hey, how are you?"`.
Let me make this a bit bigger so we can read it. We're simply embedding these two variables inside the prompt and then passing that prompt to the model where it will be invoked using LangChain.
Now we can test this: `python3 main.py`, and it says, "I'm doing well, thanks for asking." That's great, but we want to be able to continually talk with the model and store a conversation history. So, let's write the code to handle that.
I'm going to create a function called `handle_conversation`. This is where we'll put all of our main code. I'll start by storing some context, which will just be an empty string, and I'll print a welcome message. I'll say, "Welcome to the AI chatbot," and then we'll just tell them that they can type something like "exit" to quit.
On the next line, we're going to make a while loop and say `while True:`. Inside of here, we're going to collect some input from the user. We'll say `user_input = input("You: ")`.
Next, we'll check if the user input, converted to lowercase, is equal to the exit keyword. If it is, we will break out of the loop to avoid an infinite loop.
Now, we're just going to generate a response. We can take the previous code and bring it up, so let's paste that inside here and indent it properly.
Now we're going to say `result = chain.invoke(context=context, question=user_input)`. We can print the response by saying, "The bot said: {result}."
Next, I'm going to store all of this in the context so that the bot has the conversation history and can respond to previous things that have been said. I'll say `context += f"\nUser: {user_input}\nAI: {result}"`.
Now, every time I use this prompt, I'm passing all of this context back into the bot so it knows what was said previously and can use that to respond.
That is our function. Now all we need to do is call that function. I'll say `if __name__ == "__main__":` to check if we're directly executing this Python file, and then I'll say `handle_conversation()`.
Now we are going to run this function, which will simply ask the user to keep typing in things until they hit enter or type "exit." It will store the conversation history and pass that back to our model so that it can respond based on what we said previously.
Let's test this out and make sure it works: `python3 main.py`.
"Welcome to the AI chatbot." We're going to say, "Hey, how are you doing?" It's going to give us some response. I'm going to say, "What is your name?" Great, no personal name, that's fine. "Can you help me understand history?" Let's see what it says. It gives us this long response, which I imagine is saying yes, it can help us if we give it a good question.
Let's try to exit with "exit," and now we are gone.
There you go! We have now created a program that we can actually interact with a local LLM. We don't need an OpenAI key, we don't need to pay any third-party service, and we can run this open-source model on our own computer and utilize it from our Python script.
Obviously, this is a very simple example. You can do some really cool things that are a lot more complicated, but that's what I wanted to show you in this video. Hopefully, you found this helpful. If you did, make sure you leave a like, subscribe to the channel, and I will see you in the next one. [Music]