Transcription
Hello everyone, friends, this is my first video after a long stagnation. I haven't been shooting videos for reasons that are not interesting to you. So, let's get straight to the topic of the video, about how to create your own AI agent in Python in 2025. It would be better to first explain to you what AI agents are in general and how they differ from regular chatbots with neural networks. In general, this picture outlines the differences between a regular neural network like GPT and an AI agent. Well, the main thing is that AI agents make decisions themselves without user help, meaning they are more autonomous and often have more functionality than regular chat neural networks like GPT. Also, they can correct themselves and check themselves for errors. But in general, I think you will understand further during the video tutorial, so we won't go too deep into this. But, but, but, but, but, but before we start, I expect that you, my friend, know the basics of Python and programming, like you understand how APIs work, how to install Python libraries and use them. Cool. Let's start now.
So, let me briefly tell you what we will be developing. I have this diagram for you so you can understand how the structure works and how cool a project you are going to write. Our AI will be your super Python assistant. It will have the ability to write code for you, by the way, not just code, but a whole project. It will write the entire Python project structure for you, requirements.txt, Dockerfile, docker-compose.yml, possibly Bash files, Bat files, and so on. It will also be able to efficiently write parsers in Python, as it will have the ability to visit websites and analyze their source code. It will also be able to efficiently write parsers in Python, as it will have the ability to visit websites and analyze their source code. Another additional feature, so that you are aware of all events, it knows how to use the browser, well, so that it can Google various information like date and time, what is the latest version of Python, and other important news. The main thing is that it will be able to run projects, and if they don't work, it will understand it itself and fix its code. Here we will have a beautiful web interface, that is, just an HTML page. You won't need to write this frontend, it has already been written for you so that you don't get tired of my video about AI agents. Yes, through this web interface, you will send your messages to the backend. Yes, yes, we will develop our own mini-backend. Don't be afraid of this, my friend, because there will only be two functions to purely connect the frontend with the request script to the neural network. The neural network itself will be on OpenAI. Yes, I understand that it costs money, but there is also a free option. You will also need a VPN, but specifically for you, my friend, I have a solution to all your problems. Below the video, I will leave a link to a good VPN. It is paid, of course, but you will get 3 days of free access. I think that should be enough for developing this project. As for what can replace OpenAI, it is, of course, Hugging Face. It is not a neural network from Elon Musk, it is a platform that gives you limited but free access to open-source models like Llama and other models. You don't even need to worry about the code, because it fully supports the OpenAI library, and you just need to replace the Base URL. I will show you how to do this later. Well, and in the video, I will use the GPT-4 Mini model from OpenAI because, in my opinion, it is more stable and faster than the same GPT-3.5, and I will write functions for it. Yes, by the way, for those who didn't know, language models can call functions and pass the correct arguments to them. In fact, all agents use exactly this ability to call functions, which makes them multifunctional.
So, let's briefly go through all the functions that our neural network will call. This is the terminal function for running Python files, creating folders, directories, files, and so on. Well, generally everything that can be done in the terminal. Google search, we will use SerpApi. It is a service that allows us to make requests to Google and get information from there. The function to visit websites to get the structure of web pages. We will use the Playwright library. It's like Selenium. And the function to save code so that it saves code in our system. In general, that's all with the functions. Let me tell you about the Helicon service. This service helps me view the statistics of my requests. It's more convenient than the OpenAI platform. You can not use it or use it. In any case, I will leave all the links in the description below the video. It's very convenient to see how much your request cost and what the dialogue with the neural network was. You need to register here and get your key. This can be done in the settings, right here, and in the API Keys section. We also need to register on SerpApi. They will give you 2500 requests for free to make requests to Google, and you also need to create an API key there and save it in the code. This is done in the settings, right here, and you just copy it. Also, the key from OpenAI. I will also leave the link in the description. Right here, we click create key and definitely set the key permissions to be the same. For those who want to use the neural network for free, I will also leave a link to the documentation for Hugging Face. Register on the Hugging Face platform and create a key. It is completely free, but there are limits. You can see for yourself which neural networks are here and what the limits are. You create a key here. I will show you how to integrate it a little later.
Let's start by creating a folder for the project. I will name it `python_ai`. Let's open this folder in our text editor. Mine is VS Code. Yours might be Visual Studio, Notepad, or Sublime, or even Notepad or Nano. Your Python version should be higher than tenth or tenth. Let's create an environment and activate it. [Music]
Let's now create the main file. I'll call it `main.py`. And let's start with developing the backend on FastAPI. But first, we need to install this library. Let's immediately add it to requirements.txt and install it. Let's also install the `uvicorn` library to run our FastAPI backend. FastAPI is installed. Let's import it and create an `App` object. Let's create a main route that will initially just output "Hello world" and run our backend through `uvicorn` on port 8000. If it's not indexed in the library like mine, you need to specify the correct path from the virtual environment like this, and everything will work. Let's run `uvicorn`. Cool, we get a color. Let's move on. Below the video description, I will leave a link to our chat file. Let's create `index.html` and just paste it there. You don't need to do anything on the frontend. Everything is already configured, and I won't explain it in detail because the lesson is about AI agents. So, we just need to display it upon request to the backend to show us the HTML page instead of JSON. We need to import the `HTMLResponse` class from FastAPI, read the HTML file, and pass it as an argument to the `HTMLResponse` function. Let's see what came out of this. A cool interface is displayed, messages are sent, but for now, we are not processing them. I suggest we fix this. Our messages are transmitted between the frontend and backend via websockets, so let's import the `WebSocket` class from the `fastapi` library. Let's create a route `/ws` and call the `websocket.accept()` function to accept [music] the connection. In a loop, we will receive messages from the frontend. We will add the message from the frontend to a variable, remove extra spaces from it. Also, let's wrap our code in a `try-except` block to catch disconnection from the client correctly if something happens. For this, let's import the `WebSocketDisconnect` class from [music] FastAPI. Let's check. We got an error. By default, it turns out that FastAPI does not support sockets, so we need to install an additional standard extension. Let's do that and [music] do it. The library is installed. Let's check. As we can see, we received a message. Let's replace the prints with logging through the `loguru` library. They will look much better this way, and let's import `logger` from the `loguru` library. [music] [music] [music]
Next, let's create a fake response from the neural network and send it via websockets. For this, we will use the `send_text` function, and we will need to send it in JSON format, so let's import the `json` library. We convert our dictionary and send it. Let's now [music] check. Everything works perfectly. Now we need the real neural network to respond. For this, we install the `openai` library to make requests to ChatGPT. Let's import the asynchronous `AsyncOpenAI` class, create an instance of the class, and change the `Base URL` to the Helicon link if we want all our requests to go through them and they make detailed statistics for us. Here is their link. We also need to specify the Helicon key in the request headers. But storing the key in `.env` files is not safe, or as in our case, in a file. So, let's paste the key here. Also, we will need to create a `.env` file to store the script settings here. And let's also install `python-dotenv` to import variables from `.env` into Python. From the `dotenv` library, let's import the `load_dotenv` function and call it like this. It will copy all files from `.env` into environment variables, and we get them from environment variables through the library by the variable key. I also imported the Helicon API key from the `.env` file. If you want to use the free neural network, I will show you how to do it through Hugging Face. Let's go to the Hugging Face platform. On this page, we get the key. Let's add a new Hugging Face variable in the code. We change the `Base URL` to this one. Headers are not needed, and we paste the Hugging Face key into the `api_key` parameter. But in this video, I will use OpenAI, so I will leave Helicon so that my requests go through them and I get detailed statistics. In general, the code will be the same because Hugging Face is fully compatible with the OpenAI library, and you can use other models besides OpenAI. Okay, let's go back to the code. Let's remove our fake response and create a chat history list and immediately add a system prompt to it. Temporarily, I'll write that it just knows Python, then we'll change it.
Now we need to add the user's message to the list and make a request to the neural network with the chat history. [music] We will use the `gpt-3.5-turbo` model and pass the chat history in the `messages` parameter. Let's save the message that the neural network returns and add it to the chat history as well. [music] Let's return the message to the frontend and test it. Let's add the OpenAI API key to the environment variables. [music] It's important that the variable is named exactly `OPENAI_API_KEY`. In this case, it won't need to be imported into the code. The library will find it itself and use it. Let's try again. We get an error. So, I figured out what the error was. From `AIMessage`, we need to take the content and output it as text. Let's check. I suspect the error is here. Let's quickly comment it out. Let's put the system prompt in a variable. We'll get back to solving this bug. Let's put it in the `SYSTEM_PROMPT` variable in the config. I won't write it manually now, it would take too much time. So, let's go to the link in the description and even copy and paste it. The main thing is to specify your computer's data in the system prompt, meaning mine is `id: dora`, operating system, yours might be Ubuntu, Windows, Terminal. Specify what you have. Mine is `bash`, yours might be `cmd` or `PowerShell`. Bash is needed so that the neural network makes the correct commands in the terminal and also the correct version of Python that is installed by default in your [music] system. Let's check our system prompt. Excellent, everything works. It understood what it should do.
Now let's write a function that will call the neural network. For this, let's create a new file `functions.py` and write our functions there. Let's start with the save code function. It will take two arguments: `code` and `filename` for saving. It will return a status in the form of a string. Let's also import `logger` to understand that the function was called. Let's wrap it in a `try-except` block because the neural network's memory is not very large, and an error can be huge. So, let's shorten it to 5000 characters using the `os` library. Let's create a folder `ai` if it doesn't exist and write the file with the code there. Let's [music] check. The function was called successfully and created the file. Cool. Let's write the next function, and it will be the terminal function. Let's call it `run_command`. It will take three parameters: `command` for execution, `input_str` for entering any data if the script requires input. For this function, we will use the `subprocess` library and call the `Popen` function. We will specify that all commands will initially be in the current folder and write a condition that if there is any input data, we will also enter it. We will get error and success messages from the terminal during execution. Here too, we will return a limited number of characters because the response can also be unpredictable and exceed the neural network's context. Next, let's create a function to search for information on Google. Let's call it `search`. It will take the `query` parameter and immediately wrap it in `try-except`. We will need the `requests` library to send requests. According to the documentation, we need to make a request to this URL and specify the SerpApi key in the request headers. We get the SerpApi key in the SerpApi personal account and paste it into the `.env` file. We import it into the config and from the config, we import the `search` function. And in the request body, we specify our `query`. Let's save the result in a variable, and if the request status is 200 or 201, we return the result. Let's check the function. Let's ask, for example, who was the first president of the United States of America, and output the result through the logger. Let's run it. The result is obtained, everything is fine, only it's not entirely clear. Let's look at it in a formatted [music] format. In general, I think this whole text will be too much. We can output only the first three elements in the list of organic results to the neural network and extract only the titles and snippets from them. Let's do that. To convert the list of strings, we will use the `join` function. [music] And in this format, we will return it to the neural network. Let's check. Everything is good. I think it will be clearer for the neural network this way.
Now let's create a function that will open a website by URL. Let's call it `fetch_page`. It will have the `url` parameter, and it will also return a string. Let's wrap it in `try-except` as well. For opening web pages, we will use the Playwright library. It needs to be installed via `pip`. We will also need the `playwright-stealth` library because the regular Playwright library is flagged by websites as a bot, and to avoid this, we will use the `playwright-stealth` extension so that websites don't think we are some kind of bots. From `playwright`, let's import `async_playwright`. From the `playwright-stealth` library, let's import `stealth`. Let's create a browser based on Chromium with `headless=False` parameters. This allows us to see the browser when it is created, and in `args`, let's pass parameters to disable the property that the browser is controlled by a script. Next, let's create a new browser context with a user agent. Essentially, this is a separate isolated environment within a single browser instance. It's like incognito mode where cookies and local storage are isolated, allowing you to work with multiple sessions in parallel without affecting each other. Let's also set the browser width and height. Let's open a new tab in the browser. Let's use the `stealth` function to mask our browser. Let's go to the URL using the `goto` function. Let's wait for the page to load. Let's take a screenshot for debugging. Let's specify the screenshot path. And we will also need `BeautifulSoup`. This is a library for parsing web pages. Let's import it. Let's save the source code of the page we visited. Let's create an instance of the `BeautifulSoup` class, pass the HTML page to it, and specify the parser as `html.parser`. Let's go through all the tags in the HTML structure and remove `script`, `style`, and `iframe` tags from them because, in essence, they are not needed for the neural network, and why fill its context and memory with them. Let's also remove the `style` attribute from all tags so that they also don't fill the [music] page. Therefore, we will return a maximum of 20,000 characters.
Let's check our function. It's asynchronous, so we will need `asyncio` to run it. Let's create a `main` function and call it. There's likely an error. It seems I need to install `setuptools`. Apparently, this is a dependency for `playwright-stealth`, so let's install it and check again. If you installed Playwright for the first time, you need to run `playwright install` so that it installs its browsers that are needed for operation. Let's check now. You got an error. This is because I didn't specify the `async` keyword for the `fetch_page` function. And I also forgot to make the `fetch_page` function asynchronous. Let's fix this and try to open my website. The structure is not quite correct. Now I'll try to figure it out. Friends, I have also replaced the link for testing with `example.com` because, apparently, I had a problem with my website, and it was not opening correctly. Let's check `example.com` and see that we are getting a filtered, beautiful HTML page. Excellent, the function works. Let's move on.
And now we need to write descriptions for all these functions so that the neural network understands how to call them, what arguments to pass, and how they generally work. Here is the OpenAI documentation. I will also leave the link in the description. You can look at it. In principle, here is the function name, parameters, their type, description, everything you describe here goes into the system prompt. In fact, the description also cannot be very large because it can exceed the limits. Let me tell you the algorithm of function calling. First, function definition occurs. We pre-define what functions the neural network has and what their parameters are. Second step: request analysis. The neural network receives the user's request and analyzes it for tasks that can be solved by calling one of these functions. For example, when the user, in our case, writes "create file main.py," it analyzes the user's request, looks at its functions, and sees that there is a file creation function, and returns a JSON response to the server with the function name and the necessary arguments to run that function. Then, we, on the server side, call this function and pass the arguments to it. The function executes, returns the result, and we can either show this result directly to the user, or we can send it back to the neural network and generate the response again so that the neural network itself makes some kind of response. In our case, we will send the function result to the neural network.
Let's specify our functions in the variables. Let's just copy the function structure from the OpenAI documentation. Let's change the description, name, and parameters. The first function will be `run_command`. I'll write a short description that this function executes commands in the terminal. Let's specify our parameters and their descriptions. The parameter type will be `string`. The `input_str` parameter type will also be `string`. This `required` array means that these arguments are mandatory and cannot be run without them. Also, somewhere I saw that you can specify the function's response type. Let's try. I'll write a short description. The second function will be `save_code`. Let's write a short description for it too: "Creates files with content and saves code in the directory." Let's change the parameters. It will be `code` and `filename`. Let's also change the description: "content of the file, filename," and `required` will also be changed to `code` and `filename`. I'll probably remove the return here, it's not necessary. Similarly, execution for the `search` function. Short description: "Executes a Google search query." Here we have the `query` parameter. [music] And the last function is `fetch_page`. Let's write a short [music] description for it. The parameter is `url`. [music] We are done with the functions. Let's return to the `main.py` file. Let's write a loop so that the neural network constantly checks itself until it understands that the task is completed. For example, the user gives the neural network a task to write code to work with some API. Step one for the neural network will be to call the file creation function. If the function execution status was successful and the neural network believes that no further functions need to be executed and this is enough, then it returns a regular message without calling a function, and at this moment, we exit the loop. In other cases, we will send the statuses of the function calls to the neural network until it considers that no more functions need to be called and everything is ready.
Let's import `Tools` from `openai` and pass it to the `tools` argument. The `tool_choice` parameter means whether to call the function constantly or according to the AI's opinion. If you set it to `auto`, the function will be called only if it deems it necessary. The `parallel_tools` parameter is set to `True`, which means that the neural network can call multiple functions simultaneously. [music] Let's uncomment. Check if there is a function call in the neural network's message. If so, return its regular message and exit the loop. Let's iterate through the functions that the neural network called, get the function name, and parse its arguments. Let's create a `result` variable beforehand. Let's wrap it in a `try-except` block. Then we will compare the function name with our functions and simply call them with the arguments. Don't forget to import our functions from the `functions.py` file. [music] Let's make a small condition here because `input_str` is an optional parameter. If `input_str` is present in the arguments, then we call the function with `input_str`, otherwise without it. We will record the function result. [music] We'll simplify it here because the neural network can call non-existent functions, so we will also handle this. We'll just write that an unknown function was called. At the end of the loop, we will add the function result to the dialogue history. The role will be `tool`, the content will be the function result, `call_id` will be the function call ID. This must always be set. In general, that's probably all. Let's check. "Hello, write me a project for shortening links." We have a `str` error. Most likely, `tool_choice` requires all arguments to be filled, so let's remove it and test without it. As we can see, the neural network calls functions, creates files. It created `requirements.txt`, `get.me`, let's look at them. It also creates a Dockerfile and, in general, wrote the entire project with several files. But the only thing I see is that it didn't create a separate folder for the project. We see the message, we see the code. Let's fix this. I don't like that it doesn't create a folder for the project because there can be many projects, and they cannot be located in the root folder. Let's fix this in the system prompt. Friends, I have fixed the system prompt. Yours will already be fixed because you will copy it from my repository at the beginning of the video. The link will also be in the description. And let's check again. Let's wait for our request to be processed. Neural network [music] files in the `URL` folder, but it sees that such a folder does not exist, understands this, and creates this folder, and then creates files inside this folder. It also starts installing dependencies. It returned the result to us, says that the project was successfully created. Let's ask it to do it without the backend and check for operability. New files have been created. The `scripts` folder was successfully created. Here's what it does. Uh-huh, cool, everything works.
Let's now ask it to create a parser. "Write a new project for parsing the example.com page." [music] It creates a new folder `web_parser`. The project was successfully generated. It installed dependencies, but I think it didn't check. Let's ask it to check now. Yes, it runs the file, and what result will we get? The script was executed successfully. Here is the result: links to pages. Let's give it a new task to parse all the text on the website and [music] let's see. The `fetch_page` function is happening, and the neural network gives a result, but for some reason, it gave it in the chat. Let's ask it to generate a script for this, create a new file, check it, add a comment to the file, and in general, it coped with its task, and that's all. Brothers, if you liked it, give it a like. If you didn't like it, give it a dislike. Tell me in the comments what you didn't like so I can fix it. Everyone, bye. Thank you all very much for watching.