Революция AI-команд: от одного помощника к армии специалистов за 5 минут

[Music]

Hello everyone. Thank you to everyone who was able to connect to our lecture online. And today we have a new guest in the studio. I present Alexander Ogafontsev, an expert in process automation using artificial intelligence. And today Alexander will tell us about multi-agents and how to create your own team of agents in just 5 minutes. Please ask all your questions in the chat, okay? Hello everyone. Glad to see you all. Ask questions in the chat, or you can ask them under the announcement of this lecture in our Telegram channel. The link to it is in the description below the video. And now Alexander, the floor is yours. Thank you.

Hello everyone. As I was introduced, my name is Alexander. Today I will talk about multi-agent systems and how they help solve complex problems. So. Let's start, probably, right away with showing the presentation. So. And so, as they asked in the comments to show what could be repeated, if we have time today, I will even show a more practical part about how to specifically create N agents. Here, based on an already ready-made example that I made a little in advance, today, so that it is more illustrative. Here are the topics we will cover today. The first is the problem of mega-agents. This is a single agent solving all tasks. That is, how, for example, most of you probably use the conditional ChatGPT, when you load absolutely everything into it. You ask it to do design, analyze text, and, I don't know, generate some marketing creatives. So. And all this using one agent without any clear instructions. And, accordingly, the solution to this problem is the principle of division of labor. This is the creation of a multi-agent system, where each agent is responsible for some part of it. One agent is responsible, for example, for analytics, one agent is responsible for writing texts, one agent is responsible for collecting information on the internet, and so on. Well, and we will analyze this with practical examples. Different complexities. The first, probably, is even a bit clickbaity, is how NASA controls the Mars rover. Or rather, even to say Mars rovers, because it is also a multi-agent system within a multi-agent system. So how de research is structured. This is depresearch, which you could use in ChatGPT, in YandexGPT, it is also available in Claude. And as a separate product, you have 100% heard of Perplexity. So we will analyze it using the example of Claude, probably, because it is my favorite neural network. So I'm still waiting for them to start paying me for their native advertising, but since they haven't started yet, I'll talk more with examples. So. And also about the AI Loop problem, when neural networks cannot solve a problem independently and just scan, for example, your entire repository to solve a problem that is actually solved with one line of code. So we will analyze this problem in more detail and analyze the architectural patterns of multi-agent systems. What are they? We will analyze only two remaining ones, analyze their advantages and disadvantages, when, in what situations to use them, and when not to. And if there is time, we will analyze the creation of N agents not only on slides, as is done in the presentation now, but also in practice. So I will show how it works, how the coordinator agent uses different agents to solve specific situations. So. And at the end, we will look at Cloud Subscriptions. This is a recent feature from Cloud Code, when you can create separate agents to solve specific tasks. In my opinion, this is probably the most necessary function for Claude now, because it allows you to create agents for specific tasks, that is, essentially emulate an entire development team. Well, and at the end, I will leave a slide with my contact information so that you can add me on LinkedIn, write to me by email or Telegram. So. And there will also be a QR code to my Telegram channel. Well, let's start. The problem of mega-agents. This is Jack of All Trades Master of None. That is, an agent can do everything, but knows nothing. It does not have any specific specialization, all its knowledge is superficial, and there are no clear specific instructions. This problem is probably related to the fact that agents are loaded with almost all the information available on the internet. So, literally the entire internet. And there is no longer a clear specialization, who is he? He is, in essence, a huge knowledge base, where all the information is definitely there. It just needs to be limited and used in a certain way. Also, context switching reduces quality. What does this mean? If you try to put different tasks into one chat with ChatGPT, radically different and quite large ones, the further you go, the more it will lag. So. Because this is filling the context window, sometimes often overflowing, when you load so much information that it simply forgets some of it, or mixes some of it up. For example, you can give it completely contradictory instructions. So, for example, to think about design and write text, these can be quite contradictory instructions, because they are completely different. So. And it is impossible to give clear instructions to the agent, because either you will have, respectively, the instruction window will be too large, and it is simply impossible to fit everything there, and most likely, the agent will perceive it incorrectly. So imagine you ask it to analyze, analyze, and draw a picture. How will your instructions look? You need to give a separate one for each part. If it gets confused somewhere, and it most likely will get confused somewhere, it will be a complete mess. It will remember nothing, and will give you some average of all this. Therefore, well, as it were, nothing good will come of it. And, accordingly, there is a solution. The solution is the principle of division of labor. This is the creation of agents where each agent is an expert in its field. That is, each agent has a clear set of instructions, has some role model, often has a clear set of tools that it can use. And with all this, there is either a coordinator who coordinates all the agents around, that is, the agents do not communicate with each other. So I will analyze this on slides later. We will analyze everything now. Or peer-to-peer when agents communicate with each other. So agent teams work more effectively. This was proven by Claude, for example, with its Perplexity. They have an efficiency increase of more than 90% between the Claude Opus model, when it is alone, which is their strongest model. And instead, they used a multi-agent system of Claude. Claude is like an everyday model, that is, for all everyday tasks. Accordingly, multi-agent systems show themselves much better. And five specialists are faster than one generalist. Here, I think it's clear that five agents can work in parallel, and they simply work faster because they don't collect everything, they each collect their specific piece, or do their specific task, and they do it faster than one generalist. Simply, you launch five in parallel, or launch one five times. So I think it's clear here. So. Well, and how NASA controls the Mars rover. By the way, I highly recommend everyone who has a strong level of English or is interested in this topic to read the original article. I will probably share it later in the chat and maybe with a link in the comments on YouTube and on my channel. So this is an example of how NASA controls Mars rovers and lunar rovers. They are called rovers. I think this will be a correct translation from English to Russian, when they use a multi-agent system, that is, a full-fledged swarm. You might have heard of it if you were into robotics or watched a Marvel movie, where each rover performs some specific function. For example, it drives on the Moon or Mars and collects some research results. So, what is here, some temperature, some stereo cameras, they do something. And so it collects its piece. But there are many such rovers. It is definitely not one. Each of them collects some of its information and sends it, roughly speaking, to a single, to a single distributed database. So, it's not visible in the picture. Distribute DB. Each of them sends, and they form a single map. That is, one, for example, a large robot could not do this, well, it could, but it would take much longer. And so a billion small robots drive each in their own part, forming a single map. So. And at the same time, everything is perfectly fine. So. And they also have a multi-agent system within a multi-agent system. That is, the rovers themselves work on the principle of this swarm, where there are many of them, they are all independent, but at the same time they have a leader. So. And the leader is chosen, that is, it is not assigned to any specific robot, that is, it is not distinguished by color, for example, but it happens through this block. This block is responsible for choosing a leader among all these robots. Why is it done this way? Because the leader commands global planning. That is, it is, roughly speaking, responsible for the mission and for what is happening now, what measurements they are taking, and so on. But this robot can very likely fail. It can get stuck, it can fall into a conditional crater. And then a new leader is chosen again through this block. So. There are many different blocks here. There is an agent controller - this is the controller, actually, of the agents. There is a planner agent. It is responsible for planning the expedition and distributed measurements. So there is a leader, this is strategic planning, that is, planning the expedition itself, collecting some research, and a team planner, that is, a planner of what other agents are doing, and other these rovers. So. Well, and also motion planning is a very interesting topic, because there is global planning, this is conditional path planning for the next kilometer, that is, a conditional direction. So. And local planner - this is route selection, so that it doesn't bump into pebbles, doesn't run into some craters, so. And so on. I think we can move on. This system is called peer-to-peer. We will analyze it a bit more or swarm. They can be called differently. Next, we will analyze architectural patterns. So. And how clot depresearch works. This is another model. Here there is a lead agent orchestrator, that is, a coordinator, the main agent that commands other agents. The agents themselves do not communicate with each other, they do not choose a leader, the leader is fixed. So. And, accordingly, search sub-agents. These are these guys who collect information. What is the advantage of this approach? First, it is predictability. You can configure this orchestrator as you wish. It is easier to debug, it is easier to make, in general, it is easier to develop. So. And search sub-agents work in parallel, well, if you want it that way. So. And they can work with their own inputs. So imagine, you need to launch depressor on the topic of the real estate market in 2025. So the agent receives your request. So it's an arrow, it receives your request and says, the first search agent, find me information on the real estate market in all news sources. Let it be, say, RBC, Channel One, Forbes, I don't know what else there is. The second says: "Find me information from all verified sources." So let it be, I don't know, some of the most trusted, conditionally, Wikipedia. The most trusted source for us now, I think. So. And to the third sub-agent, it says: "Find information on all sorts of human forums: Pikabu, Mail.ru, not Mail.ru, well, in general, any that you can find. Pikabu, for example." So they all somehow collect information. So you see the arrows? So they collect information, give it to the orchestrator, and in the end, this orchestrator collects all this information together and gives back the final report. This is generally how almost all depressors work now. So. Because it's just faster. You would definitely get tired of waiting for one agent. Depressor. In general, it's not a fast thing, especially since it works in parallel, not with one or three sub-agents. So it still takes a long time. And if it were a single system, that is, a mono-system, it would have a lot of disadvantages. Well, the first, most obvious, is speed, as I already said. Second, no single agent could fit all this into context. Imagine that it needs not only to collect information, but also to make some kind of report, that is, a report. And this report must be one and quite voluminous. So, I sometimes made reports of 30-50 pages in YandexGPT, if the topic is quite extensive. So it just wouldn't fit anywhere. So that's why sub-agents collect information themselves. They all have independent context, that is, independent conditional recent memory. So. And at the end, they give it. You can come up with a lot more options here, but the base works like this.

[Music]

What is the AI Loop problem? So it is generally related to context, that is, to the memory of neural networks themselves, to how they think and what they actually remember. I think those who have tried some code generation with neural networks have encountered the problem that when you have part of the repository written, you need to fix something, and constant code changes begin without solving the problem. I have especially often noticed this in infrastructure files, that is, anything related to Docker or Kubernetes, because there is no specific feedback. Well, a feedback loop, let's say, is slightly broken. This problem occurs quite often there. What is it actually related to? First, it may lack understanding of what is happening. That is, it simply cannot collect enough information, or it does not have enough role model for what to do in a specific situation. Second, it is context overflow. It needs not only to collect how your code looks in the repository, but also what needs to be done, and how to do it. And often this is non-trivial, because they have limited memory. And while they were collecting information about your code, while collecting information about what needs to be done, they forgot how to do it. While remembering how to do it, they forgot what is in your repository. When they remembered what is in your repository, they forgot who you are at all, what you need. So. And, accordingly, they rewrote half of your repository. This is a very common problem. Multi-agent systems partially solve this precisely because of context separation. But, unfortunately, the problem has not completely disappeared yet, although it has significantly decreased. Neural networks do not stand still. This problem occurs less and less often. So. Well, and, accordingly, by points, I have generally outlined everything. Context overlap is when this memory overflows. And this problem often requires human intervention. How do multi-agent systems partially solve this problem? Multi-agent systems separate each agent and make it independent. That is, it has its own context, it has its own set of tools, as I already said. And this helps them solve problems more clearly. they don't manage to completely fill this context window, all their short-term memory. You can also call it short-term memory, right? So. And if anything, they can contact other agents or the coordinator. Plus, you can configure specific agents so that they fix this. For example, when some infinite changes occur, you can configure an agent that will constantly review changes and tell the agent, for example, the developer, that friend, you are doing nonsense. You can't do that. Well, as it were, sorry, but you have to redo everything correctly and think about how to do it. So. Plus, you can configure agents, for example, an architect, who will strictly plan how your specific block, website, infrastructure in general looks, how it all should interact, how it communicates with each other, and so on. All this, in one way or another, helps to solve the so-called loop problem. So. Because it's all separated.

What are the architectural patterns of multi-agent systems? We have already analyzed them with some live examples, now let's add a little theory. First, coordinator-worker or hierarchical. That is, there is a certain hierarchy. In this case, the monarch himself is the coordinator. He is the main agent. He coordinates all the work of other agents. As seen in the picture, there is, for example, a coordinator. This is this big blue one. There is a bot developer. Oops, excuse me. There is a bot developer. He writes code. So he only writes code, does nothing else. He is a feature developer, he is great. In this case, let's say we have a full-stack developer. So there is a designer. This is an agent that only develops the design of the applications themselves. He does nothing else. He only draws components, thinks about how it will all look. And this was a bot that searches the internet. Unfortunately, there wasn't enough inscription for it. This is Panifen, if anything. So it only collects information on the internet about bugs, errors, or how to fix or do a particular feature. So this is a researcher. He, for example, strictly looks at documentation. He searches how certain methods work, how certain libraries work, how anything works, and gives it to other agents as needed. And a tester bot is a bot that tests functionality and writes tests. This option helps to divide responsibilities. Nothing is mixed up. Everyone is engaged in their specific story. And at the same time, each bot will have its own set of information in its context. The developer bot has what features it has implemented, how, and, accordingly, what code is in the repository. The designer has what customer requirements are for how it should look, accordingly, how the final product should look. This bot should find what bugs the developer encountered and how to fix them. And the researcher, how not to allow such bugs, for example, through documentation. So the tester will test all this, what the developer wrote, and say: "Friend, your feature doesn't work." All this is returned to the coordinator. The coordinator distributes it further. In fact, this is probably the most understandable story for people with designing multi-agent systems, because it is what we are used to. This is the so-called department. There is a boss, there are employees. Employees are sometimes different, sometimes quite similar, but in general, everyone performs some role. And someone coordinates the task. So. There is also a peer-to-peer scheme. This is what we considered using the NASA example. This is where each agent is equivalent, that is, it does not have any, there is no main element. All agents are equal, all make decisions by consensus, that is, they are forced to negotiate. And plus, each agent can communicate with each one, transfer information, ask. So, for example, with robots, for example, one sends maps, and they think, where to go next. So, let's say they don't have a leader choice. Such systems also exist. You might have seen them in examples of drones from science fiction movies or when someone in Dubai launches huge drone shows. So. And, accordingly, everything is equal. There are certain coordination difficulties. So. But such systems exist. They have a certain advantage, which we will discuss a little later, and they have their disadvantages. The differences in approaches here are actually determined by what you want to achieve. So. Two different systems, although they perform, essentially, one task, whatever you set, but the approaches can be different. Coordinator-worker is generally easier to implement, that is, it is much easier to develop, to debug, because you encounter fewer problems. The coordinator is easier to debug, that is, it is easier to see where it makes mistakes and why, and it is easier to change it. Peer-to-peer. It's not so simple here. You need to change the behavior of a whole crowd of these agents. There can be its own difficulties, but it has its own advantages. So NASA chose this particular option. So. First, here it is clear, it is the management structure. I have already talked about it. That is, the coordinator-worker and, accordingly, the peer-to-peer coordinator-worker has no one. That is, they are equal. So. And, accordingly, this also applies to decision-making. The coordinator plans and distributes tasks. Peer-to-peer is consensus. So. As I already said. That is, agents must negotiate among themselves, negotiate where to go, what to do, and so on. So. And, accordingly, by advantages. Coordinator-worker is vulnerable to coordinator failure. If something happens to the coordinator itself, nothing works for you. Absolutely. It doesn't work at all. Why? Because the coordinator is, as it were, a single point of failure. So it receives information, it distributes it. Even if all workers are working, your system will crash. So that's why the huge advantage of peer-to-peer. If something breaks, it's okay. You still have 99 left and they all work. Yes, it's unpleasant, but nothing happened. So. This is evident in Mars rovers and lunar rovers, because you can't send a second coordinator there. So. Even if you send one, it will clearly be long. Your mission will stop, the robots will sit down, so, well, it's sad, not tasty. So that's why they do peer-to-peer there. So. And by area of application, coordinator-worker performs well on structured tasks with a clear hierarchy, because it will be easier for the coordinator to plan and receive some information from workers. And peer-to-peer is a dynamic environment with uncertainty, because even if something happens, the workers will somehow agree among themselves, so these robots, for example, they will come up with something, get information, and somehow get out of it. The coordinator will have a harder time in this regard, when to apply. Multi-agent systems are excellent, this is a complete solution to complex tasks already now, but there are pros and cons. Let's start, probably, with the pros. So this is solving complex tasks. This is a complete use of multi-agent systems. Because if your task consists of several, so to speak, layers, stages, it will be much easier for you to solve it using multi-agent systems. You will have several stages, several agents for different tasks, within one. That is, for example, you receive a task, and you need to do 1, 2, 3 with it. If one agent with one context does this 1, 2, 3, it will be a complete mess. It won't work, it will start to lag. Especially if there are more than three stages. The more of them there are. So, I don't know if you've noticed, but even in ChatGPT, when you generate images, so, the chats are constantly changing, because when your chat is cluttered with previous information, you get, as it were, a disgusting mess. So, even when I was generating these images, by the third iteration, it was already impossible to change the image normally as you wanted. It's just impossible. So. And what is also, probably, important, the need to use several tools. This applies to automation in general. Or if you, for example, use the conditional Claude, you might notice MCP. In Claude, you probably won't notice it, because mostly you solve tasks related to text, they are not particularly complex. But for example, with the development of full-fledged applications, using code generation, there you might notice that your neural networks start to lag. So. This especially applies to Zapier and other No-Code automation options, such as Make. There are a billion more options. I think you can find them yourself. So. The use of these tools needs to be separated. So imagine a trivial situation that we will analyze a little later. You need to find information, find information, so, to get information from your vector store. So, for example, Supabase, and give some option. One agent, well, there is a high probability that it will get very confused, because it needs to remember how to get information, and in what format. So these tools also work in a certain way. How to get it from Supabase, how to give it all, how to get it all. Especially if you have some structured output, that is, you give it in a specific form. The simplest example, understandable to developers, is JSON. Development is a conditional dictionary. So in English it's called a dict in Python. The language I speak is essentially JSON, where you have a key-value pair. That is, you, for example, ask the neural network to give a key, so, for example, let it be source, that is, a link to the source, a colon or an equals sign, and the link. So if you want the information to be given in this form, the neural network needs to be told about it. If you give many such instructions, they will, first, start to contradict each other somewhere, and the neural networks will get confused. Therefore, here it is 100% divided. This also indirectly applies to collecting and analyzing large amounts of data, because collecting and analyzing are two completely different tasks that need to be separated, because not everything will fit into this so-called context, not everything will fit into this memory. Even what you see now with, for example, dropping files to ChatGPT, Claude, YandexGPT, and so on, in reality, all this does not fit into the context. There is a more complex scheme with vector stores. So. Not everyone reveals it. So. I think you can read about this on the internet. Unfortunately, I'm not very good at this, so I won't take it. So. And probably the most logical is a task that requires different areas of expertise. Just like with the previous example. Collection and analysis are two different areas, they are even two different professions. So they must be separated 100%. You need to give different role models, a different set of tools, and different instructions on how to get data, how to give it. So. And all this together is related to processing unstructured data from different sources. I have already touched on this a little. This is precisely the vector store. So.

There is this storage where you put human-like information. It turns into embeddings. This is a vector, meaning a set of conditional, like, a set of weights. I don't know if there are any data scientists here, they will correct me, of course. I don't even know how to explain it more simply. Well, in general, it's conditional computer information intended for neural networks. Imagine in your head that it's a set of zeros and ones that you can't decipher. So. And when you make a request to this, actually, to this vector storage, you say, BAS, give me information about, let's say, the products of my store, starting with the letter A. And it returns to you some volume that you, like, set for it, of these very products. This is a wonderful thing, it is used a lot now. So. And this also needs to be allocated to separate agents, because even just getting information from a superbase is not trivial. If you try to do it, you will succeed 100% easily. But, as they say, the devil is in the details. If you work with some complex manuals, complex instructions, for example, my personal case, which we are currently working on - it is obtaining information from a manual for an audio equipment manufacturer. This is very difficult, because these manuals are complex, they have a lot of details. Plus, those who wrote this manual are also people. They make mistakes, sometimes not so much technical as logical. And neural networks, unfortunately, cannot tolerate these logical errors. They perceive it as a clear instruction. Therefore, if some important word is missed somewhere, well, then, the neural network starts to get confused. Therefore, you need to give certain clear instructions and separate. An agent for searching, for example, information. So, the second agent receives this information ready from the agent and does something with it. For example, it gives it back to a person in a human-readable form, wherever they say. So. And the last, perhaps, point, when to apply tasks that exceed the computational capabilities of one agent. This all concerns context, because the context of one agent, let's call it, say, one, or 10 contexts of different agents. Yes, somewhere the information will be incomplete, something might get lost, but in fact, the tasks are solvable. So. And you simply increase this, like, volume of context that you can give to your system, when not to apply. These are tasks solvable in one step. Well, this is, for example, two times two. For this task, you don't need a math agent, you don't need a text writer agent. You can simply ask not a hand, but to answer. Honestly, you don't even need any system instructions here. Therefore, if you have a simple system for automation, just, for example, question-answer, you don't need to search for information or somehow respond to it, for example, an online psychologist, so. And a psychologist, rather, there are many of them now. So. And I would suggest solving everything through a mono-agent system, when you have one agent, and it answers. So, fast requests where response speed is important. And this is yes, because in fact, multi-agent systems, although they win in speed due to parallelism, that is, due to the fact that you can divide these sub-agents somehow as you want. So. But at the same time, a mono-agent system still wins in speed on small tasks. That is, on complex ones, a multi-agent system wins, on small ones, a mono-agent system with one model, without any clear instructions, definitely wins. It will simply give you an answer. So. Plus, tasks with a clearly defined algorithm are also better solved by mono-agent systems. This is simply not even that it's better. A coordinator and workers will solve this task perfectly, but they will do it longer. It's still an unnecessary complication. If you have some kind of task, for example, I've often seen how they use this, they throw a JSON and say: "Make it into a human-readable format." So the task is solved. For this, a multi-agent system is not needed. In general, nothing is needed for this. The task is very simple, implementable, so in general, you can not worry. So, projects with strict budget constraints - this is probably the biggest minus of multi-agent systems. They are much more expensive, they require much more of these epic Ops, that is, these calls to copy the models themselves. So. And they, actually, eat up much more of your money. This is a certain price for quality. So, optimizing this is practically impossible. This is like, just logical, logical consequences of these multi-agent systems, because now you pay not for one agent, but for 10. So, although it grows non-linearly, it grows. So, here, I think, it's clear. So, and the task fits into the context of one window of one agent. Here, I think, everything is also clear. These are all simple tasks that do not require any large contexts, do not require the use of many different tools. Everything concerning simple tasks is all mono-agent systems. So. And problems requiring a unified context for decision-making. This point is a bit controversial. So, I think multi-agent systems can be implemented so that there is, conditionally, a Decision Maker agent, to which all context is transferred. But basically, I think, a mono-agent system should be used. And then see, then, like, make a decision. So, let's move on to a more interesting practical part. This is the creation of your own agents in N. This N, for those who are not familiar, is a code tool for automation. Probably one of the most popular now, if not the most. So, what's great about it is that it's visually very understandable. I didn't draw this particular construction. I just quickly whipped up an agent in ten. to understand how it roughly looks and how it works. So, specifically here it's a search agent that uses the Gemini model. So, it's in the green square, and a certain set of tools is connected to it. This is the yellow, well, a square-rectangle of an unclear color with the inscription Search Tools. The first tool is the Search Engine Tavily. In general, all the tools presented here are some kind of search engines designed for neural networks. Well, probably except for Brave. Brave is more general, it's more common. So, Fcrawl, the same thing as Tavily. You can ignore it. So, this is essentially, a browser, but of a certain type. You send a request to it, and it returns not a set of pages, but a set of information. These are specially adapted information sources that, in essence, work like a browser. So. Well, and with their own quirks. So. So, if you use these three, you can remember them, you will use them in approximately 99% of cases. I probably haven't used others. These are enough for 99%. So, where were we. So, plus N, is that you can create an agent node, this is this block, and connect anything you want to it. That is, for example, you can connect any model, so that, well, imagine, you even need to make your own tool. For example, you don't want to go to 10 different sites every time, but you want to use 10 different models without their specific features, purely so that they answer your questions. So. And you can remove any model you need here. You can make several such bots, connect all the necessary tools to them. The set of these tools is gigantic, probably for almost all cases of life. So. Plus, you can also restore community tools. That is, these are tools made by someone from this community, actually, for work. So. And how does agent creation look? Since we are talking about multi-agent systems today, the simplest one can be implemented in N. For this, you don't need to write code, you don't need to know a lot of technologies. Everything is implemented quite simply. Here you can see that there is a coordinator. This is probably almost the only possibility now, actually, to do only this. I don't know if a topper is possible or not. I don't think so. So, there is a coordinator, some model is connected to it. You can choose any here. Honestly, I chose a model from OpenAI, simply because everyone recognizes the logo. So, you can connect memory. I connected the internal memory of the system itself and some agents. Agents look like this. So, here's a search engine, this is when executed by workflow - this is a trigger, that is, this is a trigger for activation. Here, essentially, everything that the neural network transmits is transmitted. That is, when, let me draw it. When the coordinator transmits information, it says: "Search engine." So, an arrow. Gather information for me. I, let me, I will manage to show this with a live example. Find me the weather for tomorrow in Nizhny Novgorod. So, Search searches, it starts to turn green. Let's even color it. It, for example, worked, worked, and returned this information back. Analyst analyzed all this, that is, the coordinator goes further and tells the analyst: "Analyze the information received from the Search agent and find me 10 points." So, it actually analyzed it, it also worked. This, by the way, can happen not in one iteration, and this is the main advantage of these multi-agent systems. And returned the information further. So. And creator. The same. This is the simplest. So, one second, I'm a little bit. So. Excuse me. To. So, now it should be visible. Unfortunately, all my scribbles disappeared. Is it visible, right? >> Yes, everything is visible. >> Excellent, thank you. >> Uh-huh, super. So, all my scribbles, unfortunately, disappeared. But this is an approximate version of how a sub-agent system works. Anyone can do this, because each agent is implemented as you need it. Quite simply. You wanted to make an analyst, connect tools related to analytics to it. That is, for example, you need it to access Google Analytics, collect some information. Connect TO for Google Analytics, and it collects information. You made an agent for searching information from a superbase. Now you have information from a superbase. Here, I think, everything is clear. So, let's try it in practice, shall we, Anastasia? I apologize, we have received questions about NITN, and I also see other questions in the chat. We will answer them in detail after the main part of the lecture. So, let's dwell a little on this tool. Please tell me, and they are asking: "Can you recommend some interesting tasks for practicing NN, which would also be useful? That is, what can you come up with? So, here, probably the main thing, especially when you are learning, is to use what interests you. Do what you want to implement for fun, so that it works. Try to implement some of your daily routine. Try to do it with some simple examples, for example, a search agent. That is, try to replace cloud de dprarch, try to replace gemini depressch, that is, try to implement a system that will search for information on the internet itself and make some report for you. So, try to upload. By the way, actually, on YouTube now, there are probably more instructional videos on inн than needed. It is generally quite understandable. So, let's switch to an example now and show you. So, use what your soul desires. If you want to generate pictures, generate them. If you want to make some animated videos, for example, this is also solvable now. So, you just need to use paid APIs and that's it. So, everything else is there. >> Cool. Thank you. And also on this tool, how much does it cost to use it, and do you need to have paid subscriptions for neural networks? The tool itself, I think, the Cloud version costs around 20 dollars a month. I might be mistaken, I don't use the cloud version, so I can't tell you. I recommend using SH hosted. That is, you deploy some server yourself and deploy NA10 there. This is done through the Coolify tool, through a nice Uнтерfйс. And this is all free. That is, you will only pay for the server itself. This costs around, well, 50 dollars a month, so five to ten. >> Uh-huh. >> Paid subscriptions for neural networks, you use IP keys, and it varies everywhere. Gemini, I think, your IP keys are included in the subscription. For all others, I think not. Well, that is, OpenAI definitely doesn't include it, Clчно doesn't include it. Gemini, well, honestly, I don't remember. So, I mainly use klod and pti. So, you get your opilyuch, throw money into the balance. This is not related to your main subscription at all. It doesn't correlate at all. And use it. It's all counted in tokens. Roughly speaking, how heavy your request was, how many of these certain tokens the neural network consumed. So, this is a conditional unit of measurement. So. And, actually, that's how much it all costs you. It doesn't cost much. I spend around, well, 10-20 dollars a month on apishki. That's quite enough for tests. So, specifically subscriptions to neural networks, yes, I have subscriptions for about 20 thousand. It's expensive. So. But I just have a lot of them. So. And many places have max trifes. So. So, here, probably, you look at the budget, at the needs, how you need it. Uh-huh. And in the context of our conversation, as I understood, there are downloadable models. >> There are downloadable ones, they are just open-source, that is, there are, there is LLaMA, I think, it also became open-source. Open released its open-source model. >> So, and small, small connection issues. I think now, uh, Alexander will rejoin, and we will return. Uh, to the discussion. Uh, in the meantime, I see how many questions are coming into the chat. Uh, this is very pleasant for us. We will definitely answer all of them. Uh, only uh-huh, what is being covered during the lecture now, and what is additional, we will cover afterwards. So. And Alexander will be back in the studio now. Yes, he will rejoin now, yes, literally. a minute of patience, we will be very grateful to you. Uh, in the meantime, let me remind you that you can ask questions not only in the chat, you can also do it under the announcement of this broadcast in our Telegram channel. And the link to our Telegram channel is in the description under this video. Alexander has returned to us. Alexander, we were waiting for you very much. Uh, such connection issues. Fortunately, everything is being resolved. >> One more >> one more question. Uh-huh, what do you use Memory for in the context of that tool N8, uh, NateN, I apologize. >> Specifically there, honestly, I just clicked it for the slide, to show how it actually looks. So. Uh, it's not always needed. Memory is specifically used to remember what was in your correspondence. There are different memories. There is, for example, ZP - this is a full-fledged graph memory, there you can see interconnections. So. But specifically on the slide, I just clicked to show how an agent generally looks. That is, that there is a model, the most basic one, there is memory, it can be used, there are sub-agents. So, in general, this actually helps to recall some context of correspondence. So, well, you need to use it wisely, you need to understand why. >> Yes. Good, thank you very much. Let's return to the lecture, uh, to all other questions, as I already said, we will return a little later, literally in a couple of minutes. Uh-huh. Thank you. >> Uh-huh. Uh, in general, we have the last slide left, probably, it will be most useful for developers. This is the use of Cloud Code subs. This is a relatively recent feature from Anthropic. So, respectively, the creators of this Claude. You may have heard about the Claude Opus Net models and, respectively, about their application. Maybe someone even uses them. In general, in my opinion, this is now the strongest neural network for work, for general daily tasks, that is, for working with text, for working with code. Claude code in general is now almost an ideal tool. So, I highly recommend it to everyone. If you haven't tried it, definitely try it. So. And recently, they added subs, that is, the use of so-called sub-agents, where you can assign each a certain set of permissions, which is important, and a certain set of tools. So. And, respectively, a clear instruction, who they are, what they are responsible for, and so on. So, here I made a test screenshot to show how it all looks. They work in parallel. Each has its own to-do list, so you can see here, it's a bit poorly visible, but this unclear color one is the frontend developer. So, for example, it creates, makes the command NPX create next. So, and creates some application. So, DevOps makes doockerignor and is preparing, apparently, for some launch of these applications when they are ready. So, test engineer makes a to-do list, what they need to test, what blocks will be in the application, and how it will be tested. So. Plus, there are even MCPs like papetier, which allow you to open a browser and somehow look at, actually, what's happening on the site. Here, all this is also available. Security Guard is responsible for analyzing, actually, how secure our application is, whether there are any vulnerabilities. So, here you can even see that it, for example, uses npm audit to check npx packages and docritter. This is probably almost the most important agent that is here. And also two, they unfortunately didn't make it into the screenshot. We will discuss them too. So, it writes documentation. Documentation for code generation is super important. This is probably one of the most important things now that exists in general. Because if you leave, finish the work, returning to the same feature will be very difficult. Neural networks forget context, analyzing code and so on, it's all difficult, it's very voluminous. It's unclear what the plans for the future are, what other agents are doing, and so on. This cannot be lost, otherwise your neural network will get confused. Therefore, it helps with x. That is, this is probably the main application now, like, of these docwriters for generation - this is documentation for everything you have in the application. What API endpoints you have, what methods they have, how many pages, what pages they are, what's on the pages, so what's happening with your backend, what database you are using, what columns are there, and so on. Everything that exists must be documented. So, for this, it is very convenient to use another agent, well, a separate one. So. And at the same time, if neural networks forget some part or they need this information, they can find the necessary MD file in the docs themselves. MD is the most adapted format for this. So. And see, what's actually happening. So. And, so, what other agents did we forget, this is the coordinator. That is, mine is called product manager, to emulate a full team. it distributes tasks between two agents and, actually, looks at what's happening there. So. And architect - this is an agent that is responsible for, well, making the architecture of the entire application. At the same time, not only the architecture itself, but also in general everything, how it should look from a technical point of view. So, this helps neural networks to focus specifically on their tasks, and not to think them through. In this regard, it's quite harmful for them to think. How it should look. The architect has thought everything through, you have discussed it with him, reached some common solution, and, actually, checked if everything suits you or not. And then future neural networks will be able to approach the execution of their tasks more selectively, so to speak. They won't have to invent anything. They simply received the task, received the input, received the context, and do it. This helps them not to get confused. So. Then, probably, that's all I have. Here is my contact information. I think, Anastasia, we have time to show N in practice, yes, in the chat, right, for those who are interested. >> I think so. In the meantime, can I just ask one more question that came up along the way. Uh, is memory for graphrek? They are asking us. I say, well, depending on how you use it. There is graph memory. So, unfortunately, here I probably can't know exactly what graphrek is, but there is graph memory. You can use it so that your agents remember, actually, interconnections. So, this is more like human memory. That is, you have some object, and it is somehow connected to other objects. So. So, there is an agent, oh, there is an agent with agents, excuse me, the presentation is completely in my head. So. So, there is, conditionally, an object, say, Sasha. So, Sasha talked about multi-agent systems. Anastasia was watching his live broadcast. So. And all this can also be placed in graph memory with some interconnections and details. >> Uh-huh. Thank you. And our subscribers, our viewers, I would even say, demand a demonstration, so let's, let's show it. >> Uh-huh. So, well, I actually made a simple one. It's a bit different from the screenshot, it's a bit simpler. A multi-agent system. So, it's more for an example of how it works. Unfortunately, I didn't have much time to program all this. So. But here's a simple example. We ask a question. The question is asked to the coordinator agent. What is the weather in Nizhny Novgorod for tomorrow? Let's even turn off memory so that it doesn't take it from memory. So, look, it received the question. That is, the coordinator, now it will work. it will return some answer so that we can see everything. So, tomorrow, August 13th. Tomorrow, tomorrow is not the thirteenth. Well, it made a mistake, right? Because of this, no, tomorrow is the thirteenth. Well, never mind. So, in the morning, the temperature. And here it returned some of its information, how it looks for the coordinator. It received what is the weather in Nizhny Novgorod for tomorrow. So, this is chat input from this chat. Some input point can change for you. You specify what to pass as a prompt. So, you can pass a system message. It, by the way, was here. It got lost for some reason. Well, never mind. So, let's fix it now, let's try. So. And you give it a choice of these sub-agents. So, don't look at this, this is separate. So, we have a working agent - this is a search agent. So, this is more for beauty. So. But let's look. Let's break it down by agent, like, by one coordinator agent and by one agent performing a specific task. Let's move on. Now let's save. So. And let's move on. Let's see how our Search sub agent works. Yes, everything has been transferred. Super. So, Subagent. Let's even copy to editor. This is, by the way, a useful feature for those who want to debug all this. So, you pass. So, look, it asked: "What is the weather in Nizhny Novgorod for tomorrow?" So, your Search Agent analyzed all this. So, it even has some system message here, which can be viewed. And used the Tavily tool and sent it. Weather in Nizhny Novgorod for tomorrow. Tavily returned all this in this format, so what URL it viewed. So, weather pogoda.mail.ru and content pogoda.mail.ru in Nizhny Novgorod for today, tomorrow, and the coming days. So, with a summary and what else? And Yandex weather. So, this kind of content. So, it returned all this, the coordinator received all this and gave it. So, let's see if it's visible better now. For example, like this. Weather forecast in Nizhny Novgorod for tomorrow, August 13th. Well, there's a glitch with August 13th. All this is fixed with prompts. Actually, this is not a big problem, because neural networks don't understand what today, tomorrow, the day after tomorrow means. So, all this can be configured with this system message. So, so that it rechecks it and maybe even has it. So. And here it gives you a human-readable version. Well, like human-readable, for returning to another neural network. What it analyzed, what sources it reviewed. So. And here you see the links? I hope it's poorly visible. So. But I think on the broadcast you see it a little better. So. So, it returned all this with links, returned the weather forecast in some volume. So. And returning back. So, save. So. On slides. So. Returned in some form. So. And, actually, returning back. Now copy to editor. So. How it was. Now let's ask a question. Let's even ask a new one. What is the weather today in Moscow? Penacent. So, it also analyzes. Let's, by the way, even fix it now if it returns dates with an error. Let's try to fix it. What are the main, probably, recommendations? What can I say? Well, first, on YouTube, really, there are more videos on it than needed. You can easily watch it, so easily find it. Lots of examples, especially on English-speaking YouTube. There are a lot of them, so, for literally all cases of life. Therefore, I think there will be no questions here. So, on the main components, I think it's also clear here. You can specify any model. So, look, let's delete the connection. So. And here you can see all the models that are available. So. Don't like Open.ai, let's use Anthropic's. So, let's use Anthropic's now. So, that's it. And let's ask the same question. What is the weather tomorrow in Nizhny Novgorod? So, now, as we can see, the Claude model is thinking. It's more structured, Claude in general is more designed for developers, so it has more structure. So, in general, I think it's more than clear here. So, the main thing you will operate with is the prompt, this is the user message. It can be supplemented. That is, for example, like this, and drag it. So. And supplement it. You say: "Answer the user's question." colon, and like this, and answer the user's request. Colon, what is the weather tomorrow in Nizhny Novgorod. And you can also do this, for example, and stop in quotes. So. It supports various magic operators like JavaScript, these two little things. So, it's better to look at this in videos. So, it's a long story. So. And here you see how we changed the model and how structured the answer returned. Tomorrow, August 13th, and you need to look at the prompt why it's not like that. The main ones are shown for tomorrow morning, afternoon, evening, night. Recommendations. So. And compare it with the answer from OpenAI and ChatGPT. So. So, in general, that's it. So, this is probably what I wanted to talk about today. >> Thank you very much, Alexander. Uh, in parallel with your

Questions are pouring in from the demonstration. And please tell me, the answer was from three sources, which one has priority? This was asked a minute ago. This can be configured. That is, if you specifically use, for example, a search agent and connect Tavily Firecroll Brive Search to it, if you, well, based on experience, know that Tavilley gives better answers, you can specify in the system prompt, if there is, I'm making it up, if there is a controversial situation, give priority to answers from Tavilley. That's all. That is, here you operate with this system prompt. Here it is, for example. No, and this can, by the way, improve the situation well. This is the system message. This is a message that is sent before the user's request. Here you can say, for example, "You helpful assistant. You are a weather forecast assistant." I, by the way, recommend generating all these prompts through neural networks, not writing them yourself. They do it 100 times better than a person. Well, first, use the search agent to search for information on the internet. Second, check the date there. Do not always use the one the user asked for. Well, let's see if this helps. Well, here's the weather for tomorrow in Nizhny Novgorod. Bang. Now it has a slightly different system prompt. It is likely that it can double-check it somewhere. It can go through this search agent several times. It can call these sub-agents several times. Well. No, it didn't help. Well, so be it. He >> Yes, sorry, I interrupted. >> Uh-huh. No, no, it's all good. >> Uh-huh. One more question. Can it work regularly? Are requests sent every hour or continuously? Yes, continuously is possible. This flag is responsible for that. It says "inactive" here. You switch it, and it becomes always active. Here, mainly, well, by the way, you can even make this chat window public so that it is accessible via the internet. But probably the easiest way to make a bot that will always work is to use Telegram. >> Here. That is, you use this "on message", it becomes here instead of this chat. Here. And your Telegram bot is always active. Here. If you are interested in how a chatbot works, you can google it on the internet, but basically it's just an API within Telegram. That is, N listens to what is happening in this API and reacts accordingly. Commands can even be configured here, as you wish. Alexander, our viewers don't want the internet. Our viewers want you. I suggest we practice working with Cloud Code and sub-agents. >> Oh, that's a long one. That's a very long one. >> Firstly, >> yes, yes. >> Consider this as a prospect, >> I think we can consider it, think about it. Here. Well, it's about an hour and a half, because, firstly, the code generation itself takes a certain amount of time, plus debugging is still necessary. No neural network currently copes perfectly. Only some conditional Lavable N10, oh, not NATN, V0 and so on. WSF, for example, they generate simple websites without any super complex functionality, but they work almost perfectly. >> Uh-huh. We'll take it. >> Agreed. Ah, and one more thing, there's a request. Ah, it's interesting to quickly run through Cloud Code sub-agents, how it works there. There is knowledge of Python, elementary level. I think this can also be considered as an idea for the future. And now let's move on to more general questions, right? >> Uh-huh. Yes, let's. >> So. Ah, and so, let's go in order. Does a multi-agent system for software development management exist today? >> A prime example is Cloud Sub-agents. And in general, V0 Lavable. They use different agents one way or another. That is, they are not single-agent, they have their own trick, they don't particularly reveal it, but we can assume they have several agents, each responsible for its specific functionality, because the same conditional one cannot draw design and generate code. It will do both poorly. So here it's 100%. >> Specifically process management. Uh-huh. >> Yes, specifically processes. >> What do you mean by processes? We hope, if you are still with us, please clarify what you mean by processes, what processes are in your understanding, and we will return to this question. And now let's move on a bit. If you put many sub-agents, you'll hit the neural network limits for the number of tokens. How is this solved? Sub-agents are important for this. >> Ah, yes, frankly, not at all. It's solved with money. You just throw more money into the API, and it will do it for you. This is a certain trade-off with speed. Here you need to understand, like, what are your priorities: to make products faster, better, or cheaper. Here. In general, actually, it turns out to be six of one and half a dozen of the other, because here it's like driving fast or slow? You drive slowly, you use less gasoline, well, and you drive slower. Here. And you'll spend plus or minus the same for 100 km. If you drive very fast, you've covered 100 km, consumed a lot of fuel, here, but you've taken less time. In general, it's the same here. It's practically impossible to optimize this. You just need to, well, not go completely crazy with a billion agents. You need to understand, like, what it is and why. Here. What? So. >> Uh-huh. Ah, and clarify processes. Ah, development processes. What is meant by processes? That is, are these processes of managing development by people or also by agents? If by agents, then is there a multi-system management of >> processes today? Yes, like AD. >> I don't know about that, probably. It will be difficult for me to answer. I know that there are now MCPs that allow, for example, creating tickets, analyzing statuses, analyzing comments, and so on. MCP, I don't know if I need to expand on it or not, I'll expand on it just in case. This is model-based protocol, it's, roughly speaking, a plugin for a neural network. You can use this, for example, in Cloud Code now. You can connect anything you want. And in this vein, I think you can connect a conditional plugin. Well, like an MCP for Jira, I don't know if it exists by default or not, it probably should. Here, which will analyze what's happening with your tickets, and create, for example, new tickets. You can write to the bot that we have, for example, a bug, an object cannot be created when you click on a button, and through the Telegram bot, all this will be sent to the MCP and it will create a ticket for you. That can happen. Alexander, they are begging us to do a stream about Cloud Code. I suggest we listen to this in the future. And now we have many, many questions. Ah, hello. What settings do you use for the coordinator? I mean, do you set a prompt for it or in another way? >> In general, I always generate prompts for all agents through neural networks. That is, you upload some context, some requirements, what you want to get, and ask to write a detailed prompt to solve a specific situation. It copes with this perfectly. I have often encountered people giving lectures on prompting. I consider it complete nonsense and a scam. Generate all this through and that's it. And don't worry. Here. And it does it 10 times better than you, like most tasks now. >> How is this done? That is, just an expensive LM, write me a prompt and then you write, explain in your own words, and it rewrites it for you, or how does it happen? >> Yes, yes, yes. And it gives you a detailed text. Well, you tell it: "Draw me, actually, now most neural networks work under the hood like this." When you ask GPT, for example, to draw a picture of a cat, it's not "draw me a picture of a cat", actually, magic happens under the hood. Here, first, there is a conditional check for NSFW. NSFW is not safe for work content. That is, GPT will not draw a naked woman for you, unfortunately or fortunately. Here, they have it locked by policies, and it all works strictly at the level of these pre- and post-checks, by the way, because ChatGPT itself copes with this perfectly. Here, their generation model, it generates anything you want, but it cannot give it to you. Here. Then there is a conditional prompt improving. That is, it takes your prompt and modifies it to make it normal. That is, instead of "draw me a picture of a cat," it's "draw me a ginger cat against a background of a birch forest with a starry sky at 12:00 PM." Here. And such a large thing gets into the conditional final model. Here. And then it's generated. Here. So you always need to write a detailed prompt, because it will figure out a lot for you. Well >> Think for yourself, otherwise the LM will think for you. Ah, yes, yes, yes. Yes. >> We have a question from the Telegram chat. How is this deployed on a local system? I mean the tool, excuse me, N8N. Judging by the time, it's about that. >> Uh-huh. I think the site is called. That is, it's a system for, roughly speaking, Docker, but without Docker. That is, you can click on what you need there, what you need, and just click deploy. From the knowledge you need, you just need to link a card to a hosting provider like Selectel or the hosting providers you use. I have foreign ones. I won't advertise what I use. Here. You can take a conditional Selectel, link a card there, deposit money, and create a virtual machine. Here. It doesn't have to be super powerful. I think it will cost no more than 2,000 per month. Here. You deploy it with one command. Lifay, they have a bash command on their page. Well, you just copy it, paste it into the terminal, and then the magic happens. It works by itself. Here. You open the IP address of your virtual machine in the browser. It's available on some port. Here. The port can also be available. If something is unavailable, VS Code. Here. Open it, connect. Here. In the terminal, when you type Lifay, you can do port forwarding. Here. This sounds complicated. In reality, you have a "Ports" tab there. You just, I don't remember which port it is. Well, let's say it's 8080. Here. You type 8080 and press Enter. And that's it, the port is forwarded between the virtual machine and, in fact, your local computer. Here. And you open localhost and the port you specified in the browser. Everything works. The only thing you need to do is set up a domain. Here. Because many services don't work without it. Plus, it will be a nice link. Here. You set up a domain, do a little bit of unpleasant things with these domain names, records and non-records. The first time it can be quite painful. Here. Well, patience and hard work and Google will overcome everything. Well. Then in Lifay, you just specify your address in the settings. It checks the domain record itself. Here. And it deploys itself. That is, there's nothing like that here. It's true that the first few times, well, it's unclear, it's difficult. In reality, I think you can figure it out without N8N. >> Uh-huh. Ah, and a clarification comes. What if you have your own equipment cluster, the data with which you work, excuse me, with which you work is under commercial secret? >> Uh-huh. Well, as far as I remember, legislation strictly prohibits working with foreign neural networks. There is probably a workaround to create a separate bus between foreign neural networks and your conditional data cluster. Here. But you cannot upload any personal data. You cannot upload any sensitive information anywhere. You cannot upload it anywhere. Neither to Russian neural networks nor to non-Russian neural networks. Here. Because it's like with the internet. Everything that gets there will be used for training and may be revealed to someone. Here. And I don't know if you've seen it or not, stumbled upon it, but ChatGPT indexes public links. That is, if you click "share chat," it appears in Google, and any well-intentioned person can go and read what you're chatting about with it. Here. So you need to be very careful with this. This is a certain hygiene of the 21st century. Here. If there is something that cannot be disclosed anywhere, you can deploy open-source models. Here. But using something external, I think, is not allowed. >> Uh-huh. Ah, and they say there are locally deployed ones. >> Well, you need to check if it's a trusted source and it really works locally, well, you can accidentally download some phishing software. Well. But I don't know, people already download viruses. Here. So, if everything is verified, use it. It won't go anywhere. It runs locally. It's not cheap. Here. There's only one nuance, because you'll have to rent a server, 99% you'll have to rent a server with a GPU. For example, I amused myself with Huanyu. This is a Chinese neural network for video generation. That is, it has text-to-video, and image-to-video. It doesn't have any restrictions. You can generate anything your heart desires. Here. And you can also use purely your own and create, for example, your own service for generating cards, for generating video from an image. >> Uh-huh. Ah, and a separate discussion has already unfolded in our chat, and we are moving on to the questions from our subscribers, our viewers. Can Cloud Code sub-agents be connected to Lavable? To >> coding sub-agents Lavable? I don't understand why, because it's a service for generating, in fact, websites that they have. Here. It deploys it itself. And I think not. And I think it's just a stupid use case. Here. It's for different things. That is, inside it, it codes itself, deploys itself. Here. I think it's better to either code the application yourself using, for example, sub-agents, well, that is, the same Cloud Code, for example, or use Lavable if you just need a business card website. I don't see the point in overpaying now. You can just give Lavable three prompts, ask to fix it, do everything. Here. It does it perfectly, looks great, and works. Here. Which is surprising, it costs zero money. Here. So, only pluses. Yes. Ah, thank you. Ah, what server is optimal for running your own LM? >> It depends on the LM. Everything is very different. If, for example, purely text-based. I haven't worked with them myself, but I can assume they don't require much memory. If, for example, an LM for video generation, text-to-video or image-to-video, it doesn't matter, it will consume a lot. I tried my own experience, there were 4 GPUs. It cost around $5 per hour, and generating one video took about 15 minutes. Well, consider that adding more GPUs will take about 7 minutes. If you want it to work with some reasonable response, you need to be really good at optimizing work on GPUs, which I, unfortunately, have forgotten. And frankly, I never knew it at the level where I could use it super effectively, but the Chinese definitely do it. For example, they have Mimic Mimic PC service, where you can try some hosted neural networks, where everything is already done for you in the Confi UI. Here. All these magical manipulations are done for you, how to use it, so that it works faster, but the quality is lost. >> Ah, so, ah, okay, thank you. Ah, ah, do you use MCP in CLD Code itself? >> Yes. Yes. I have several there. The first and probably the most important is sequential thinking. This is sequential thinking. I recommend this to everyone who uses CLD Code or Cloud Code, because it's an absolutely free improvement of results by about 50%. Here. Your neural network starts, roughly speaking, communicating with itself and thinking about what is actually required of it. It decomposes the task, it starts to reason, what is actually happening, what does the user want from me, how is this done. Here. It starts to comprehend some concepts, here, and only at the end, after some reflection, it gives some result and proceeds to action. Here. Because without it, you get a stupid answer to your question. A stupid answer to your question often leads to nothing good. Not just, for example, text generation, working with text, also turns out to be nonsense. Here. And Cloud Code. Uh-huh. Ah, Alexander, we continue to receive questions in the chat. You are thanked and they demand you to come to our studio again. Ah, do Cloud Code sub-agents replace agents through N8N? >> No, these are tools for different tasks. Cloud Code sub-agents are strictly agents for development or analysis, for example, of text. By the way, a life hack for those who will work with some massive document. Cloud Code won't fit, it won't fit anywhere. You upload it to Cloud Code and ask it to analyze and break it down, for example, into reasonable parts. Here. I'll send the MCP name. Here. >> Thank you. >> Ah, >> yes, you ask it to divide, it will divide. Ah, agents in N8N, you can create anything you want there. Here. It's more of a set of tools, more of a set, in fact, for automation. You can put specific custom logic into it. That is, you, for example, want your agent to have some condition. Here, by the way, they sent it to the chat. Thank you very much. Thank you. >> Here, it has some condition, it has a command. This is useful for Telegram bots. It simply writes that when it receives a message, it checks if there is a slash, for example, in the command. >> If there is, it goes down one branch, if not, it goes down another. That's all. For example, I had memory cleanup this way. A convenient thing for testing, by the way. Here. That is, a slight deviation from the workflow. If you write a slash, your session memory is completely cleared. Here. This is an excellent approach for debugging, so as not to mix warm with soft. Uh-huh. >> How to do that? Ah, without N8N? You really need to write a separate program and use different APIs. Without >> N8N, there are several, well, depending on what you need. If specifically without N8N, there are a lot of other tools, there's Make, there's Zapier, and there are actually very many of them. You can search. They all look more or less the same, work more or less the same. Here. Each has its own advantages and disadvantages. If you need a fundamentally different approach, that is, you need to write code, then yes, you need to write programs, you need to write this logic in code. This is a much freer condition. You can do anything your heart desires there. Here. Because N8N is still limited in scaling. It is, of course, a great thing for launching MVPs. Here. But a certain number of requests, for example, it no longer withstands. Here. Whereas in code, it all depends on your skills, abilities, and so on. Here. There are many different libraries from completely different large companies for creating these agents. Here you can also use tools, you can also create MCP servers, you can also use memory and anything else you want. >> Uh-huh. Ah, and we have received another comment. Please invite Alexander again. We are inviting him right now. >> Thank you. I am very pleased. >> And now let's move on, please, to the next questions. Ah, another question about sub-agents. If they work in parallel, how to ensure that they don't conflict and that one doesn't edit the same code according to its own logic? Well, first, there's the option through Work 3, Git Work 3. For those interested, you can read about it in lectures, you need to delve deeper, but basically, just create agents that will not use the same pieces of code. That is, for example, front-end uses, well, completely different folders. The back-end developer doesn't need to touch the front-end and change something there. The front-end developer doesn't need to touch the back-end and change something there. Here. Monolith. I don't know how to write multi-agent systems. Unfortunately, I don't know how to write monoliths at all, and I recommend everyone not to know either. Here. So use an architecture where each separate block works, well, conditionally, independently. Here. This solves many problems. Of course, new ones appear, for example, connections. Well, it can forget, but this can all also be done with Docker. Here. That is, with a folder of files, a folder of how your application works, and so on. >> Uh-huh. >> Ah, returning a little bit to this tool N8N, please tell me, you yourself said that there are many other tools on the market, everyone has their pros and cons. Why are you talking about and somehow paying more attention to, using this particular one? >> As far as I know, it's the most extensive. That is, first of all, it's the most flexible. Here. Secondly, it's one of the most powerful, because it's more complex than all the others, but at the same time, it has more customization options. Here. And plus, it's hosted. Here. I don't remember what other hosted tools like this exist. Here. And I try to keep all my infrastructure on my own servers. >> Uh-huh. On what principle does the agent coordinator choose which sub-agent to delegate execution to? Agent coordinator. Well, can you stop the demonstration, please? I'll show you. >> Yes, of course. >> Uh-huh. Look here, there's a description. Description is, in fact, why and in what situations you should call this tool for a specific task. That is, here we have an agent. Call the tool to find information on the Internet. Everything is clear here, use this tool, use this agent. When you need to find information on the internet. Here you will have the next agent, questions. I just added it so that it's there. Here. Use this agent to find new clarifying questions, to get enough information. Here. It can also use it. It's just not specified in the system prompt here. Here. You can, for example, try to specify it. Here. Use, use. Ah, questions, what is its name? N for getting clarifying questions. Let's even make it so that it definitely uses it always, you can even do this. Neural networks react to this. Here. And, for example, what is the weather? Let's do rain, you know what? Let's make it the first one. Like this. So. So. Weather tomorrow in Nizhny Novgorod. Now it will return to our sub-agent. Yes. Look, it went. So, is it the correct sub-agent? Yes, the correct one. Look, it first went to the clarifying questions agent, and it returned. The user wants to know the weather forecast for tomorrow in Nizhny Novgorod. Lost information. Exact time of day for the forecast. Required meteorological parameters. For what time of day do you want to get the weather forecast for Nizhny Novgorod tomorrow? What weather parameters are you interested in? Do you prefer a forecast from a specific source or service? Here. And, in fact, it returned the information. It most likely used the search agent somehow. Here. Because, you see, there were two iterations. Here. First, weather forecast for tomorrow, temperature, wind speed, and second, current weather forecast for tomorrow, today's date. Well, here. And it also returned. That is, by the way, MCP works on the same principle. They actually have, let me quickly show you, ah, MCP. Here they have a description. Here you can see credentials to connect. This is what you add, let me even show you, you add this MCP server by adding commands, arguments, and environment variables here. You don't need to remember all this, it's there. Here, you will most likely see "npx -y -y name of your MCP that you want to install" written here, and your environment variables will be. Here, I'll switch now so you can see. Here, ABC equals 1 2 3. Here, this will be an environment variable. Well, here everything is clear. Here you only need to replace this, most likely. Here. It's specified there. That is, you'll figure it out in the instructions. Here. Close. Here. Tool description. Here, what I just showed, this is when to use it. You can manually set MCP client to and give some description. Here. Or you can do it automatically. Here there is some built-in description of what to use a specific MCP tool for. Here. It also works in conditional desktop applications. That is, in Cloud Code, for example, when you install these MCPs, you can't set them manually, but the neural network guesses what to use in what situations each time. Plus, it often needs help. That is, a good prompt so that the neural network doesn't get stuck is to tell it: "Use MCP such and such" for getting information on the internet, for example, and it won't forget to use it. >> Uh-huh. Super. Thank you. Ah, remove it. Yes, let's move on to the next. >> Yes, yes, yes. You can remove it. >> Excuse me. Ah, at the moment, they say there are problems with sub-agents in Cloud Code. They freeze, lose processes and context. Do you have such a problem? In general, neural networks have this problem, they get stuck with it. This problem existed before Cloud Code. This is partly solved by what I call "ix", when you can ask the neural network, if it's really stuck, to go back and see what needs to be done, what the current implementation is, and what.

It needs to be done. Here. But overall, this problem becomes much rarer when these cloud code subscriptions appeared. Before that, these neural networks were much more stupid. Here. Plus, one way or another, there are still updates with, say, 3.5, 3.7, 4.41. They partially solve these problems. Here. So I think the question here is probably more about how neural networks work in general, because they have short-term memory, but they don't have so-called, well, these connections like humans do. That is, you don't live by the last 15 minutes. You remember, for example, you leave home, you were going somewhere, you're unlikely to forget where you were going in 15 minutes. Neural networks will forget. They left home, looked around, they don't understand where they are, what they are, what they are doing here, where they were going, where they came from. Well, that's why you need to write these adogs for them. I also saw it helps with tasks, well, full-fledged tasks for agents. I haven't tried that. I saw an implementation through plane. This is, well, like Jira, open source name, where full-fledged tasks are created, statuses are set by the agents themselves, and so on. I haven't had time to try this option yet. Here. But it all depends on the implementation. And, but a person is still needed. To completely delegate the development of a complex application now. Not just Windows, but a truly complex application is impossible. You will still need to watch when the neural networks start to get stupid, stop them and say: "Hey, friend, don't do that. Do it normally." Here. And they will do it normally. Possibly not the first time. Here. But they will definitely do it. The main thing is to remember that it is still cheaper and faster than, say, a full development team. Or even if you do it yourself, it's 100 times faster. I have a friend who, without any programming knowledge, created an LMS for launching courses. Well, for those who are interested, I will appear there soon, I think I will have an announcement. Remember that everything that is done there is done absolutely without programming knowledge, purely by code. >> Super great. Alexander, come more often, they write to us. This is just super. This is complete delight. Make a lecture on Claude. This is Megan. >> Thank you. >> I think they are leaving us no choice but to actually give a lecture on Claude. A demonstration, perhaps. >> Uh-huh. >> Alexander, can we continue answering questions? Yes, >> yes, of course. I just wanted to say thank you, I'm very pleased. Here are more comments. >> Ah, thank you. Thank you. By the way, have you worked with Lin? Ngraph. Do you have any feedback on these tools? >> I haven't worked with them specifically. The creator of the product I know has worked with them. Unfortunately, I haven't had the need for such complex tools yet. If it arises in the future, I will definitely come again, I can tell you. >> Uh-huh. Thank you. Are there any working workflows for refactoring large systems by agents? For example, if I want to break down a monolith into microservices? Hmm, I think it should be tried iteratively through cloud code. Here. But there's a funny picture there, by the way, for those interested, you can find it. I launched a neural network, I don't remember which one, I think it was ChatGPT. You broke down the monolithic microservices. It worked for 40 minutes, it broke everything down. Very beautiful. Too bad nothing works. Here. But I think cloud code can do it. Well, it depends, of course, on the size of the monolith and its complexity, how much time it will take, but that it is possible, 100%. >> Uh-huh. >> There are probably no full-fledged tools yet. >> Uh-huh, I see, thank you. How do you combine aify and Supase? >> I don't combine them at all. IFify is a tool for deploying infrastructure on your own servers. Supabase is a database. Here. So they are slightly different things. Well, for example, through Supabase, you can deploy it yourself in one click, if you have such needs. >> Uh-huh. So, and how can I, excuse me, how can I test the front-end with agents in autopilot so that they themselves analyze what is happening on the screen? >> The browser emulator partially solves this task. The most, well, there is a separate web browser, which, if anyone has done parsing, raises Selenium and, conditionally, Puppeteer. Here. Plus, I saw somewhere on the forums that they also do it with requests, but I've never tried it myself. Here. But that's all I can remember. They take a screenshot of the screen, here, and analyze it. Now, in general, neural networks can analyze images, what is happening on them. Here. That is, you launch it, the page opens, the screen is photographed, here, and the neural network makes a conditional analysis of whether it looks normal or not. Some, say, minor interface glitches, it, well, it probably won't tell you, but big ones, for example, that you have an error in large letters across the whole screen, it will easily. Here. When, for example, you go to a page, click a button, and an error pops up. Here, especially somewhere in Django, in NPX applications, the errors are quite beautiful. Here, the neural network will have no problems with this, and you will spend less time. Uh-huh. Uh-huh. Thank you. So, clarification, is this through some MCP server? Yes, yes, yes. You can Google it. Playwright and Puppeteer. Here, it's even in some model context protocol libraries. There are quite a few of them. Uh-huh. And yesterday, when we published the announcement of your lecture, a very interesting comment came in, a bit, it seemed to me, a cry from the soul. A subscriber noted that there is a lot of talk on this topic, on the topic of multi-agent systems, but there are only a few competent specific architectures and solutions. What do you think is the reason for this situation? And is it really like that for you? And if so, how to fix it? Maybe read something, research something? >> Yes, I think you just need to watch. >> Training videos help wonderfully, because often, even if they are, well, synthetic, you can adapt them to something practically working. The entry threshold for automation or product creation is zero. That is, you don't need to get an education to learn how to use any tool or to automate something. You need a little free time and effort, here, and the ability to watch videos on YouTube, here, or in any format convenient for you. For example, I personally find YouTube inconvenient because I perceive text more. Here. But I understand that not all training videos are in text format. That is, I also watch YouTube, quickly skip through some things, here, and it all works. Regarding practical, well, multi-agent systems are generally a relatively new direction. Well, within the overall development of AI. There are still not many real practical cases. Those that are, let's say, demonstrative. I probably showed them today in the presentation, but in general, what can I say, now almost all agents are multi-agent systems in one way or another. It's not just access to an LLM deployed on a server. They have a lot of magic under the hood. Now, when training LLMs, the entire internet has already been uploaded there. Information will not increase significantly. Here. Therefore, the competition now is in algorithms, that is, how much people know mathematics, here, how they optimize all this so that it works faster, cheaper, better, and so on. Here. And, accordingly, with these agents, prompts, all of this. >> Uh-huh. >> So. >> Thank you. And if we return a little to the architectural patterns of multi-agents, can it happen that agents in peer-to-peer, well, so to speak, do not agree with each other? And what will happen then, and how to avoid it? >> I think this question should be asked to NASA, not me. Unfortunately, peer-to-peer is more in the realm of robotics, in fact, for basic automation, peer-to-peer. Well, a more debatable thought. I think something similar to P2P can be implemented in Cloud Code Subscriptions, that is, to remove coordinators and let them communicate directly with each other. Here. But they don't disagree, that's not the case. This is more human stubbornness. Here. But neural networks are still used to, so to speak, respond to a request. They can get into a long loop of asking each other clarifying questions, but in general, they always have to agree, they have no other options. Here. The only thing is if there are contradictory conditions, that is, if you give them exactly opposite instructions, how they will behave here, well, I can't answer that. But that's a question of your implementation, why you did that? >> Uh-huh. Is it possible for one of its assistants, sub-agents, to take over the role of coordinator? >> Takeover, I think if you implement it, as we have done, they have a leader selection, that is, they have a constant, well, check. The leader is not even, not so much sane, he doesn't even exist. He's not broken, not stuck, not drowned. Well, it's hard to drown on Mars, of course. Here. Well, he didn't fall into a crater, he wasn't blown away somewhere. Here, he moves somehow, or doesn't move. Here, if they realize that he is stuck, not responding, stuck, here, they will quickly choose another leader for themselves. Here. Because why would a leader stuck in a rock? You need the one who runs the fastest, to conquer the Martian rocks. >> Cruel competition in the world of LLMs. >> What sub-agents in CLD code, in your opinion, are the most successful, the most useful? And are there any that, well, either don't make a difference, or are even harmful? >> The ones you implement? Well, there are some built-in ones. I don't use built-in ones because I like a certain control in this regard. Here. But purely the ones that you create specifically for your task. That is, here, I probably can't suggest anything. Here, first of all, you need to know the specifics of the task. Are you building a website, an application, if an application, then what kind, if a website, then what kind? This is, not OS. Here, in my opinion, among the harmful ones is Docker. Everything related to infrastructure, infrastructure should appear at the very end. Here. When everything is working perfectly, then you activate some agent that will write Docker compl, docker files for you. Otherwise, your feedback loop will just be extended. AI can get feedback in their terminal. Here. And if you have to rebuild Docker every time, you'll be amazed how much time it will take. Plus, for some reason, sub-agents, not just sub-agents, but neural networks in general, work poorly with infrastructure, with Docker, with Kubernetes. It seems their training didn't cover this. Here. This is generally, perhaps, a drop in the ocean compared to code development. Here, they start to get seriously stupid. Here. Uh-huh. Uh-huh. Alexander, last question. Please tell me, you asked about specific implemented projects that are in production, right, in our Telegram. Please tell me, are there any multi-agent systems that you have created, implemented, that you are proud of? Any projects that you can proudly say: "I did this, I implemented this, I participated in this"? >> Well, I don't disclose the names of my clients' companies, here, but I currently have certain multi-agent systems that automate processes in companies. Well, from the most basic, what I talked about today, which will be understandable, is, for example, research plus generation of marketing creatives. That is, a topic is found, and a batch of text plus pictures is released for it. Here. By the way, I generally recommend this to everyone who has a small or medium-sized business, who doesn't want to spend money on marketers. Here. This is a fairly simple way to save on salaries. Here. This is from the simple ones to understand. Here. There are more complex ones, when there is interaction with a vector database, search by embedding. For example, I touched on this topic today with audio equipment. Here, it's sometimes quite difficult, that is, one agent is not enough, and they already have it hard, they are poor, they don't understand what physics is and how to connect all this. Here, and you also have to use Supabase, and use it in such a way as to get something useful out of it. Here. Because each agent has a strictly defined task, how it interacts with tools in a certain way. That is, it's like this. >> Uh-huh. Thank you. Answering a viewer's question. Will this stream be available offline on our channel? Yes, of course. The stream will be saved at this link on YouTube. The stream is posted on our Telegram channel. You can subscribe via the link. So everywhere, everywhere, on VKontakte, Yandex Music, on all platforms. Alexander, I hope we are not abusing your attention. We have another question from a viewer. >> Well, yes, let's. I tried to make sub-agents on Claude, judging by what they were doing, they always lacked the information that was passed to them from the main flow, namely the project context. How to adjust this? >> This is exactly what docs are for, that is, so that agents write their own brief about what they did, what they are doing, how your project works in a certain way. Well, it's probably easiest to explain to developers. Here you have a backend, the backend has some API endpoints sticking out, so that they perform some information. The frontend developer doesn't care about these endpoints. He doesn't understand what they do, what they do, what they are responsible for. And you make a document for him where you say: "This does this, this does this, this does this, this does this." Here. You essentially describe all the components of your application or mobile application this way. Here. And the frontend developer will refer to this document, look, like, ah, I need my frontend block to work with this endpoint. Well, that's how it gets along with each other. Here. If you don't describe this, it will do it haphazardly. Here. Because it still has to answer your question somehow, to implement something. This is, by the way, the main problem of neural networks. They always have to generate code. This is what, for example, is joked about with seniors, right? A senior is not a person who writes code, but says when it's not needed. Neural networks lack this greatly. Here. Because they will write code in any case, if they encounter some problem, error, and so on, they will write even more code. Here. To prevent this, you need to stop neural networks in time. In general, even without programming, there is enough understanding of what it is doing nonsense. Here. And say something like, friend, hey. You forgot your original task. Analyze these documents again, understand that you were wrong, repent, and finally do what you were asked. >> Example. A programmer agent creates a domain with repositories, classes, services, and so on. The tester agent needs to create unit tests and acceptance tests for this domain. How here? Write for each class, for each function, well, not for each function, but conditionally for each file, what is happening there. Here. The tester agent will refer to this documentation and look. Here. Plus, a project manager helps. They also form a to-do list for themselves. Here, the to-do list is quite extensive. There is even, if you click with shift tab, there will be a planning mode, for example. If your sub-agents are specifically not coping, you call your coordinator, that is, I have a project manager there, and he says: "Assign the tester to cover 80% of the backend with tests." Here. And you say, here, in this file, there is information on how we work. Here. Make him and ask the neural network to write some number of tests, some introductory information. Here. And that's it. Here. And ask the tester to record where, for example, he has already implemented these tests. Here. I saw an implementation on Reddit when guys did it with Tasks MD, when, for example, agents communicate with each other, but this was before sub-agents, there were their own quirks. Here. But agents communicate with each other through this markdown file, here, where they say: "The developer writes to the tester: 'You, for example, make tests for this function.'" Here. And it does. Here's how it is. >> Uh-huh. Alexander, thank you very much for such an interesting lecture, for, uh, all the answers to the questions. We sincerely, we, like our subscribers, invite you to our studio again. We hope, we, uh, hope you will tell us more about Claude, CloudCode, and the functions in it. And I remind our viewers that with us was Alexander Agafontsev, an expert in process automation using artificial intelligence, and, by the way, the author of the Telegram channel EasyBusiness. The link to it is in the description under this video, as well as the link to our Telegram channel AFDF. Subscribe, stay with us. We will have a lot more interesting material. And also, if you have any remaining questions, you can ask them in the comments on YouTube, in Telegram. We will try to pass them on to Alexander for an answer. I hope Alexander will not refuse us. Well, that's all for today. Have a nice day and see you soon. >> Everyone. Thank you. Everyone. Alexander, how can we find you? YouTube, GitHub, and so on. >> Telegram. Let's show your contacts again. The last slide. >> Uh-huh. >> This is generally everywhere where you can find me, where I am public. >> Yes. And there is also a link in the description under the video, so you can go there too. And that's all for today. Thank you very much and see you soon. >> Thank you everyone. Goodbye everyone.