Transcription
I believe that few people understand what architecture and an agent are. You put this on your server, and that's it, you don't need anything else, well, maybe a database, and you don't need anything else. Well, maybe in some cases yes, but in some cases definitely no. As soon as possible, don't try to set up something ideal right away, but just go incrementally from something that already works somehow, because only then can you understand what the weak points are. To pack complex logic into docker-compose, I'll be mega-happy. That is, that is my goal. That's why I get upset when some supporter of some, oh, yet another element in the system, well, what can you do? And anyone who has been involved in DevOps in the clouds before agents, they will be able to do it with agents too, because, well, it's just an application. Hello, my name is Petr. If you've always been interested in neural networks and how they can be applied in business, how to use new features, which ones to use, which ones not to use, then come to my Telegram channel, I post my latest findings, lectures, life hacks, links, and so on. Hello. Hello. This is a commercial product, actually, which we are making about automation, yes, reporting, as part of this Mod Crew too, well, a framework, an open-source part. We are thinking about a strategy on how to harmoniously combine the development of one and the other. Developing the framework is important in a more long-term perspective and for the sake of the community, and for the sake of this thing simply existing, because it's cool and useful. So we will carefully think about which parts we can open-source. And I have a feeling that everything is changing wildly. That is, well, something new has appeared. I've been looking at MSP servers for the last 2 days, all this, it's not that it's something new, but it's some kind of new wrapper that changes something a little bit, yes, in the general perception. Many elements of the entire system have accumulated, yes, I roughly understand how to connect them. And that's what I wanted to talk about, how to combine some views or, conversely, adjust views. Well, I believe that few people understand what architecture and an agent are, or it's too simple. Well, for some, I understand, like you put N8N and everything will be fine, yes, if you approach it seriously, then it grows to an extent that I didn't even, well, I didn't really imagine. I want to talk about this, because I'm starting to drown in it a little bit in some way. I drew a Miro board here. I would like to show that from my point of view, which is not very deep in the code, but also not very far from business requests, so to speak, and how on average it is perceived now, and an agent, well, for people who have dug somewhere more or less, it is perceived as follows. there is some orchestrator that can have a relationship with other agents. Either functions, or some workers, I call them, these are some skills. That is, if we go by the classics, then an orchestrator, to whom we give a task, he says: "Ah, okay, I understood, what do I have, for example, I have a writer, I have an editor, I have some second editor or proofreader." We can call some agent that launches some single conditional function. Functions can be sequential, that this agent can create this function itself as it goes. That is, it created a Python script, executed it at the next stage, via API, yes? That is, they can be any. This architecture is quite simple, but if you dig deeper, it turns out to be not simple at all. Maybe you have some remarks here, maybe something is missing. In general, the concept is clear so far. The question is rather what exactly you are describing here, and an agent or for a specific case. Any agent is a highly individual thing, so you won't be able to draw such a template, and it will be individual, well, that is, rather universal. That won't work. These functions are just an example. I'm just giving an example of user stories, working with documents, working with statistics, updating knowledge bases. Just some random functions for some random agent. For example, there can be thousands of them. To strengthen the terminology, there is a function, and there is a specific element of the system. That is, a user story as a function, it can relate to both memory and knowledge base. Consequently, I am now outlining the elements. I just sketched a little bit before the meeting. That is, we execute some functions. Then we have a database. We store some statistics, something, in short, working with numbers, geodata, maybe something else. In SQL, in JSON, we want to process documents that come to us from somewhere. JSON databases can be any documents, including correspondence, and then, accordingly, we already have two databases. And before, I thought that we should do the search directly in the database. That is, we put PGVector there, went through a function to the database with some query, got some answer, and this is already, well, a good system. But then I realized that, after all, no, the correct approach, well, or the production approach or something like that. You need to search for answers on top of or next to these databases, that is, Elasticsearch, for example, then search for answers within our databases and not only databases. Well, actually, it's not necessary. Well, that is, it depends on what kind of search you want, yes? If you want, well, that is, you can, of course, set up some external solution, like search, yes, so that there is a cool full-text search with indexes and so on. But in many databases, for example, in PostgreSQL and in MongoDB too, I think, if it seems like there is also this thing, well, in short, there are built-in mechanisms for full-text search. If we take some information from documents and put it into a knowledge base, then the database and these three elements are not enough for us. That is, we need some pipelines to do the transformation and extract markdown from PDFs and so on, yes? And I perceive this within the framework system. Of course, this can be divided into two parts: incoming and outgoing. But the framework system is something where we have some pipelines that allow us to dynamically update this knowledge base. For example, correspondence from chats should be saved in some form later. We save them in JSON, but at the same time, we pre-process them. Or we processed PDFs, or something else. That is, these are dynamic incoming and dynamic outgoing. Short-term memory in a chat is when we, well, the user communicates, and then from this conversation memory, we also save some data long-term. Consequently, this is still not this, not this, not this, and not this. This is something separate. This is some kind of dynamic small system. Well, someone uses Redis, yes, probably. Therefore, here is another one in this list, yes. Now we also have MSP servers. which seem to be a function, it seems to be a Python function, which we previously called a function, yes, we made an API request, got something. And it seems to be the same thing, but it seems not. And most likely I missed something here. Well, that is, I, okay, I also missed the frontend, and this doesn't relate to this at all. This can be a Telegram bot, or it can be something on a website, it can be built into some system, also into this piggy bank. What else am I not considering? Where is the line between endlessly expanding these elements and still staying with some minimum set? And it seems that two or three more elements can be added here. But I wish they wouldn't. And how is this line, but in fact, many of them can be combined. For example, you had a separate SQL database and a JSON database, yes? The question is, why can't you, for example, store everything in one PostgreSQL, yes? You have something else, for example, Elasticsearch, which also, well, can be used and can not be used, yes, well, and then MCP servers. What is an MCP server? It's essentially an API. But the point is, as far as I understand, yes, that it's just a way to expose these tools to agents. Essentially, it's a collected functionality and documentation, that is, a set of functions and documentation for it. The LLM, every time before calling something, yes, at the beginning of work, a session or something, yes, or before each call, I don't even remember, it takes the data schema, yes, and thus this schema can be updated at any time. And if a traditional API would break such a thing, yes, then you need to do separate versioning, notify users, update all documentation, and make sure everyone has switched, then you don't need to do this. It, essentially, the LLM comes, asks what format you need, what you can do, yes? And MCP gives it to it. Here's what I can do, this and this. To do this, you send such a request in such a format. Here's a very simple idea, and it's logical. Does this mean that now all tools need to be urgently migrated to MCP? Of course not. No, old tools work perfectly, yes? That is, if there is some tool that just runs locally somewhere, connects somewhere, conditionally, I don't know, to some API, I don't know, Google search or Wikipedia, then why bother with a separate MCP server for it? It's completely unclear. In short, in some situations, it's a useful thing, but you need, naturally, to use it only when you understand that you need it. For me, there are two things that are really кардинально, well, let's say, joyful. The first is that before we wrote a function, that is, we read, there is some server, we read the API documentation, we wrote our function, and we are responsible for everything not breaking. And as a result, companies, well, I was waiting for a turn when every company would have a door you could knock on: "Hello, what do you have? What do you have there?" "We have this, this, ah, give me a list of your products." Well, that's it. And the responsibility for this door. And what's behind the door? Not on the developers. This is specifically the company or server, oh, that is, services of some platform, and so on. And this is very cool, because before, the developer had to monitor all this. And, as you correctly said, something changed and broke. And now we are moving to a different paradigm, that we can have a lot of doors, we can knock, and those who made the doors are responsible for the updates, for everything that happens there, and so on. This is cool, it really frees up your hands, actually, because we can connect quickly and we don't have on our side this thing of breaking something because something changed somewhere. This is awesome. And for companies, well, traditional businesses, this is finally the door I've been waiting for. That is, any business should have a sub-server where I can knock and get what I need, without any browsers, so to speak. And this has finally appeared as an idea. That is, maybe it will even change later, but as an idea, it's awesome. Well, that is, I've been waiting for a very long time and finally it appeared. Well, this is just a standard, essentially, yes, when a convenient standard appears, everyone rejoices. This, yes, this is great. In the system that is being written, I understand that there is a principle, let's not expand, let's simplify and combine something, but yes, we are encountering limitations. And in general, what did I forget there? First, I forgot the monitoring of the entire agent that you showed me, for example, the system. It's completely, that is, it's separate. I forgot this, didn't show it. What other elements did I not account for in general? Well, and in the sense, I didn't show on the diagram, which are probably still needed. Well, well, there was no infrastructural part at all, yes. This is also an important thing, like how to deploy all this, how these separate things, how much they are separate, how they communicate with each other, and so on. Do they live there as different microservices, or is it some kind of single application, yes, you can come up with a lot more, throw it in, that depending on the use case, but this is again depending on the use case, yes, so in general, well, this is the skeleton of some kind of regulation, yes, you have quite reasonably outlined it, I think. If you are going to build a real system, then of course you just need to start with something simple, yes, using familiar technologies. You can figure out anything when the need becomes clear, when you are building a system, yes, you start to understand what pieces are missing. And that's why I'm looking, yes, I really wanted to go through the database for everything. And then I still ran into, well, some limitations and thought: "No, after all, search is a separate big feature that constantly needs to be improved." Well, that is, like this. And, consequently, I have separated the search as a separate item, because for me, the framework system is generally the fundamental, like my philosophy is that if a person without memory, if a person forgot everything quickly, everything he understood, then in general, well, development would be zero. Therefore, memory, and the framework is, essentially, memory, well, a fundamental thing. We had a classic search, we have another way of searching, by vectors. This is just an element of the system. That is, you cannot take a new element of the system and abandon everything previous that we had. That is, you need to take standard search systems that we had, and just integrate vectors into them to improve them. But it turns out that when the vector appeared, we immediately forgot about everything else. Like, there's a separate topic with graph retrieval, yes? Well, that's what we're trying to do now, yes, we're trying to fit data into a graph, find it using, say, hybrid search or vector search, then look, then go without the graph, yes, then go into the graph, see what, well, take, take these nodes in the graph, yes, corresponding ones. This is also a separate piece that in cases where it is needed, not everywhere graph retrieval is needed, yes, but where it is needed, for example, in our case, we simply cannot do without it, well, obviously. By the way, with graphs, I have such a point, and if you take a graph, it's even, well, that is, here's our vector, here's our graph, and this is definitely a separate search, that is, this is not from the database. We, that is, if we have search engines with graphs, we can work with them. Well, depending on the graph, but generally it's not designed for this, yes. You need a separate database, yes, when we want to improve something. That is, if we take these three elements, search, knowledge base or database, the interface, yes, if we want to improve something from this, well, in a specific case for a specific use case, as soon as we delve into one element, then it breaks down or is supplemented by something. Well, that is, and therefore the system develops, well, in the sense, expands, because we want to improve some element with additional functions. My position is rather that you need to start with the minimum, yes, you need to do something working, then understand what it lacks, yes, and accordingly, proceed incrementally. That is, as soon as possible, don't try to set up something ideal right away, but just go incrementally from something that already works somehow, because only then can you understand what the weak points are. As you said, I don't have on the diagram how we deploy all this, by and large, well, that is, and how we assemble and so on. What can be said here? Well, the easiest way and a bit more complicated. Well, of course, hardcore like Kubernetes is not needed. Well, that is, something simple, and that you use yourself. A long-running virtual machine, on which there is simply, conditionally, an application written in Python, for example, somewhere in Docker, a database, in which all these elements are simultaneously in one place. It's simple in terms of configuration and deployment, yes, but this has disadvantages, for example, in terms of scaling, well, yes, you can simply set up one instance of this somewhere in some cloud and use it. Well, especially if it's for internal use, then in principle it can work. In short, there's the topic with serverless, yes, these serverless approaches in modern clouds, yes, this is often used when, conditionally, you don't have a virtual machine that you constantly give a Docker container to a service and say: "Here's my application." Well, and then it fully handles deployment and scaling for you, yes? This is a cool thing, yes? That is, we use, for example, for serving the frontend, for the API too, there is this Google Cloud Run service. It's very convenient for this kind of thing. In short, you don't deal with any server management, yes? You just give it a Docker container, and it scales it itself as needed. But, so, what might be the catch in the case of agents, yes? In short, all the state must be stored in the database, yes, with the conversation history, yes, if it's all in memory, then it will simply disappear. Uh-huh. And therefore, in the case of some long-running server, just with a virtual machine, yes, it's normal. It can be normal. But if it's such a server platform, then it won't work anymore. You need to store this message history somewhere in the database, yes. Plus, these agents need to be able to start quickly and so on. We use such a hybrid approach, when we have, well, actually, our, well, it's actually like this, serverless, yes, and it's maximally thin, that is, all agents live precisely in Kubernetes, this backend, accordingly, communicates with the cluster where the agents are running. The scheme that we use is like this, you have a conditionally long-running server. Well, let it be a VPS, yes, in fact, it's not necessary to build Kubernetes. Let it be a VPS where only agents live, yes? It only deals with agents there, yes? Or it's a cluster of VPSes, yes, which can scale, for example, yes, because agents also generate some computational load, and so on. And you have a separate part, you have your API, well, your frontend, yes, which, essentially, communicates with the frontend and there, well, in short, with users. Well. And we have such a frontend, it's in a serverless platform, yes, it's stateless, it just serves requests, yes, and then these requests, in turn, this frontend communicates with this machine where the agents are running. The only thing you need to think about is scaling this thing where the agents are running. Well. And maybe there won't even be such a problem, yes, if, for example, you have a limitation, you have only a few users, each user has at most one agent working simultaneously, yes, then you can get by with just one machine, and you're already getting a bit into the weeds of all this DevOps and how to deploy applications. Actually, yes, I actually wanted to make one of the goals of the conversation to say, guys, not everything is as simple as it seems, yes? That is, you install N8N, everything is fine, that is, well, not always, but let's say, the vast majority of videos about agents, which, at least, show me videos or something like that, is to install 8N on your server and that's it, you don't need anything else, well, maybe a database, and you don't need anything else. Well, maybe in some cases yes, but in some cases definitely no. There are platforms that promise to greatly simplify agent deployment, yes, that is, conditionally, here's the code of your agent, upload it to us, and we will do it ourselves. But still, that is, all these elements that I drew on Miro are still separate, conditionally speaking, Docker containers. And at least you need to connect them. And at least it's not entirely separate. Well, that is, if there was an agent that just pushed Python code inside one Docker and forgot. But actually no. We have microservices, well, at least five elements, each in its own Docker. We essentially connect the agent as its brain, this is Python code, conditionally, or logic, but then it has hands and functions, hands, these functions. And these functions can be, firstly, many, secondly, they are different in terms of load, processing, and so on. And plus, we also need to save the state all the time. Uh-huh. Well, here, yes, it starts to be just the harsh reality of building a fault-tolerant system with all the nuances. And this is simply necessary, yes, if you want to set up such a thing, then of course it will never be done with one button, yes, you need to design the system, like, it doesn't mean that any application that uses agents should look like this, yes? That is, this is our reality. If you have some tool for internal use, then it can all live in one Docker Compose, and everyone will be happy. Well, I, honestly, am striving for that. Honestly, I'm striving for that, yes, I have to pack complex logic into docker-compose. I'll be mega-happy. That is, that is my goal. That's why I get upset when some third-party, some: "Oh, yet another element in the system, well, what can you do?" Can it happen or does it seem like it will happen that some agent is essentially a booking application, and another agent is like a calculator. And accordingly, different architectures, different loads, and so on. Well, do you agree with the thought that agents don't differ much, well, that is, the architecture itself, deployment, all that doesn't differ much from our usual applications. It leads to what we discussed last time, yes, beautifully, that exactly, yes, like again, like, where is the specificity here? Like, an agent is some stateful thing, yes, that runs, like, yes, we don't have anything fundamentally new here. This is all like the same, the same infrastructural tasks that are being solved. This is like a new, new element in terms of logic, yes, of operation, yes, and it has its own peculiarities in terms of instability, this non-determinism, so to speak, yes, an important element that needs special attention in this case, yes. Well, fundamentally, again, it doesn't bring anything new to what we have, yes, like it's, well, it's a complex system, yes, that we need to understand, like how to architecturally lay it all out, yes, and deploy it to the necessary infrastructure, yes, so all the old rules apply here too. And anyone who has been involved in DevOps, in the clouds, before agents, they will be able to do it with agents too, because, well, it's just an application. This slightly removes this veil of hype. Well, I would say so, these are the thoughts, because it seems like it's wow, something really new, new, but if you dig deeper and then figure it out, in fact, out of all that we had, well, maybe 20% of something new, which, of course, frees up our hands. Of course, we have something completely new, and something we have fundamentally, but it's a certain level-up of applications that we couldn't do before, yes? That is, yes, but 80% of the rest is classic. You need to start with something, make the program do something, and then just incrementally improve it, first let it just work on your laptop, for example, then you can assemble it into Docker and send it to someone else so that it works on their laptop too. Then deploy it to the cloud, then, for example, move the database somewhere separately, and so on. Well, that is, and just move towards some more complex distributed system that will feed everyone and everyone will be happy. [music] [music] Yeah.