AI engineering 2026 - LLM, RAG, AI агенты, разработка и инфраструктура

Gary, Gary, Gary. So, guys, hello everyone. We are starting the broadcast. I hope I am well seen and heard. A couple of words from me. What will be the broadcast today? Before New Year, we held a conference with the results of the 25th year. The link to it is in the description of today's broadcast. And many cool speakers performed there, including Refat and Valera, who will be today. But the guys clearly didn't have enough time. We heard feedback and wishes from the viewers, so we allocated a full broadcast from Refat with Valera. And we will add both practical parts to the guys' reports and time for questions, because many questions were not answered last time. Here. And a couple of words about the speakers from me. I, in fact, get incredibly high every time I listen to Refat and Valera. Because, what is interesting to me, is this mix of engineering and business parts, which both Refat and Valera have, yes, as if each of them perfectly understands how business works. They know how to speak the language of business, and at the same time they are both wonderful engineers who dig very deeply into details. Here. Well, and you, most likely, will see it yourself today during the broadcast. With that, I hand over the word to Refat. Refat, start. Ah, guys, you can write questions in the chats. Ah, hello everyone, Kolya. I will need access to sharing. Yes, ready. Moment. So, it should be visible. Yes, everything is visible. Yes, everything. Okay. So, for those who were at the conference results, as Kolya said, some slides and some content will be familiar. This is a more expanded version of that same report. And, actually, there will be a little more about cases, there will be time for questions. So let's start. Today we will talk about how the world has changed from the perspective of a developer and systems. This is more of a technical report, but it will also be useful for founders and everyone who is building or interested in building AI systems. A little about me. I am the co-founder of Death Stark and Spread Simple. We build AI-first products. I have 12 years, even more if you count commercial development experience, in development for 3 years. I am involved in GENI systems, my team and I deliver production systems for enterprise and startups. And probably the most interesting thing is that a lot of projects pass through me, there is a lot of experience, and many of them I have had the opportunity to see from the inside, including large projects, besides those that we develop ourselves. I write about AI engineering. There are links here. So, agents are a pretty obvious trend. Actually, quite a lot of people have talked about this. I will only say that this trend will intensify. I will not dwell much here on the fact that this trend is real, how it affects business and development. I will only say that, according to the large analytical agency Gartner, this trend will only intensify. And I believe this forecast. And how has agent infrastructure changed over the year? On this slide, I wanted to show how the infrastructure of agents for developers has been strengthened. There have been quite a few events. I have included only the big three: OpenAI, Google, and Anthropic. And even these are not all releases, but key ones. And note that what we are used to appeared quite recently. The response API, it seems, has been with us forever, but it only appeared in March 2025, that is, less than a year ago. MCP was a little over a year ago. Codex SDK is practically recent, yes, after, significantly later, Anthropic, the agent SDK, which we will talk about today. And all this is to the point that the Completion API has become much more agent-like, and the API is becoming more and more agent-like. Here is a screenshot from OpenAI documentation, how they position the completion, or rather, the response API compared to the completion API. Many of you know the completion API, that is, essentially, it is what we are used to. That is, you send an array of messages, you get a result, yes, that is, completion. And unexpectedly, I think for OpenAI, it became a standard for the entire industry. Almost all providers picked it up very early. And even those who do not support it directly, for example, Google, offer a compatible API. Not to mention that practically all open-source solutions and Chinese LLMs support the completion API. In this regard, it is a great standard, it provides interoperability, models can be interchanged, and everything was actually great, yes, and continues to be. Well, as we will see further, there are a lot of new features. For example, grounding, yes? That is, when we see in feature releases that the model has gained the ability to search for something, a content interpreter has appeared, context caching, editing, compute usage, deep search, and so on. And how much of this is about the weight of the model itself, or how much relates to the model itself, yes, practically nothing. These are all infrastructure features. Next, I analyzed the feature table, a rather large table on several slides, you can take it later. What is interesting about this table? The structure is as follows: there is the feature name, links to the documentation of the providers of this big three, and standard alternatives. Of course, very conditionally. Some alternatives are broader, some are narrower, some are not drop-in replacements at all, yes, but nevertheless. And the first category, retrieval from the web, is essentially RAG with web grounding. That is, with a modern API, which is agent-like, in my opinion, you get the ability to do RAG without any external tools. And this is also helped by searching external sources, that is, Web search, web fetch, that is, models, models in quotes, can fetch data, that is, read it, see it, yes, and even launch entire deep searches. Well, this is not news, but perhaps many did not know that deep search is available as an API directly within Google, Google Gemini API. That is, this is not just a product feature, it is a whole API. Yes. And a rather unique feature for Google specifically is mapping. That is, it is possible to directly analyze various geo-data, for example, to solve the traveling salesman problem and see where the nearest places are, and so on, and so on. All this is available, again, through the API. Well, and here are some alternatives, all these links are clickable, you can see how to do it yourself. Next, documents. Here is a category of features related to documents. For example, the file API allows you to upload files to the servers of these providers and do something with them, for example, retrieve from them or reuse a file multiple times, for example, analyze PDFs, and so on. That is, to create your own small project, which you might have encountered in ChatGPT or in Code. And quite advanced document understanding. Again, these are not model features, pay attention. That is, it's as if the providers are trying to blur the line between the model and the infrastructure. Although I, as an engineer, I think, understand that for the most part, this is a kind of, well, very, roughly speaking, an add-on around the models, not the features of these models themselves. Memory state and state, a lot has appeared around state management. That is, for example, OpenAI has state directly in the response. That is, you are not obliged to store the state yourself, that is, you can store messages directly on their infrastructure. You can do advanced context management, for example, relatively recent context editing. By the way, my company started with this, with this release of Context Editing by Anthropic, which essentially, we used to do it ourselves, that is, we did a tricky surgical runtime update of the context, and Open AI, Anthropic in this case, released this feature, which does it much better. And this is a built-in feature in the API. And, of course, all this leads me to the point that often you can save a lot of time if you know the API of a particular provider well, and you can avoid certain risks if, again, you know certain practices, which I will mention later. There is, for example, a memory tool, that is, these are already agent-like features, more and more agent-like features are built directly into the APIs of these providers. I'm not even mentioning prompt caching, which, in fact, is, of course, an infrastructure feature. A huge number of releases related to tool infrastructure, to tools, occurred in the last quarter, in the second half of last year. Especially related to Anthropic, that is, search, programmatic tools, streaming to a huge number of specialized tools, for example, even with OpenAI shell to iOS, which appeared earlier, and special tools for working with code. All this saves a lot of time when used correctly. Programmatic Tools are something in between code execution and ordinary. The infrastructure executes several tools on the provider's servers, aggregates the results, and returns the result itself to the model's context. That is, a huge amount of this "sheet," massive use of "of" does not get into the context window, which is super effective, but it has its, of course, its price, at least the price of vendor lock-in. Code execution is a super important part, which, obviously, does not relate to the model, but relates to agent infrastructure, but is often presented as a model feature, even in official press releases. This is code execution. And you might have heard that, for example, Opus can work for 5 hours in the background. Well, come on, this is not Opus as a model, this is actually a numerous infrastructural add-on, which makes, keeps this agent cycle, and constantly performs these completions, yes, to give the desired result in the desired direction. And as we see, there are alternatives. That is, one of the reasons why I added this column and why I think this table can be useful in many places. Because often you can make a choice for yourself, whether to use a built-in feature. Of course, you will save a lot of time and often get the best result you can get with a limited team. Or still use an alternative. I will talk about this at the very end. How to develop such, one might say, intuition, what to use when. Valera will talk a lot about when there are some non-functional requirements, yes, when, for example, due to legislation or company security policy, you cannot afford it, or simply the founder's paranoia. But this is a separate story. This is not always, let's say, a technical solution. This is a policy, yes, that is issued. I am talking now about when you have freedom of choice. So, code execution has also become a feature of the API relatively recently, that is, the APIs of the big three. All of them allow you to execute code from models directly on their infrastructure and use it as results of your workflow, yes, as part of your workflow. Here it is called interpreter, at Anthropic execution, well, and so on. Alternatives and AI, of course, have not gone anywhere. Well, compute usage can also be done independently. You can use built-in features. According to my tests, built-in features almost always significantly outperform all attempts to do it yourself, except for rare exceptions when it's some kind of domain-specific thing, and there you can indeed get better metrics, but perhaps this is a temporary phenomenon. Yes. And, of course, Skills is a relatively recent release, which is being actively adapted, yes, a separate standard appeared at the end of the year, Agent Skills, and other frameworks and providers are actively adapting it. Why did I mention it here? Because, in fact, Skills can be used as an API. That is, the skill that you have packaged in your code, in code-code, you can package it in an archive and use it through the Anthropic API for your scenarios when all you have is a request that you can make to their server. Yes. And, finally, output, yes, output and operations. Well, there is the possibility, so, batch processing, there is the possibility to configure safety settings, yes, it's a whole thing to do safety wire, good guardrails, and fine-tune them. Google offers this as a built-in feature and says it's a model feature. Even Google offers observability, that is, some kind of replacement for, say, Fuse, and so on. And you can see each request in their Google Cloud in detail. And much, much more. So the table would have grown even larger. I included the most important things. I think it's clear here, models are catching up, the gap in infrastructure is growing. Often we see benchmarks where, indeed, JLM or DeepSeek or we haven't heard from DeepSeek for a long time, and Kwen, achieve almost parity with the big three. But we don't see this in practice. So why? Precisely because infrastructure is super important, and that's why you choose a stack, not a model. And often this synergy, which you can get by using a set of infrastructure tools that comes with the model, you get some unattainable performance. At least, this is the status quo today, as I see it. And here is an analogy that I chose, where all this is moving, as it seems to me, yes, again, email used to be, well, email is a very old standard, as you know, even older than the WWW. And its own SMTP, its own server used to be the norm. Now imagine that we are deploying our own SMTP for corporate email, despite the fact that there is open-source software for this, and you don't need many GPUs, in principle, server resources. It's super difficult because Gmail is a whole ecosystem. Besides the protocol itself, there are a huge number of infrastructural and product features that are hard to imagine without it. So, chat was a standard. What now? It's funny that when this report was first presented in a shortened version, there was nothing like it, and I even predicted, or rather, asked for some alternative to chat completions. And what we have, in fact, OpenAI released Open Responses literally a few weeks ago, which is great, I think, and shows how justified this trend feeling was, probably. And this is a very, well, a wonderful step, in my opinion, but it's still questionable whether it will work, whether it will take off, because there is a lot of skepticism in the technical, professional community, that Google will adapt this standard as they did with the Completion API. Nevertheless, many open-source and not only libraries and providers have already declared support, for example, VSCEL, VLM, Ollama, and so on. I looked into this standard. In fact, it doesn't cover everything that PNI offers yet, but perhaps this is only for now. So, the second part of the report will be more about a case, and at the end I will talk about the decision-making framework. So, yes, first of all, it must be said that RAG has not gone anywhere, it has simply changed. And, if before, very often through Telegram channels, blogs on Substack, on Reddit, and everywhere, there were many questions like, which embedding model to choose, which embedding model, well, a lot about retrieval, about these pipelines, about graph RAG or ordinary RAG, hybrid RAG. And, actually, now it's actually 25th, 26th, yes, already, what source to use. And we understand that retrieval is just one of the agent's tools. Whether it's in the form of embeddings, in the form of file search, or an SQL query, is secondary, or rather, depends on the context. And the agent decides whether it needs retrieval or not. Before moving on to the case itself, I wanted to say a couple of things about where the roots of such solutions grow from, the prerequisites, yes? So, coding is the leader in AI adoption according to many, and including research, for example, Anthropic proves this. Their Economic Index shows that 44% of API calls are for development, much more than all other categories. Each, each code-code showed phenomenal success and continues to be heard even more this year, and not without reason. In general, coding practices are the leading area of AI, in my opinion. And here is the most mature tooling and best practices. And sometimes bash is all an agent needs. Yes. Why? Because it's trained on code. Bash is a rather, well, old, let's say, way to tell a computer what you want from it. So, you get quite accurate results when you need them, yes, so grab find is super fast and super accurate, when you don't need it, you can use more fuzzy algorithms, including similarity search. This doesn't prohibit such an approach in itself. Minimal context, because files are loaded on demand, so you don't need to think about how to re-rank and so on. And here it's not even about files replacing databases, but about rethinking when a database is excessive, and when files are all you need, yes. Well, and regarding debugging, again, these are ordinary C tools and so they are available both for humans and for agents. The philosophy of agent SDK is that essentially agent SDK, we are talking about Anthropic agent SDK, but a similar practice is already offered by many other providers. The philosophy is that the agent needs to be given a computer and use the same tools that programmers use in their work. File system, bash, file editing, API calls, MCP, and so on. That is, essentially, these are wrappers, but this is also a tool, yes, and the cycle, the typical cycle, is to gather context, perform some action or a set of actions, sometimes check the work in parallel. Here, a super important part is reflection, yes, on what has been done, and repeat as many times as necessary until the result is achieved. I will say right away that I am talking a lot about code, about code agent SDK and so on, but this does not mean that I am saying, use only this stack and that's it. There are excellent alternatives from the other big three, including, for example, OpenAI Codex SDK, Gemini CLI, you can make them work in a similar way, although, as far as I remember, they don't have an SDK around it. There is Open Hands, it has been developing for quite a long time, there is Open Code, a very good open alternative, and many don't know, it also has its own SDK, which, in general, is not bad. Only it works a little differently, yes, they deploy, start a server, you access it via REST API. But it doesn't matter, sometimes it's even more convenient. So, I'm talking about the approach in general. And the architecture of a File First agent, as I call it, is approximately this, yes. We have data. We bring this data as close as possible to the agent, to its environment, to its place where it actually executes code, because this achieves minimal latency and maximum ability to use bash tools. There is a runtime, it can be a local machine, it can be Docker, it can be a container, sandbox, and so on, and so on. The main thing is that there is a runtime. Yes. And there is the possibility to use these coding agents for non-coding tasks, well, maybe for coding tasks too, depending on your case. Tasks on top of this, there is an interface. The interface can be MCP, it can be chat, it can be API. Here again, it depends on various requirements and on how, well, again, how it will be convenient for the user, or how it will be faster to implement. Closer to the case. This is a European company, it is involved in marketing. They came with a request, saying that they tried RAG and it didn't work. And, moreover, they put a lot of effort into it. They used a lot of techniques. I looked at what they tried to do. And, of course, this cannot be called a naive superficial approach. They used hybrid RAG. They probably only didn't get to graph databases and ontologies. They used almost everything they could find at the time. So, Query expansion, I won't even mention it, a lot of techniques, they got hallucinations. The main case, they want to search, that is, they need new management and good understanding of the knowledge stored in Google Drive in huge quantities. And everything they need to know about projects is there. These are scenarios, various information about client positioning, information, call transcripts, everything, everything, everything, practically everything related to the project itself. I'm not talking about documents like contracts they signed, although that's also possible, but it's not about that, but about what is useful knowledge for projects. They have stored it in Google Drive historically for a very long time. Yes. And they need RAG for it. In order to, again, do their work, create new creatives, prepare for calls, and set tasks, and so on. In general, the idea came during a call to replace it with agent SDK, because they had already started adapting code-code, and I was actively involved in it then. And we did a pilot, and it went very well for them. It worked and works perfectly. And periodically, small adjustments to prompts are needed, adding skills and tools. I will talk about this a little later. But in general, the speed of implementation and the quality of the result were much higher than expected. Key components in this specific case, that is, file synchronization, occurred through Aircon. And Aircon is essentially an RRS for cloud storage. In this case, Google Drive, but you can use Yandex Disk, Dropbox, and many, many others, there are many providers. Search was used, for example, with the help of the bash command `tree`, you could quickly get key files and directories, especially directories. With the command, you can search, including advanced search, both by content and by binary files, and using certain C tools, you can perform, including fuzzy search. And, by the way, here you can also fine-tune semantic search. It's just that in this case it's not needed. That is, it works quite well, just as code-code works without embeddings in its code editor. And certain tooling was built around reading, because it's one thing to read text files, yes, markdown, and, for example, txt files, transcripts, and quite another thing to read PDFs, images, and so on. PDFs and images are built into code-code itself, it reads them perfectly, with certain limitations, for example, PDFs up to 30 MB only. But other formats are read excellently by tools like pandoc, or custom scripts. Some of these scripts were specifically adapted for their specific reports. That is, we know the structure of one Excel file or another, and we simply speed up the search and reading of such a file. That is, the tool is adapted to their specific knowledge base. Well, and on top of all this, agent SDK, virtualization through, that is, sandboxing through Docker, and MCP, I will also talk about it. So, in this case, what advantage did it give? No re-indexing. This was one of the biggest pains for this client. And, in fact, we implemented similar things for other clients as well. Every time your approach to RAG changes, you have to change, you found out that the chunk should be of a different size. You re-index, you found out that another embedding model works better. Again, re-indexing. You just added the idea of summarization, re-indexing is normal, only this is data engineering, work that not everyone can afford, let's say, and well, it's just resource-intensive. The answers are quite accurate, because if the agent doubts its cycle, it will recheck. And if we think it doesn't doubt enough, we will force it through prompting, through skills, and so on, to check its work better. The agent sees the entire context of the document, very often it loads it entirely during the process. And the domain expert iterates himself, adding skills, mcpools, engineering, and so on. It's funny, yes, this was probably one of the interesting experiences when we worked directly with the domain expert in interactive sessions, that is, we launched, watched what was not working, immediately changed the settings of the coding agent, for example, the system prompt of code-code or a command or a skill, and so on, and immediately saw the results. That is, during such a one-hour, two-hour session, the performance could be significantly increased. Because the system itself is higher-level, that is, you focus on instructions and domain knowledge, rather than on some engineering gears that need to be reassembled every time in more rigid workflows. Agents, they need an environment, yes, as we know, they need a runtime to execute code. The execution environment is

Sufficient, well, it's like a new structure for its infrastructure, relatively new, but you need to constantly think about it, because if AI writes code, then how it executes it and where it executes it, this is a very important question. First and foremost, a security issue. And specifically in this case, the architecture, this is, by the way, the file structure, it was difficult to extract something super specific. I tried to extract as much as possible so as not to violate the NDA. And this is the file structure, where you can see that we have, you know, actually, the worker that is mounted into the container, and there is certain code that launches this, including MCP and including task manager, yes. The logical diagram looks like this. We have the external world, we have Airsyn synchronization, outside of Docker we get a folder that is always up-to-date. Synchronization works regularly via cron there, yes, and we only mount what is needed into Docker, with the correct permissions. For example, in this use case, we were not allowed to change, that is, no file changes were required, so we immediately gave it access to this folder, because if, well, it's just a policy of minimal permissions. And inside the task manager, which launches, actually, agent SDK instances. And it is allowed to write scripts for itself, to optimize. Very often, in the process, you can see some request that requires data aggregation. For example, gather information about all marketing campaigns over the last year and do some analytics, right? So, RAG can hardly do that. Code is directed on the fly. It simply writes a script, launches pandas, rechecks its work, and returns some metric that it was asked for. All this happens in the background. It can even reuse this script later. I'm not even mentioning that through skills and through the same scripting, but already deterministic, when we have pre-written the necessary scripts, more optimal execution speed and greater reliability can be achieved. So, yes, why did I show this here? Because isolation is a super important concept. Do not just run agents on your local machine that execute code like you do, and always think about security. And here, in particular, there is even Network Isolation, meaning it is not allowed, it is allowed to visit certain domains, for example, to download certain dependencies for itself and interact with, in short, there is a whitelist of domains, everything else is prohibited. When does File First approach work? When does it not work? Actually, this is a very conditional table, but I think it will be useful to talk about it. When you have hundreds, thousands of files, actually, in that project, well, I find it difficult to say now, because the number of files is constantly growing, well, tens of thousands of files. And it handles it perfectly. Including because most issues relate to a specific client, and it has a specific folder, and the agent will simply not search in the wrong folders, it will look where it needs to. Files are updated frequently. This is suitable, this is a green fact. This means that you don't need to re-index everything constantly, but you need simplicity and speed. There are no resources for this data engineering pipeline, because with good skill and good resources, RAG can yield some colossal results. But often, in my experience, teams run into data engineering problems. That is, something works perfectly for them with hundreds of files, but then scales poorly with thousands. That is, they don't understand how to build durable background processes, these workflows, essentially, how to do ETL correctly. This is the domain of data engineering, and it goes beyond the scope, well, that is, it intersects with engineering, but many lack the qualifications for this. Complex tasks, advanced reasoning, it works perfectly, for example, with pandas and so on. And there is no fixed workflow, flexibility is needed for our case. When it doesn't work, when, well, there's just a huge number of files on a scale, I doubt that some kind of, say, a support agent, when they were creating their, you know, some kind of support agent, would create a similar system that is in isolation, in a sandbox, for each client, to answer, to do all these extra actions. That is, this is all more and more within corporate scenarios. Semantic search is needed, also not a blocker. This can be bypassed, in principle, because through the CLI, and not through the CLI, you can do MCP, you can, you have a huge number of ways, including using embeddings in such an agentic approach. When there is critical latency, yes, because it is clear that this optimized, directed graph will, of course, be maximally optimized for speed. Critical token cost is also not very suitable. A lot of users, as I said, it's hard to scale. Although a lot of users, the concept is expandable. For some, even 10,000 users is not a problem if there is proper orchestration. Well, and there is a clear workflow, that is, we have cases when we simply have a directed graph, we know exactly what should follow what, if there are any conditionals, they are quite transparent, understandable, not needed. We don't strive to complicate it through the approach. Simply because, well, it's more reliable. So. Yes, there's still a little time, I'll tell you about MCP. This is a bit off-topic from agents, but I think it's interesting because it's related to this case. MCP, when they appeared, yes, and still local MCP is something completely geeky for most business users. When you do it yourself, like, nothing complicated. JSON config and that's it. When you try to explain it to a business user, how to add MCP to their environment, for example, desktop code, it's rocket science for them. That is, well, it's understandable. Open JSON config, insert config strings there, configure environment variables, restart the application, install some dependencies, etc. That is, this is not how USB for models should work. This is not the UX we are looking for. We made MCP for this case. And it works, it worked, and it works perfectly. MCP, many of you know, and Valera also wrote about it. In essence, it's the correct MCP, as it should be. It's an HTTP-based MCP with OAuth 2 authorization. Firstly, it's secure, and secondly, it's super convenient. We did this through OAuth Kit and Fast MCP. Funnily enough, to connect this, we did it at an early stage, and we rewrote the project from TypeScript to Python simply because Python has Fast MCP and it has the necessary connectors. And in general, Fast MCP is so good that we even changed the programming language for this project from TypeScript to Python. It wasn't very painful, and it was at an early stage. Yes, no configs. I recorded a short video to show you how it works. So, now it's the beginning. That is, we simply add, we click, go to connection. This is on the example of code, but others work similarly. An agent is added, well, in this case, an MCP, and authorization occurs through OAuth, and that's it. That is, this agent is available in our list of connections, and now we can make requests with this MCP just like with a regular tool. What's here? What advice can I give? OAuth, if it's not already implemented and in principle not there. In this case, there was a case where this project didn't have OAuth. It's very convenient to do it through Workflows and OAuth Kit. There is Keycloak and so on, there are standard solutions. The key here is, of course, the ability, the ability to, well, just a moment, okay, I won't go off-topic, I won't remember the term now. Remembered. DCR dynamic client registration, roughly speaking, OAuth Kit solves it in four lines of config, and it turned out to be so fast and convenient, and we get top-tier security. These are not some tokens that don't update. It's very reliable and very convenient. So I don't see any reason not to use it in similar scenarios. This same MCP can be connected, and that team connects it to code completion, and to chat GPT, and so on. So, there's a little time left, I'll tell you about the decision framework. So, so, so, build. We'll skip this. When to build, when to buy. Let's go back to the beginning a bit. And we understand, yes, that we have a huge number of features. There will be more. That's the catch, that we see a clear trend that providers will increasingly add features that are absolutely unique to their platform and, moreover, work maximally cool precisely in their ecosystem. That is, code plus code will always work better than code plus GLM, for example, or in the near future. It will definitely be like that. When to build, when it's your core IP, it's your competitive advantage. And your unique data, you should build, use this right column of the table and in the table and figure out how to do it yourself, how it could be replaced with your own controlled solution, and if necessary, with another solution. Buying is easier when it's an infrastructure task. For this company, for a marketing agency, it's not their main business. That is, knowledge management search, even advanced, even AI knowledge management and some AI scenarios are their secondary business. They are not going to resell it as SaaS, it's their corporate automation. And plus, many things are better to buy, because you won't build such a search at Google yourself, no matter how much you want. Or sandboxing, it's probably cheaper to buy now than although this is also a question, actually, it depends on non-functional requirements. Okay, I've covered this. So, I'm already on time, so I'll wrap up and hand over the word to Kolya. Yes, Arif, thank you very much. As always, it's interesting to listen to you. Let's now throw in some questions. Some that I saw, some that came to my mind, from those that were already asked. The first question is, how much does a solution on Cloud Engine SDK cost in terms of tokens? Because Anthropic is quite an expensive pleasure. I can say that for this client, it was in the range of 10 cents to 70 cents per question. Even though some questions that were like 70 cents, the agent dug through them for a very long time. It wrote a script, it ran it, it evaluated it, checked it. That is, it could work for 10 minutes, and that's normal for such a task. And specifically for this client, characteristically, their hundred-dollar subscription worked. It was enough for all their tasks and is enough. That is, they don't spend these tokens via API. Damn. I didn't have access to their RAG, let's say, expenses, but he complained that it was also not cheap. Tokens were not free there either. It's not cool that it falls into the B2C subscription for business. That's cool, of course. Okay. A question about the File First approach. How problematic is it when there are multiple languages? I mean, the language of the documents, I assume? Yes, I think that the language of the documents and when the language of the document differs from the language the user works in, is not a problem at all. Well, I assume there are certain nuances, of course, in this specific case, it was Dutch, which is very similar to English in its kinship, if they are from the same subgroup, it seems, and they constantly switch between English and Dutch. And there is no such problem. I think the problem with languages can only be if it's an unpopular language. Russian definitely falls into popular languages, because as far as I remember from statistics, the top looks like this: English is first, then German and Russian share second and third place, it seems so. So Russian is definitely in the top three most common languages on the internet in terms of content. Listen, can you explain to me on your fingers how this works in the case of Russian, if, for example, our documents are in English, and the user asks a question in Russian? And moreover, there are also documents in Russian? How does the agent understand what to grep there? Should it grep Russian words or English words? The beauty or convenience of this approach is that you have levers of influence on it. That is, you can prompt it in such a way that, for example, it considers the folder prefix. Or you can tell it in advance, that is, consider how to query, or you can even ask it to query directly in both languages. That is, we tuned this approach to querying, how it made requests, precisely through prompting and immediately checked. That is, you see the result, you have a very fast feedback loop. You give it instructions, see how it works, draw conclusions, and either skip benchmarks. And often, the main thing is to make it understand in which language, that is, your preferences and what languages the knowledge base has. Agreed. I'm actually also using similar approaches on a couple of client projects now. Somewhere custom agents, somewhere like Cloud. Open code. And it's surprising how easily changes are made in the system. That is, you can completely change the system flow with just a couple of sentences. Yes, it won't be rigidly nailed down, but it will still work stably because the agent always has this recovery system if something goes wrong. And here's another question. Someone wrote that bash is trash. The chances of the agent getting stuck are 99%. Can you explain how this works, that the agent somehow doesn't get stuck, and if it does, it returns? Well, here, of course, there are some life hacks, maybe here's how to make the agent's work more stable? I can say that I haven't encountered this. And here it would be better to listen to the arguments of the one for whom bash is trash. And I don't understand how it can be trash. If it works perfectly in coding, and even in such scenarios, if it gets stuck, again, code-code has a timeout instruction. If it hangs, it will try another approach. And all this is finely tuned through cookies, if this is a regular problem for you. We had a problem with MCP timeout. That's real trash. That's a separate issue, how we made MCP think longer than the MCP client itself makes it think. For example, code has 30 seconds, and this is an unchangeable value. So, we had to resort to trickery to make it work longer. But with bash, there are no problems in practice. At least, we haven't encountered them. Okay. I also had a question about isolation. Actually, I see a lot of misunderstanding about how to do sandboxing in general. So, maybe you can share some life hacks here too. I saw that you mentioned Docker, but maybe there's a comparison, say, of Docker with other approaches. And why Docker was chosen, maybe some heuristics about how and in what cases to choose something other than Docker. Yes, about isolation, it's indeed an interesting moment, because the range of solutions is quite large, meaning you can get very involved, there is more native isolation, a finer part, for example, the same Bubblewrap in Linux. There are specialized solutions like Datadog, E2B, and so on. There is the same Docker, which in this case we chose for simplicity and better developer experience, because it was convenient to enter the container at the right moment, see what was happening there, and there was no need to spin up a new container for every little thing. This is one company, they have a common container that works on different tasks, they don't have a billion parallel requests. So there's no such problem. So I would choose this heuristic. If you highlight, if you have a need for, first, ask the question: is it a monolithic or microservice architecture? That is, if you, well, as here, for example, one agency and everything revolves around it, then it's one solution. So here Docker fits perfectly, and even quite possibly, it's one that doesn't shut down. If you have a huge number of clients, that is, some kind of multi-tenant project, and you need to start and stop a huge number of containers constantly, then there are a huge number of solutions from Codeflare, from Datadog, and so on, they provide sandboxing as a service, meaning they provide a high-level API, and they allow you to launch these sandboxes on demand as much as needed. They execute, they launch, they shut down, and so on. About sandboxing, I'll also say that Docker alone, meaning there are a huge number of tools. That is, you have a large number of network isolation options, how to mount read-only volumes or how to work with temporary packs, yes. Docker itself provides a huge number of tools. And I'll also say that there's a cool technique, it's proxying network requests. That is, you can route all your requests through a proxy, for example, Nginx proxy. And you, that is, for the agent, yes, and inside the container, it won't even be obvious that there is some environment variable that is being substituted. That is, it will be a normal request, and you intercept this request from the outside and inject, for example, an environment variable. This is a very cool technique, we sometimes use it. It's a bit more complicated to implement, but it's top-notch for security. That is, in principle, even if code-code goes into another folder or, you know, it has no access to anything, that is, in principle, everything it has, it thinks it's making a normal request without a header, and the header is added from outside the container. This is a very powerful technique. Damn, I highly recommend everyone who is concerned about security and is going to use something similar to re-listen to this part. I really like what you said. Let's ask the last question and move on to Valera. I'll combine several into one. The first was about how testing is done. Were there any question-answer pairs, or was it an LLM as a judge, how is such a system tested when it's not a classic RAG, but something more flexible and agentic? And also, this prompt engineering, which is essentially reviewing the result, was it manual, or was it already an agentic evolutionary self-improvement theme, when the prompt changes based on some logs, as in the recent ERC3? With reflection. Yes, yes, yes, I understand. So, in order. First question about tests, benchmarks. We made a very simple benchmark at the first stage, it was essentially with checkboxes, i.e., boolean. Found, not found. That is, we asked a specific question, and the client came up with 50 questions, and we checked with regex if what he was looking for was found. They were all meticulous questions, i.e., non-obvious. And most of them, well, these types of questions failed one way or another in RAG. At least due to output limitations. This is a primitive enough setup. We have now connected, well, we connected almost immediately, rating and collect logs to make the next round and benchmark based on the real data that their agency uses, i.e., real data. In general, benchmarking, well, evaluating agent systems is very complicated, it's more complex, many moving parts. Evaluating it by internal parts doesn't make much sense, because that's the point, you give the agent freedom. Externally, well, there are three directions, traditionally. deterministic, evaluation, as a judge, and human as a judge. We used deterministic and we, well, R-based, and we used human as a judge, that is, the agency employees themselves actively tested it and said where something was wrong. In general, as an evaluation framework, I am very skeptical about LLM as a judge. This is a topic for a separate talk, probably, but in short, it's a separate topic. Second, about prompts. The first version of the prompt was written by code-code based on input, including call transcripts, and having studied it without access to the dataset on which we tested, but with instructions on what scenarios might be. Then we iterated together with the client in such interactive sessions. Great. Thank you, Rifat. Very interesting questions, interesting answers. Let's move on to Valera. Thank you all. Bye. All, bye-bye. Valera. Yes, can you hear me? Yes. Okay, I've already introduced you and Rifiat a bit at the beginning, but now a few more words. If Rifiat spoke mainly about proprietary technologies and cloud solutions, then Valera will now talk about corporations that will never transfer data there, and about the specific cases he had and what he achieved. He had many cases. Valera, start. Yes. Hello everyone. Let me share the presentation I prepared last time, but I probably wanted to. It's slightly updated. Just a second. So, let me briefly tell you what we'll talk about. As they say, a monologue, I'll talk to myself and to you. I want to touch upon the topic of local agents. Rifiat told us great things about the cloud. Actually, I'll tell you, I really liked a recent meme, Cloud Code SDK support was added to Ollama. And those who tried some quantized local models will never launch Ollama again and won't connect it to Cloud Code just to, in principle, get a feel for it. And here, of course, there's this trade-off between using the cloud and getting maximum benefit from models right now. And using local, yes, locally available models, downloaded, yes, from Hugging Face, under open licenses, and trying to adapt them to our business. Accordingly, this will be even more, I would say, technically complex now, let's say, a presentation. I'll try to explain in very simple terms, focusing more on the business. There will be many complex diagrams, and I'll stop at each one in more detail. I'll have more time. We'll go through a small historical overview. You know, like, let's go back a bit in history. And accordingly, in principle, let's look at some solutions that are currently spinning in my head, as, as Sasha Abramov, Head of AI Principal Engineer, told me. Let's introduce ourselves again. My name is Valera, I am Head of Fire Metro. Currently, I have launched quite a number of different services, interacted with various industries in Russia. Specifically, companies that have their own products, goods, or in principle sell, yes, their solutions to a large number of clients for a long time in IT. Last year, I started developing an agent framework. I'll talk about that too. A brief historical correction. No, for 10 minutes. And you know what the question is? When I was sitting at this presentation again, I had a thought: are there really agent solutions, yes, for any frameworks for local agents, like a core, because many people like to talk about frameworks, about end solutions, about platforms. It's generally, yes, in the enterprise, anything that's not a psh or not a deed, it's a platform. New platform, next platform, updated platform. Accordingly, I want to understand what needs to be known, because FOMO is catching up, and we are constantly missing something. A huge amount of news comes out daily, and you can't escape it all. There's always some cloud of tags in your head. Hardware, security, LLMs, yes, tel, sandboxes, MCP, volumes, everything that FatAT talked about. But now imagine that we don't have a response service, yes, well, where the provider has closed all this from us, and, as they say, we've become vendor-locked into the API, and we've stopped developing ourselves, and in principle, we say: "Well, great, the MCP will spin itself, the depressch will work, and internet search, and I'm now sending my history there." Yes, I don't have to carry these huge arrays of messages myself and come up with something with them. And something with RAG has already been invented, and documents are unpacked themselves, you know. Everything is great. But it would be a fly in the ointment, not as cool as if we understood that all this would be available to us. In the local business, first of all, we try to protect our clients, to do everything according to certain norms, standards, laws, yes, so that it's all internal. Plus, yes, having, well, experience interacting, for example, with banks or with more serious organizations, there

It's not really possible to just take and create an agent on some Open ASDK AG, yes, to give it tools. And then we just ran off somewhere into the cloud, yes, to send Haiku requests. That won't work. Therefore, yes, I want to discuss trends with you a bit now, right? So, we discussed the cloud, that there are such thoughts, how to actually build local agents of the core or platform in 2026 and what, well, basically, we would like everyone to go through, and maybe, in passing, in principle, hear about it, and it will layer onto your experience, and you will gradually become more skilled guys. This is also a presentation, I would say, very competitive. At the moment, everything is changing rapidly. From Open AI around 2022, when they talked about their experience building RAG, and how much needs to be applied in aggregate, not separately, right, to achieve some high accuracy metrics across various measurement systems. Here you can see different approaches separately, like potential documents, expansion is probably familiar to someone, when we expand the user query, when we experiment with chunking, sliding window, changing vector models, doing some experiments, yes, so we vectorize something, make top queries, experiment with it somehow, install and launch re-ranking models. All this works step-by-step, well, in the logic of things, with classic RAG, increases accuracy. But at the same time, yes, of course, as was also said, yes, it entails, probably, a huge amount of code, a codebase. And, uh, initially, when there were no ready-made frameworks, especially open source, for example, like RCKflow, Defy, Mastra, everything that Open Source now, in principle, covers more or less out of the box. There are, of course, difficulties, challenges with adaptation, but now, for the current year, I would say that most of these things are automatically covered by open source frameworks. But there is one caveat, yes, probably from the perspective that if we work in a large organization and adopt a framework, as soon as we move from experiments to a production-ready stage, then we will encounter something like security. And, well, for some reason, it turns out to be a secret that any framework that is brought into a large, quite serious company must undergo, well, at least, checks, like SAST, vulnerabilities, yes, some basic things. But given how quickly everything is developing, security, yes, there is usually such a tab on GitHub for some solutions, it is probably present in about ten percent of open source frameworks that truly adhere to and monitor some security in general, multi-tenancy, distribution, ensuring that it is at least, you know, enterprise-ready. And such a thing, well, is unavailable in most cases. Or Open Source Work earns on this. That is, you get some, yes, a dopamine hit from the fact that I will now take a lot of open source, apply it here locally, cover all of this, and not do engineering development. In my opinion, this is mistaken from one perspective, and from another, well, as it were, you can experiment with it, but as soon as you start to reach a truly large scale, you will have to redo and refine a lot. Well, if we consider, yes, classic RAG schemes, they really move away from architectural problems, code problems, and security issues to how much needs to be done, because, well, it's not available to us, it's not available to us. We have GPTOS and GLM, which was recently updated, by the way. Thanks to the guys. A thirty-billion model. Tell me there, how do you test it, like with a board. I haven't gotten there yet, I'm still on GPTOS on Queens. But given the capabilities of these models, we can, in principle, build such pipelines. I won't dwell on it, this is classic RAG, it's, in principle, naive. There are minor improvements. And such RAG, yes, in which there is everything the same. There is no React, it's not an agent. Everything here is on workflow. Moreover, workflow in code, not even a graph and not even some framework. It's just, in principle, separate things in code: guard rails, planning, query expansion, document embeddings, some kind of router, hybrid search, yes, checking if further clarification is needed, launching another cycle. So, all this is just a workflow that is advanced enough to run for about a minute on a local model and try to extract slightly more accurate data and work, for example, with multifaced queries, when in one query we don't just ask how to get an office pass, but a whole bunch of possible search questions. And all this, in my opinion, in a corporate environment, is perceived by engineers as a tool. Now it's already some tool in MCP, behind which the entire architecture is hidden, either built on an open source framework, which runs somewhere separately, isolated from the general environment, or it's a self-developed story, chosen from somewhere like LangChain, other frameworks, and integrated into the system, yes, the corporate environment, into the core, having passed all checks and its validity, yes, when we say that we want to move away from classic flows to agents, then here we can consider different cycle management systems, systems for managing, yes, planning processes. But if we break it down, we recently did a stream with the guys, again, on the much-hyped enterprise challenge, specifically the third agent one, and it turned out that the simplest agent, yes, on Open ASDK, five lines of code, with a slight overfitting, it has a distribution, yes, for tasks, it takes first place, simply by using a powerful model. Doesn't this seem to you or me that we often create some overhead or use too much, you know, this hardware and this onboarding engineer magic. So, maybe everything should really be solved much simpler. We fit prompts for tasks, do classic React loop, which has been around since 2022, and let's go. Yes, we don't do anything additional here. But that's not the case. In reality, a lot of things have been invented that allow us to build more deterministic systems. After all, React is all about quickly winning, getting good, good, acceptable quality of solutions, even on local models, but it's not about 95, it's not about 98, yes, 98% accuracy. And here I would also like to draw attention to the history, to the history of such a pleasant solution that makes LLMs return JSONs to us. But if you are still prompting an LLM to return JSON, that's good, of course. In principle, cloud services can handle all of this and even sometimes don't sneeze or break on quotes. Well, and if we talk about using local frameworks like VLM, for example, SGL, then specialized systems have long been invented for these frameworks. Here's a small history of how it developed. The structure was developed, and then function calling was added. And now, indeed, when using function calling or structured output, we get very good working JSON objects that we can then use in our business logic. And this was preceded by really extensive historical work by the Open Source Community, which developed systems like XGrammar. In general, in principle, I would say, systems that support grammar control within frameworks. That is, roughly speaking, when the model generates the next token, we don't just generate it and give it back to the user in streaming mode. We additionally check if it conforms to the declared schema. And such an approach, in principle, generates a deterministic part of all this huge number of possibilities that you wanted to have. Yes, function calling didn't work very stably at first, and it was, in principle, a long time ago, yes, the historical note will still continue, it's not for 2 minutes. In 2024, Open achieved indeed, like, 100% accuracy, and somewhere, yes, in 2024-2025, they reached parity. That's it. And local models can also generate JSON accurately. I'm not saying without hallucinations. And, accordingly, we are now approaching the point where the agent core is indeed a sufficiently, yes, combined path, where earlier you had to write, probably, this much code. So now these branches are either assembled into ready-made frameworks, or we have long passed this over the past years of engineering and understand, yes, how to build RAG, how to connect MCP, how to write it on Fast MCP, which SDK to choose. But I haven't written percentages here. If you decide to build an agent platform in 2026, then, in my opinion, this is approximately the percentage distribution of employment you should have. And I specifically haven't set priorities here. What to do first is up to you. Sandboxes, evaluations, choosing an SDK that will manage all agents, choosing some agent loop, which will be a graph-based React. Well, you can read about the competition between OpenAI and LangGraph, yes? Well, in principle, you choose yourself what to build a bicycle from, and what your standard will be for LLM integration, how you will write MCP, declare for your different product directions or individual teams, and how it will work together. Accordingly, if we talk about choosing LLMs, then, probably, yes, about six months ago we came and helped select models for various companies, that is, in principle, we created, yes, internal evaluations, yes, we tested them somehow, but now, probably, some popular leaders have emerged, which are quite broad, that is, for example, we constantly experimented at the beginning of 2025. What will we do, what benchmark will we collect, how to accurately check that the model will respond well, but now we have an absolute leader in open source, yes, it's QuT, there's still 4, but I would say now that GM, Gomka, yes, and in principle, they also fall into this category, judging by benchmarks, not yet by popular recognition, but Reptios is not for nothing in first place in various agent approaches. I confirm my practice here, probably, you know, not practice, but my observation of what people write, I confirm by what I saw that a sufficiently large team uses GPT 120B in a very huge project. And it is precisely this that is the core, yes, of this system. And here I would say that we experimented with it at the start. And here it can be said that the model initially did not show good results due to harmony problems. I myself tried to patch Volm, make some settings, in general, I returned to stable working Quen so that everything simply worked. I also conducted various experiments with Gigachat. At the moment, yes, so I will still experiment with GM, because I want to see how the thirty-billion model actually works and what it can produce. But at the moment, there is a kind of parity. We have several open source leaders. We are waiting for Anthropic to release something open source, I hope. Oh, I'm keeping my fingers crossed that they will also break into this competitive story with some small open source model. But if we talk about model sizes, yes, so we chose some model, some groups of models, then at the current moment I would distribute it like this: for the agent core, you can use OS or KN for coding, again, based on insights, from communication with various companies that are trying to integrate it internally, and I would like to make a remark, this is not that we place the cursor and connect it to OpenRouter. No, this is, we will download Open Hands or, yes, install, write our own plugin for VS Code and install a model that can tap, do some React logic and write code. So, this is approximately the range, so Quen 235B or Quen 480B, depending on the available VRAM, will be able to cope at the level, at the level with Anet fourth with coding tasks. What at the moment, probably, yes, you see marketing materials that smaller models can handle large projects. Well, that's fundamentally not true. In reality, we simply have a situation where everyone divides differently what coding tasks are for them. For me, a coding task is when I can have four simultaneous terminals locally, yes, and develop my local, yes, in parallel four tasks, where the codebase has already grown to 3,000, yes, lines. And there is a huge number of microservices and infrastructure. And someone considers a local coding agent to be one that can suggest a function. Well, why not, yes? And this, in principle, works in business. It works in practice. Rena Renat touched upon the topic of sandboxing, but a very cool article came out on January 5th. This can be considered an update. About how to choose a real sandbox, especially in a local environment, and how to do it safely. Containerization is good, of course, but if we talk about security, then a container has more or less the same core as the host system. Therefore, we return again to the history of some years ago, when KVM appeared, full virtualization, yes, from the kernel to the devices, which provides a higher degree of isolation. And now the top is microVM, like Firecracker, where higher-level isolation can be used. This will be very safe indeed. By the way, VS uses this microVM approach. If you have used BDR Agent DK, it turns out that for sandboxing, microVMs are used, which boot up in a few thousandths of a second. This is very lightweight virtualization that allows complete isolation of the core from potential injections, potential problems with, yes, you know, I think there was a trick with empty documents. The model read an empty document, a script was injected into it, it ran, and consequently, all documents from the sandbox went to, yes, the attacker's servers. This worked, I think, at Anthropic for some time until they fixed some network-related issues. But this, in general, also relates to security, of course. We want, yes, that code agents and the core of our platform are indeed secure. And we want to believe that we are indeed building it securely enough. Therefore, microVM, in my opinion, I haven't touched it myself yet, but I have studied this article very closely. There is a QR code here. Yes, on the presentation, it's visible, you can read it. Very useful reading, a huge article from the developers or a developer from Mistral. If we have chosen an LLM, yes, we have talked about sizes, we have talked about sandboxing, specifically local, yes, so how it should look safely and what can be done with it, then I would like to note, you know, a certain phenomenon. It still doesn't cease to amaze me in terms of how it could be done. And I will again, let's say, for 200 people plus, I think others will watch later, tell you that if you plan to build an agent platform this year and in the first streams there isn't something like this system, for creating isolated benchmarks, score verification, task-level isolation, creating API method emulation, so that you can run different architectures of different agents, different approaches, in general, in a local environment for your product, then you have probably gone astray or you are being pushed too hard on deadlines, because at the moment, I think this is a precedent, and few people have paid attention to it, and in general, the recipe has passed and everyone has forgotten. And I consider this to be the second, you know, like Kaggle for ML models. Here is an example platform. I hope it will find the strength to improve it and make it available for uploading your benchmarks and using it, in general, for good purposes. But pay attention to the Ger platform in terms of how it is organized, how isolation is done, how sessions are done, what tasks are there. This is not about it being a silver bullet, I don't know, for choosing an architecture. It's about the fact that by using such a system, you can potentially control the development of your agent or agent environment or agent in general, yes, over thousands and millions of runs. You have local infrastructure. Why not run an agent on Quen a thousand times, which will also rewrite its own prompt in isolation with some mock data, and the prompt will be improved, for example, by Anthropic, yes, because we don't have anything from the corporate environment there. And then we will launch it isolated on the local environment, on local models, already in our enterprise contour. And if we talk about hardware, then once again I want to remind you, yes, we have, yes, a quite large, wide selection of graphics cards now. And I often see that consumer inference is indeed used, and this is a normal situation, because it is difficult to order, yes, H100s, A100s specifically at your location. They are currently busy everywhere, even in the clouds. That is, if we go to rent a large number of A100s for a long term, more than six, then you won't find them simultaneously, and it will be quite difficult. You will have to go through the clouds. Sandboxes, microVMs, agent skills, React loop, tools, MCP, yes, and, for example, well, my still top for inference is VM and EV platform. A small set. I just simulated all this through a cursor, made a small table. But in general, it reflects, yes, just a small summary of what we have discussed. Well, how to solve the problem of building local agents, yes, this is also a separate story. I specifically left a teaser there, Open Code. Yes, or I also have here, in principle, a solution, yes, about Codex, this is about some, well, open-source ones. Now, and probably, NChain, Defy, Master, and Q can also be attributed there, so that, in principle, something can be built. And the most, in my opinion, problematic story, which I resisted, is that it is very difficult to get a team not to build complex bicycles on, say, LangGraph, LangChain. And coming in two months to launch some direction, you realize that the agent is this much codebase, the guys are stuck on version 10 versions behind LangChain and in general, yes, they can't even read normal logs, and they forgot to connect fuses at all. And here a thought came to mind or an idea, yes, that the creations that were made with IganCore, which is now a framework, we released it on PyPI, it has Docker containerization, and soon there will be a ready-made Docker image. Yes, to assemble such a solution and distribute it within the company as a standard, well, again, not mandatory, turned out to be a somewhat winning situation from the perspective of support and from the perspective that I gathered a small group of people who are engaged in the development of this agent framework. We ourselves have built quite good quality agents on them, passed them through benchmarks, understood that they really work on the models we chose. We packaged this framework and started, in principle, to develop it further. Now we have a small core team inside that works well with our local solution, with our framework. And also, of course, if you can also build cool, yes, internally on OpenAISDK or on other frameworks, that's generally welcome, but the essence was precisely that we can easily translate our expertise now, quickly come, understand, there is no, dependent codebase that is huge, that is pulled by the open source community that has accelerated to light speeds in commits. We have reached the minimum, possible, necessary functionality for our framework and continue to develop it. This, probably, was a distinguishing story. Well, plus we released it into open source. In fact, most companies did the same, but some forked LangChain, while remaining on some old versions, which are now being completely rewritten from scratch. This is, as it were, the thing that I didn't want to do. And we released it, yes, gave access, and configured it well for precisely the local models with which we work and which we deploy ourselves and deploy to clients. Accordingly, there are more stars there, almost approaching a thousand, but this was a conscious, yes, choice of what to make a lightweight framework, yes, and not on a whiteboard, it was on a whiteboard at the start, now it's being done by engineers. And in principle, from some point, we started connecting, say, the cloud for writing some new features, like search by tool, yes, for example, a more universal context system, and so on. But this is still our framework, we manage the backlog ourselves and know when we have a new release. And we distribute this release, you know, like a virus within the company, what new things we have brought. Yes, and we don't have to constantly chase open source frameworks in terms of updates, yes, studying what has happened, what has broken, waiting for contributors, waiting for PRs, we manage it ourselves. And if we talk about the distinguishing features, yes, of the framework, here it is simply, in principle, we applied a sufficiently complex approach with a reasoning tool, and for non-reasoning models with an instruction, this thing works. If you, yes, plan to build a solution based on a graph framework with a reasoning model or a cloud model, you know, it will definitely work worse, because we refined it for small open source models and instructions. Accordingly, how such good control is achieved in our framework for us. We have made, yes, several approaches with iterative planning, but without additional tools. That is, the agent plans within the tool call. This is the most distinguishing story that, in principle, yes, out of the box, I don't know, somehow stands out from other frameworks. We have well-developed planning within the framework with small models. We have tested it, benchmarked it, checked it with a lot of guys from the community, confirmed the hypothesis, use it everywhere as an opener and a starting set of functionality. Our own agent framework, local, small, tested by all security service articles, which is compiled for, it has a PHP version, and that's enough for us. It turned out to be successful. Such a thing can be applied to quickly deploy, yes, simple prototypes and bring them to the next stages of development through testing and evaluations and connecting, for example, to different types of interfaces. Where we have already built this SG Core and with whom, there have probably been different cases. I will say this: we first built them into our own products. That is, all products that we have now, yes, in robotics and which I oversee, agents work under the hood on this framework. This also adds value to the company and value to the products, which, in principle, well, as I would say, you know, it smells like marketing again, but it is marketing, and it works well. And this is the thing that I would like to share with you. Indeed, to use not what, yes, is currently at the top and open source, well, everyone can do that. But it was interesting, could we create our own framework and distribute it to clients, would it even work. It turned out that a Russian company, yes, guys like me and our experts can do such a thing and sell it excellently to construction companies, sell it to large banks, and indeed integrate these solutions. And they also, most importantly, yes, work. If we want to delve a little deeper into the details, then indeed there are RAG and concepts. I keep returning to it, we talked about agents, but locally there are some problems, probably, when you want to build it from scratch again. And Refan, I remember, said to choose a file-first approach or RAG. I am currently, right now, precisely struggling to answer what is really worth choosing at the start, when you have nothing. That is, you have neither RAG, nor an agent platform. Because to do File-First, you will still have to bring the agent core into your company and build this system with sandboxing, tooling, and so on on it. That is, if we are talking about some quick prototype solutions that are not scalable, of course, here you can download some ready-made solution from the perspective of Open DCK or some other, yes, agent frameworks and show it locally on your PC as a proof of concept, yes, as a presentation, it's okay on local hardware. If we are talking about scaling your capabilities, then last time I said that, yes, you start with a feast approach and indeed it will probably save you time quickly, it works well in the cloud. But if we are talking about, yes, when we build RAG or when we build a first approach in a company, there are measurable things in terms of laboriousness, then if we want to make it so that a huge number of people can use this thing and it is protected, it is safe,

So, in principle, yes, I would start with RAG, but in the process, I would integrate the agent core at all levels of the platform or at all levels of the product, and immediately hire when reaching some parity. With infrastructure for boxing and security, already a file-first approach, yes, in order to reduce the load and decrease database duplication. And if, you know, how to switch to what to ride on, yes? So, we talked about what our bicycle machine in general consists of, right? So, a different set of some engineering solutions that need to be brought in, made safe, made thoughtful, and, yes, with some trade-offs, then at the current moment, I would say that we have, well, two leaders for sure, yes, and many, I know, are refining them and are definitely making some updates from the logic point of view. So, someone patches VLM for specific settings, someone makes some changes in SGN that they need, for example, with specific data formats or their own fine-tuning models. But now, at the current moment, probably the most stable solution, and stable means at least four versions ago. And something is definitely written, tweaked with a file, yes, with Belming - it's somewhere at a version further back, because they have a bit more unstable versions lately. Well, perhaps all this will be resolved, yes, when the Linux Foundation takes a bit better care of Zylon, it, by the way, was also transferred there, just like MCP was transferred to CPP quite recently, in my opinion, excellent, friends, install it on your laptop if you have, for example, a Mac on M1 with a large amount of memory or at home you have access to a graphics card with 16 GB of video memory and Windows. We download ready-made, yes, images or KD or applications, download models and test some fast inference, yes, of what the model can generally answer. But in my opinion, any model conversion from the initial format still carries some losses. Although the metrics say otherwise, my personal feelings are that when we are dealing with FTN with a model in its initial resolution, yes, FP16, it is much more reliable in terms of long-term prospects for eliminating errors in production. If we are talking about a model, then at the current moment there are either ready-made frameworks that cover all this, or if you don't use ready-made frameworks, you can take a separate solution. So, we take, for example, the top of some formats that are used in our company. And we are not talking about pictures now. Pictures, scans are a separate story. Definitely, if they were in the company and you were involved in their recognition or, yes, in principle, locally, then there have long been pipelines that have been worked out. It's just that now OCR pipelines can be improved with, I would say, micro VL models, small VL models, so that, in principle, the OCR results can be sent to an LL model and, for example, yes, somehow improve them. I've seen such pipelines, and they work well. But when we talk now about high-precision automated document recognition without human presence and additional verification, well, VL models still cannot cope with this with sufficiently high accuracy, yes, they create a sensation in terms of the fact that you can throw 5-10 documents into them for testing and say, wow, it recognizes them, but then you start to figure out that somewhere in a long guest it confused numbers and so on and so forth. There will be a lot of this. Therefore, at the current moment, for a profitable local solution for multimodal recognition, yes, of documents, I would say that, well, loudly no, but it is being refined with a file by each company in terms of its requirements. Well, if we are talking about the classic format, like PDF, where we have tables, yes, where we have text, well, some presentations, where there is also text, where it can be extracted, Excel files, yes, and so on, then there are already long-developed favorites, you can, in principle, pay attention to them, or combine them into some, you know, pipeline one after another, or choose one. For example, when I watched a presentation from Justy a long time ago, the guys chose Doclick as their favorite in their teleplatform. When I watched what the guys, I think, were talking about somewhere at the Pedation of SNK, they took markers, they refined the marker [ __ ]. So, in principle, yes, I took the top that I heard about. If we are talking about MCP in local solutions, then let's also switch to the topic that, again, if I return to the previous speaker, yes, Refat said such a thing that we had a repair in MCP, and it, in principle, clarified many points, many points on how it should look. And probably, I will say this, the main problem that existed, most fast, high-quality software that has all the functionality is written in Python. And for AI, yes, if we take Fast MCP, the fastest updating repository is fast MCP in Python. And if your company writes in PHP or writes in, I don't know, Java, then you have to wait for ports or for their sister repositories to catch up with this functionality. Because, for example, OAU came out 2 months ago in the PON repository, and in PHP it comes out a month or a month and a half later. Either you have to refine it yourself, or wait for the guys to finish everything, based on the standards that were transmitted from above. This is the first thing. The second problem, probably, yes, is that there is some misunderstanding of why we need MCP at all. But I would like to explain here that at the current moment in local solutions, when we take and develop MCP, in my opinion, the only answer you should give yourself is that I am doing it so that a lot of clients within or developers who make agents, that is, for example, we have a CRM product within the company or a product, I don't know. Let's say. The HR department has its own API, for example, yes, but all other departments want to connect to this API. All other departments develop agents to communicate with these APIs and so that, yes, we know for sure that everything corresponds to a single, well, business logic, that all tools are unified, yes, tools are checked, tools have the correct business logic, name, that is, yes, name, action, brief description, well, the classic naming approach, because if we give the HR department and 10 departments agents, then imagine that you will have 10 toolkits, everyone will have different naming, everyone will have different descriptions, and everyone will perform this logic differently. This is where we come to this standardization, that we actually wrap an API method in a single standard for other departments, who will develop their agent based on the HR API. We will know for sure that when we call search employees, yes, the same string will be generated for the description and the same tool description will be for all agents that are built within the company. I don't know of any other explanation at the moment, except if you want to offer your product externally to a lot of other clients. In principle, if there is no such need, then you can stick to one file, one folder of TOS, where tools are written simply in code, where there is a tool declaration at the top, and an execution choice below. And here I want to tell a case from the point of view of Remote MCP, how, in principle, yes, the procedure looked like in the development of MCP for Vkusovall. I can say now, because the case had not yet been released for me. Initially, I made a prototype by parsing the site. You can look at my channel, yes, Vkusovall is probably the first retailer that released MCP in Open Source. So now you can, in principle, take, in Cloud, in Cursor, yes, anywhere, throw in a URL and their MCs, find the necessary goods for, I don't know, any dish and make a basket from it, and from the basket, go to checkout, yes, it performs only three actions. This is search, this is extract additional data, and create a basket. But it was indeed, of course, developed. So, we initially did site parsing, packed them into Toolib, then packed them into MCP, after that we developed new Rest API methods, rewrote part of the business logic, and wrote a separate service that, roughly speaking, repacked another internal API. And in about 20-30 minutes, I wrote an example in Python on Cloud, gave it to the development team, and they wrote it in PHP. Indeed, in local services, it is very painful that initially, yes, we would like to perceive the MCP protocol as precisely, you know, such a KPI gasket, but this is not always the case. And binary files cannot be put there at the moment. PDFs, pictures, yes? So, all this will still require workarounds through skills or through direct loading into the resource, where, for example, there will be instructions on how to do it. There is a slight cultural resistance always. Well, not always, but in my opinion, in the enterprise environment, it still exists and is spreading that we have made our own solution, it works, we will not try another one. And this is not, by the way, a common problem, these are my observations. When a team is developing an agent, it wants to get higher metrics, but it doesn't listen well to suggestions from the outside. And here it requires a personal example, literally, this is, you know, my case, where I really don't just, yes, tell, show some training materials, but literally come and reassemble the agent, yes, on Open SDK or on SGE literally with a wipe code in one evening. And, for example, in the previous section, four requests from this team cost, yes, $0.27 for using a cloud model, for example, for tests. And I, for example, use a cheaper model, yes, much fewer tokens. And, in principle, all this helps me, yes, coding agents. Cultural resistance is that we have made our own solution, we will stroke it, caress it and adore it. We won't even look at anything else. Therefore, at the current moment, well, this is probably happening everywhere with completely new AI, but this is just my personal example. I don't know, maybe someone resonates with it, but so far I manage to eliminate it by personal example. And this is a story, as it were, from the point of view of positioning who you want to be. Probably, I've already talked about the results, right? So, what I use in my work constantly. Well, here, I won't repeat myself about it. Yes, so this is such a gentleman's set. Now Cloud Code has completely replaced Cursor. In general, Cursor only opens as a visual IDE. I can't get away from it, but the terminals at the bottom are open, of course, with Scon. Here we will share the presentations again with everyone who came. Newcomers will definitely get links. Here again is a summary table of what we have discussed. About local infidс, how to build it, what to know, yes, in 2026. And in principle, how I see the agent core, yes, in principle, I would say, my agent core does not differ much from the agent core that Fat talked about, from the point of view of, yes, how to do it. Well, here, probably, the history of local models, how to infer them, about, for example, 152 FZ, about, yes, transboundary issues, about security, code verification by the security service, and much, much more, because it is very important to know and in reality, well, you know, ignorance does not exempt you from having to rewrite your agent Slowain into some real ordinary code that clients or enterprises are currently doing. Thank you. How much did I stay within the timing? Is there still time for questions? I think there is. Yes, Valer, go ahead. Since today is not a strict conference with a lot of speakers following each other, I think we can chat a bit more. There are interesting questions here. And in general, I wanted to say that I am, of course, super close to this approach of not dragging huge frameworks, but making some small thing for tasks, especially when you know the subject area. Here, as far as I understand, correct me if I'm wrong, the strength of this SGR Core framework is that you have already eaten a dog on typical orders for the Russian Federation, and you understand what needs to be brought in and what doesn't, or am I misunderstanding this? That's right. Probably, I would also say that with RAG Core, I manage to show examples of working agents in Reacloop quite quickly, personally, using local models, yes, and closing, showing build cases, that is, not saying that I will first develop the agent core or rework some for a local model, that is, there are already wooden tools described there. Super clear. Mts are very small, the flow is strict, deterministic, with a lot of checks, strict everywhere, the model cannot deviate left or right. That is, there is a constraint. And all this is out of the box. When you take a cloud model, you still don't have constraints or strictness there. You try to remove it, because in general, it also slows it down. But when we say that we are in a local environment, we want the model to do what you initially intended, a little more deterministically than to invent React. Therefore, at the current moment, the framework from Shared Reasoning has turned into just a framework that works well with local models, which, in principle, performs business functions. We have written a lot of various convenient APIs for it, and we are continuing to integrate it into business. That is, we have, roughly speaking, eaten dogs. And what else is interesting? We are now going to conduct a rather global training within the company on the use of the framework and, moreover, with passing the store benchmark at least 85% using this framework. This turned out to be a cool idea. And everyone embraced it, including a competitive spirit within the company. That is, imagine, now 40 people on Idgar will go to torment Renat's platform again. I think that the store benchmark is really one of the coolest cases for training in general and engineering. By the way, someone in the comments asked what to study. I posted channels there. But besides channels, I highly recommend trying to solve the store benchmark in C3 in Renat's LLM under the hood channel. Yes, yes, let's go back to the questions. Look, you say that this framework works very well with small local models, and you gave two columns, right, very small models of the 4B level and larger models. Can you roughly estimate some tasks whose solutions you can, well, extract from, yes, from, from this framework in conjunction with models, there, 4B and in conjunction with larger models? Well, yes, let me say it broadly, in terms of categorization, yes. The main task is Deep Research, that is, in principle, well, there is still quite thoughtful tooling for integrations. We are working on it separately, how to compress context, how to show models, how to, yes, artificially, break a step and transfer it to a new session. There is also such a thing, so that the model doesn't die on the tenth step. If we take QuB2507 Instruct, then here we can say that the model can quickly perform a small number of queries for Squarey Expansion in some API, for example, Git, if you need to search the database, summarize and give an answer to the next stage of the pipeline. So, such small models are mainly assistants, as Max would say. That is, it is an assistant. That is, it is not a model that can do write, yes, or delete, or create. That is, it is about support, about searching for information. Here we take hypothetical document bending, query expansion, and a couple more techniques, technologies, yes, and advanced RAG approaches that we have, and simply pack them into the model in Reacloop. This is what they can do with this framework. And in fact, it just closes it quite deterministically. Slightly larger models can do research on the codebase. This is directly proven. For example, you have a React agent Quen 3 30B 2507 A3B, which is a Mac. For reading, analyzing, and quickly writing code, we take a Mac code of the same type. And in conjunction, these two stories with the framework can do quick research well, yes, on quite large repositories, simply grep, search, and make some, yes, logical conclusions and analysis. Okay. Something like that. Uh-huh. Essentially, yes, 4B is enough for solving ordinary text tasks. That is, you can run it on your laptop. And I will have my own press. But for code, you need larger models. And here are the comments, why wasn't it mentioned and that Snimak and Num recently released a model that is available for deployment. Look, the thing is, everything boils down to the entire infrastructure and how much cache you have available. Almost, well, few in business run models larger than, let's say, 30 billion active parameters. This is not suitable either by time, it is not active, right? Because if we look at PTIOZB, it has 5 billion active, and Knatri, yes, and all this fits well into the hundredth or H hundredth, yes, I think in Hopper there is hardware acceleration. It fits well into the Oson. That is, you don't need to have problems with launching on several, yes, with Invelink, if you want to enable some speculative decoding. All this is not there. You have a thirty-billion model, one card with five parallel requests on Vilm. And if we talk about a four-billion model, then two or three instances fit there. And this is with the maximum available, ordinary context. But it is not always, well, never used. The limits at which local models currently work in contexts, purely for summarization, are 80,000 tokens. Everything else is simply cut, well, in pieces and summarized in pieces. So here it is all about infrastructure. Yes, let's ask the last question about technical details and then we'll go to philosophical ones. There's a cool question here. Yes, go ahead. On the technical side, look, you just talked about Asata, about Asota. I remember you talked a lot about how you set up a cluster of 4,000, I think 90s. Yes, 4090. I can show it right now. I have this cluster open. I thought about showing a small demo on it. Well, these are the graphics cards. Two RTX4090, then throttled. Valer, you can't see it. Your screen is open. One moment, and it will be visible on the desktop. Now it's visible. This is what this story looks like. Three terminals. On the left, VLM is running with Knat 330B 3Brct 2507. On the right, we have a terminal with NVDS SM. And here you can see that it's actually an ordinary consumer computer with 62 GB of RAM. With a small amount of VRAM. And I have my developed local desk open. I can, in principle, now run some small depressor using this Knat. And, in principle, choose such a temperature. Let's search for AI news today, especially find AMD's comment. I'm writing it incorrectly on purpose. And I'm sending the request now, and it has gone here. You can see it in the logs, yes? So, here's my system prompt, everything has gone, inference has started, yes? That is, the model is already searching for something, extracting, and, in principle, processing it. Valer, but the question here is precisely why 4090, and not some RTX Pro 6000, which has 96 GB instead of 48. First of all, the 4090 in terms of its speed is not much different from Asota in performance. Well, it's even more powerful. Availability. I don't need any industrial server. To understand, literally. Valer, Valer, I think the question here is about RTX cards, which are not Nvidia at all, as far as I understand. Well, it's expensive. Let's put it this way. No, RTX is Nvidia. RTX A6000, it's called Ada. Which, if I'm not mistaken, costs around 4.5 million rubles with 96 GB of video memory. If I'm not mistaken, if we are talking about such a 4090, now on the market you can buy it, in principle, for 440, well, let's round it up to 4,000 dollars, 440,000 rubles, roughly, there, 450. And, accordingly, two such cards plus a server cost the R&D department, yes, one server. That is, a deployed local Quen 330B Instruck now costs 1,200 rubles, because it's a consumer server, it's a four-unit rack-mount chassis into which a consumer motherboard is installed. An ordinary consumer, as I say, that is, an ordinary motherboard with a 550B socket. An ordinary Ryzen 7 is installed there, which, well, you understand how much it costs, ordinary DDR4 memory, an ordinary 1000W power supply. And this server, for example, has been working for a very long time, for a very long time, in terms of cards. They were bought, I think, even a year ago. They were serviced once, and they are in the server room. So here, the question is, yes, it continues to search for news. The question here is, of course, much more complex, probably economic. I would say that at the current moment, I have probably managed to assemble the cheapest inference in Russia at the purchase price for 30 billion models. And it is deployed now in 300,000 tokens of KV cache, which is not available. This means that this model can be overloaded with parallel requests of 80,000 tokens, up to six at once. Quite a large performance. For a large number of people, this one server is enough for experiments. Many such servers have been purchased for coders, for VL models, and for other experiments. I don't see the point of putting more than two cards in one server. You see, under load, it consumes no more than 120 W. Therefore, I have talked about local cards many times. Of course, when we talk about higher load, stability, and we have proven the hypothesis, we buy Asots and hundreds of servers, six or eight pieces, and then we go to prove the cost. From the point of view of ROI. You are in the experimental stage, you don't know what hardware to choose. I think it's a must to buy at least one such server and, in principle, at least one such card and understand what can be deployed on it. How many requests can such a server handle in parallel? In parallel requests, well, look, the thing is, when the model is loaded, it is loaded in size, you can probably even look, if you do this. Kv. Not like that. And if I'm not mistaken, I'll try to find it quickly, if not, then approximately 300,000 tokens are loaded and available. That is, in principle, divide them by requests. Here, now 9.1 requests are Running. Simultaneously, you can send to it, that is, you have 330 available, and you can cut them into small ones, as many as you want. For example, if your agent consumes 14,000 tokens at startup, well, that's a normal agent. Here, in principle, it is. Well, by the way, you can look at the speed. This is with caching, such a speed. 5,000 tokens input, output plus or minus, the speed is also visible there, so it's normal, right? This is a lot. That is, the model is not broken. In short, you have access to the maximum context. In general, Knat, it has, I think, 250,000 tokens, it is trained on that. So I don't know, there are no such limitations either. And it is now the most okay. GPT 120B is also deployed quite easily. Well, in general, the numbers are clear. Yes, let's move on to the last question. Here's some philosophy. Where is all this heading? Let's go, let's go. Will agent developers be needed by the end of 2026, or how long will such a role live? How do you see it? And what might this role transform into if it doesn't appear soon? Listen, well, as they say, someone wrote to me on my channel, saying: "Why did I study code?" Yes, well, probably, it's all leading to the fact that, well, now a large part of experimental verification code is definitely being written and is already being written. That is, roughly speaking, I talked about local desk today, but in general, it is entirely my development with the community's help. Not a single line of code, in my opinion, has been written by people. This is a portable desktop tool for working with models. I experiment with it myself, play with some Open Color, it allows you to run a mass of different experiments, yes, up to consensus mode, running simultaneous tasks, and so on. And everything is on your desktop with its own sandbox. And at the current moment, well, I am a proponent of the fact that if you are not yet coding through AI, then those who code through AI have definitely overtaken you in the number of unfinished projects. But if you approach it wisely and don't overdo it, because I met with guys, they tell me: "My sleep has been disturbed for the last six months." I say: "Are you worrying so much?" They say: "No, I'm sending so many requests to the cloud. I think, now I'll go to sleep, I won't send anymore, I'll stop." But in the end, he sends more and more and more, and so on until 2 am, and it's time to get up, take the child to kindergarten, and so on. In general, it all leads, of course, to the emergence of new dependencies, new problems, but, in my opinion, it is a pleasant dependency if you have moderation in everything. Coding will not completely disappear from people. Just some part of it, yes, when people want to experiment, if they do it wisely, and when people want to understand the goal of what they are doing and can understand this goal, then, in principle, it leads us to achievement more often than if we didn't have it. That is, AI, in particular, agents, coding solutions, and in general, yes, information search. I hear a lot of good, positive voices, that, yes, to appease FOMO, people are really building agents that collect information and somehow accumulate it. And this is happening even literally, you don't have to look far. I look at who reposts my channel posts, you go in, and there are six bots. Like this, you understand? They reposted someone who wrote such a solution for themselves into their channel. This indicates that the trend of writing ready-made services, yes, by paying $20 for a subscription, a person can make a bot that will solve the problem of information collection. It has been here for a long time. But there is also a big gap, that is, between those who really have access to all this most powerful, we are definitely power users and the future for us, in particular, but there are also those people who cannot get access to it and, for example, still have difficulties starting. Therefore, here I would, there was some interesting sound. Right. And I would, in short, say this: that it's great. I'm glad that it's generally appearing in our lives. And the philosophy boils down to the fact that there are both bad and good sides, but if you see the golden mean for yourself and you are comfortable with it, why not? Get involved. We have talked about all our channels many times. We try to review and make dry summaries of what guys, what Kolya from his experience, how we are moving through all this history, how we are moving. Therefore, I definitely suggest starting this year not only with runs, but also with installing cloud code in the terminal. And on this positive note, I think we can finish. With us today were Rifat Ametov and Valera Kovalsky. If you liked these reports, then I advise you to look in the description of this video. There is a link to the conference and the results of 25, where there are many more cool reports, both reports on development with artificial intelligence, and on the integration of artificial intelligence from the point of view of building processes and from the point of view of business. With you today was I, Kolya Shayko, as the host. We are finishing now. Have a good evening, good weekend, everyone. Goodbye, everyone. Goodbye.