AI engineering 2026 - LLM, RAG, AI агенты, разработка и инфраструктура

Gary Gary Gary. Okay, everyone, hello. We are starting the broadcast. I hope I am visible and audible well. A couple of words from me. What will today's broadcast be about? Before New Year, we held a conference with the results of the 25th year. The link to it is in the description of today's broadcast. And many great speakers performed there, including Refat and Valera, who will be here today. But the guys clearly didn't have enough time. We heard feedback and requests from viewers, so we allocated a full broadcast with Refat and Valera. And we will add both practical parts to the guys' reports and time for questions, because many questions were not answered last time. Here. And a couple of words about the speakers from me. I, actually, get incredibly excited every time I listen to Refat and Valera. Because, what is interesting to me, is this mix of engineering and business parts, which both Refat and Valera have, yes, like, each of them perfectly understands how business works. They know how to speak the language of business, and at the same time, they are both wonderful engineers who delve very deeply into details. Here. Well, and you, most likely, will see this yourself today during the broadcast. With that, I hand over the word to Refat. Refat, begin. Ah, guys, you can write questions in the chats. Ah, hello everyone, Kolya. I will need access to sharing. Yes, ready. Moment. Okay, it should be visible. Yes, everything is visible. Yes, everything. Okay. So, for those who were at the conference results, as Kolya said, some slides and some content will be familiar. This is a more expanded version of that same report. And, essentially, there will be a little more about cases, there will be time for questions. So let's start. Today we will talk about how the world has changed from the perspective of developers and systems. This is more of a technical report, but it will also be useful for founders and everyone who is building or interested in building AI systems. A little about me. I am co-founder of Death Stark and Spread Simple. We build AI-first products. I have 12 years, even more if you count commercial development experience, in development for 3 years. I am involved in GENI systems, my team and I deliver production systems for enterprise and startups. And probably the most interesting thing is that a lot of projects pass through me, I have a lot of exposure, and I've had the opportunity to see many of them from the inside, including large projects, besides those that we develop ourselves. I write about AI engineering. There are links here. So, agents are a quite obvious trend. Actually, quite a lot of people have talked about this. I will only say that this trend will intensify. I will not dwell much here on the fact that this trend is real, how it affects business and development. I will only say that according to the large analytical agency Gartner, this trend will only intensify. And I believe this forecast. And how has agent infrastructure changed over the year? On this slide, I wanted to show how agent infrastructure has strengthened, specifically for developers. There have been quite a few events. I have included only the big three: OpenAI, Google, and Anthropic. And even these are not all releases, but key ones. And notice that what we are used to appeared very recently. The response API, it seems like it's been with us forever, but it only appeared in March 2025, meaning less than a year ago. The SDK is a little over a year old. The Codex SDK is practically recent, yes, after, significantly later, the Anthropic agent SDK, which we will talk about today. And all of this is to say that the Completion API has become much more agent-like, and APIs are becoming more and more agent-like. Here is a screenshot from OpenAI documentation, how they position the completion, or rather, the response API compared to the completion API. Many of you know the completion API, so essentially it's what we are used to. So you send an array of messages, you get a result, yes, so it's a completion. And unexpectedly, I think for OpenAI, it became like a standard for the entire industry. Almost all providers picked it up very early. And even those who don't support it directly, for example, Google, offer a compatible API. Not to mention that practically all open-source solutions and Chinese LLMs support the completion API. In this regard, it's a great standard, it provides interoperability, you can swap models, and everything was actually great, yes, and continues to be. Well, and as we will see further, there are a lot of new features. For example, grounding, yes? So when we see in feature releases that the model has gained the ability to search for something, a content interpreter has appeared, context caching, editing, computer use, deep search, and so on. And how much of this is about the model's weight or how much relates to the model itself, yes, practically nothing. These are all infrastructure features. Next, I analyzed the feature table, a rather large table on several slides, you can take it later. What is interesting about this table? The structure is as follows: there is the feature name, links to the documentation of the providers of this big three, and standard alternatives. Of course, very conditionally. Some alternatives are broader, some are narrower, some are not drop-in replacements at all, yes, but nevertheless. And the first category, retrieval from the web, is essentially RAG with web grounding. So with a modern API, which is agent-like, in my opinion, you get the ability to do RAG without any external tools from the big three. And this is also helped by searching external sources, i.e., web search, web fetch, so models, models in quotes, can fetch data, i.e., read it, see it, yes, and even run entire deep searches. Well, this is not news, but perhaps many didn't know that deep search is available as an API directly within Google, Google Gemini API. So this is not just a product feature, it's a whole API. And a rather unique feature for Google specifically is mapping. So you can directly analyze various geo-data, for example, solve the traveling salesman problem and see where the nearest places are, and so on, and so on. All of this is available, again, through the API. And here are some alternatives, all these links are clickable, you can see how to do it yourself. Next, documents. Here is a category of features related to documents. For example, the file API allows you to upload files to the servers of these providers and do something with them, for example, perform retrieval on them or reuse a file multiple times, for example, analyze a PDF, and so on. So to create your own small project, which you might have encountered in ChatGPT or in Code. And quite advanced document understanding. Again, these are not features of the model itself, pay attention. So it's as if the providers are trying to blur the line between the model and the infrastructure. Although, as engineers, I think you understand that for the most part, this is some kind of, well, very, roughly speaking, an add-on around the models, not the features of these models themselves. Memory state and state, a lot has appeared around state management. So, for example, OpenAI has state directly in the response. So you are not obliged to store the state yourself, so you can store messages directly on their infrastructure. You can do advanced context management, for example, relatively recent context editing. By the way, my company started with this, with this Context Editing release from Anthropic, which essentially, we used to do it ourselves, so we did a clever surgical runtime update of the context, and OpenAI, Anthropic in this case, released this feature, which does it much better. And this is a built-in feature in the API. And of course, all this leads to the fact that you can often save a lot of time if you know the API of a particular provider well, and you can avoid certain risks if, again, you know certain practices, which I will talk about later. There is, for example, a memory tool, so these are already directly agent-like features, more and more agent-like features are built directly into the APIs of these providers. I'm not even mentioning prompt caching, which, in fact, is, of course, an infrastructure feature. A huge number of releases related to tool infrastructure, to tools, have been made in the last quarter, in the second half of last year. Especially related to Anthropic, so search, programmatic tools, streaming to a huge number of specialized tools, for example, even the OpenAI shell to iOS, which appeared earlier, and special tools for working with code. All of this saves a lot of time when used correctly. Programmatic tools are something in between code execution and ordinary ones. The infrastructure executes several tools on the provider's servers, aggregates the results, and returns the result itself to the model's context. So a huge amount of this, this sheet, massive use of, simply does not fall into the context window, which is super effective, but it has its, of course, its price, at least the price of vendor lock-in. Code execution is a super important part, which, obviously, does not relate to the model, but relates to agent infrastructure, but is often presented as a model feature, even in official press releases. This is code execution. And you might have heard that, for example, Opus can work for 5 hours in the background. Well, come on, this is not Opus as a model, this is the numerous infrastructural add-ons that run this agent cycle and constantly perform these completions, yes, to provide the desired result in the desired direction. And as we see, there are alternatives. So one of the reasons why I added this column and why I think this table can be useful in many places is because you can often make a choice for yourself, whether to use a built-in feature. Of course, you will save a lot of time and often get the best result you can get with a limited team. Or still use an alternative. I will talk about this at the very end. How to develop such, one might say, intuition, what to use when. Valera will talk a lot about when there are some non-functional requirements, yes, when, for example, due to legislation or company security policy, you cannot afford it, or simply the founder's paranoia. But that's a separate story. It's not always, let's say, a technical solution. It's a kind of policy, yes, that is issued. I am talking now about when you have freedom of choice. So code execution has also become a feature of the API relatively recently, so the APIs of the big three. All of them allow you to execute code from models directly on their infrastructure and use it as results of your workflow, yes, as part of your workflow. Here it is called interpreter, at Anthropic execution, well, and so on. Alternatives and, of course, nothing has gone away. Well, computer use can also be done independently. You can use built-in features. According to my tests, built-in features almost always significantly outperform all attempts to do it yourself, except for rare exceptions when it's some kind of domain-specific thing, and there you can indeed get better metrics, but perhaps this is a temporary phenomenon. Well, and of course, Skills is a relatively recent release, which is being actively adapted, yes, a separate standard appeared at the end of the year, Agent Skills, and other frameworks and providers are actively adapting it. Why did I mention it here? Because, in fact, Skills can be used as an API. So the skill that you have packaged in your code, in code, you can package it into an archive and use it through the Anthropic API for your scenarios when all you have is a request that you can make to their server. Well, and finally, output, yes, output and operations. Well, there is the possibility of batch processing, there is the possibility to configure safety settings, yes, it's a whole thing to create safety wires, good guardrails, and fine-tune them. Google offers this as a built-in feature and says it's a model feature. Even Google offers observability, a kind of replacement for, say, Fuse, and so on. And you can see each request in their Google Cloud in detail. And much, much more. So the table would have grown even larger. I included the most important things. I think it's clear here, models are catching up, the gap in infrastructure is growing. We often see benchmarks where, indeed, JLM or Deepseek or we haven't heard from Deepseek for a long time, and Quen, reach almost parity with the big three. But we don't see this in practice. So why? Precisely because infrastructure is super important, and that's why you choose a stack, not a model. And often this synergy, which you can get by using a set of infrastructural tools that comes with the model, you get some unattainable performance. At least, that's the status quo as I see it today. And here's an analogy that I chose, where all this is moving, as it seems to me, yes, again, email used to be, well, email is a very old standard, as you know, even older than the WWW. And its own SMTP, its own server used to be the norm. Now imagine that we ourselves deploy our own SMTP for corporate email, even though there is open-source software for it, and it doesn't require much GPU, in general, server resources. It's super difficult because Gmail is a whole ecosystem. Besides the protocol itself, there are a huge number of infrastructural and product features that are hard to imagine without it. So chat was a standard. What now? It's funny that when this report was first presented in a shortened version, there was nothing like it, and I even predicted, or rather, requested some alternative to chat completions. And what we have, in fact, OpenAI released Open Responses literally a few weeks ago, which is great, I think, and shows how justified this trend feeling was, probably. And this is a very, well, a wonderful step, in my opinion, but it's still questionable whether it will work, whether it will take off, because there is a lot of skepticism in the technical, professional community that Google will adapt this standard as they did with the Completion API. Nevertheless, many open-source and not only libraries and providers have already announced support, for example, VSCEL, VLM, Ollama, and so on. I looked into this standard. In fact, it doesn't cover everything that PNI offers yet, but perhaps that's just for now. So, the second part of the report will be more about a case, and at the end I will talk about a decision-making framework. So, yes, first of all, it must be said that RAG has not gone anywhere, it has simply changed. And, if before very often through Telegram channels, blogs on Substack, Reddit, and everywhere, there were many questions, like which embedding, which embedding model to choose, well, a lot about retrieval, about these pipelines, about graph RAG or ordinary RAG, hybrid RAG. And, essentially, now it's actually 2025, 2026, yes, what source to use. And we understand that retrieval is just one of the agent's tools. Whether it's in the form of embeddings, in the form of file search, or SQL query, that's secondary, or rather, depends on the context. And the agent decides whether it needs retrieval or not. Before moving on to the case itself, I wanted to say a couple of things about where these solutions come from, the prerequisites, yes? So coding is the leader in AI adoption according to many, and research, for example, by Anthropic proves this. Their Economic Index shows that 44% of API calls are for development, much more than all other categories. Each, each code, code showed phenomenal success and continues to be heard about even more this year, and not without reason. In general, coding practices are the leading area of AI, in my opinion. And here is the most mature tooling and best practices. And sometimes bash is all an agent needs. Yes. Why? Because it's trained on code. Bash is a rather, well, old, let's say, way of telling a computer what you want from it. So you get quite accurate results when you need them, yes, so grep find is super fast and super accurate, and when you don't need it, you can use more fuzzy algorithms, including similarity search. This doesn't prohibit such an approach in itself. Minimal context, because files are loaded on demand, so you don't need to think about re-ranking and so on. And it's not even about files replacing databases, but it makes you rethink when a database is excessive, and when files are all you need, yes. Well, and regarding debugging, again, these are ordinary shell tools and so they are available both for humans and for agents. The philosophy of agent SDK is that essentially agent SDK, we are talking about Anthropic agent SDK, but similar practices are already offered by many other providers. The philosophy is that an agent needs to be given a computer and use the same tools that programmers use in their work. File system, bash, file editing, API calls, SDK, etc. So, essentially, these are wrappers, but this is also a tool, yes, and the cycle, the typical cycle is to gather context, perform some action or a set of actions, sometimes check the work in parallel. Here, a super important part is reflection, yes, on what has been done, and repeat as many times as needed until the result is achieved. I'll say right away that I'm talking a lot about code, about code agent SDK, and so on, but this doesn't mean that I'm saying, use only this stack and that's it. There are excellent alternatives from the other big three, including, for example, OpenAI Codex SDK, Gemini CLI, you can make them work similarly, although, as far as I remember, they don't have an SDK around it. There is OpenHands, it has been developing for quite a long time, there is OpenCode, a very good open alternative, and many don't know it, it also has its own SDK, which, in general, is not bad. Only it works a little differently, yes, they deploy, start a server, you access it via REST API. But that's not important, sometimes it's even more convenient. So I'm talking about the approach in general. And the architecture of a File First agent, as I call it, is roughly like this, yes. We have data. We bring this data as close as possible to the agent, to its environment, to its place where it actually executes code, because this achieves minimal latency and maximum ability to use shell tools. There is a runtime, it can be a local machine, it can be Docker, it can be a container, sandbox, and so on, and so on. The main thing is that there is a runtime. And there is the possibility to use these coding agents for non-coding tasks, or maybe for coding tasks, depending on your case. Tasks are on top of this. There is an interface. The interface can be SDK, it can be chat, it can be API. Here again, it depends on various requirements and on how, well, again, how it will be convenient for the user, or how it will be faster to implement. Closer to the case. This is a European company, it's in marketing. They came with a request, saying that they tried RAG and it didn't work. And, they put a lot of effort into it. They used a lot of techniques. I looked at what they tried to do. And, of course, this cannot be called a naive superficial approach. They used hybrid RAG. They probably only didn't get to graph databases and ontologies. They used almost everything they could find at the time. So, query expansion, I won't even recall, many techniques, they got hallucinations. The main, the main case, they want to search, they need new management and good understanding of the knowledge stored in Google Drive in huge quantities. And everything they need to know about projects is there. These are scenarios, various information about client positioning, information, call transcripts, everything, everything, everything, practically everything related to the project itself. I'm not talking about documents like contracts they signed, although that's also possible, but that's not the point, but about what is useful knowledge for projects. They have stored it in Google Drive historically for a very long time. And they need RAG for it. In order to, again, do their work, create new creatives, prepare for calls, set tasks, and so on. In general, the idea came during a call to replace it with agent SDK, because they had already started adapting code, and I was actively involved in it then. And we did a pilot, and it went very well for them. It worked and works perfectly. And periodically, small adjustments to prompts are needed, adding skills and tools. I will talk about this a little later. But in general, the speed of implementation and the quality of the result were significantly higher than expected. Key components in this specific case, so file synchronization, occurred through Aircon. And Aircon is essentially an API for cloud storage. In this case, Google Drive, but you can use Yandex Disk, Dropbox, and many others, there are many providers. Search was used, for example, using the bash command `tree` you could quickly get key files and directories, especially directories. Using the command, you can search, including advanced search, both by content and by binary files, and using certain shell tools, perform, including fuzzy search. And, by the way, semantic search can also be improved here. It's just that it's not needed in this case. It works quite well, just as code code works without embeddings in its code interpreter. And certain tooling was built around reading, because it's one thing to read text files, yes, markdown, and, for example, txt files, transcripts, and quite another thing to read PDFs, images, and so on. PDFs and images are built into the code interpreter itself, it reads them perfectly, with certain limitations, for example, PDFs up to 30 MB only. But other formats are read perfectly by tools like pandoc, or custom scripts. Some of these scripts were specifically adapted for their specific reports. So we know the structure of one Excel file or another, and we simply speed up the search and reading of such a file. So the tool is adapted to their specific knowledge base. And on top of all this, agent SDK, virtualization through, so sandboxing through Docker, and SDK, I will also talk about it. So, in this case, what advantage did it provide? No re-indexing. This was one of the biggest pains for this client. And, in fact, we implemented similar solutions for other clients as well. Every time your approach to RAG changes, you have to change, you found out that the chunk should be of a different size. You re-index, you found out that another embedding model works better. Again, re-indexing. You just added the idea of summarization, re-indexing is normal, only this is data engineering, work that not everyone can afford, let's say, and well, it's just resource-intensive. The answers are quite accurate, because if the agent doubts its cycle, it will double-check. And if we think it doesn't doubt enough, we will force it through prompting, through skills, and so on, to check its work better. The agent sees the entire context of the document, very often it loads it entirely during the process. And the domain expert iterates himself, adding skills, SDK calls, engineering, and so on. It's funny, this was probably one of the interesting experiences when we worked directly with the domain expert in interactive sessions, so we launched, watched what was not working, immediately changed the settings of the coding agent, for example, the system prompt of code code or a command or a skill, and so on, and immediately saw the results. So in such an hour-long, two-hour session, the performance could be significantly increased. Because the system itself is higher-level, so you focus on instructions and domain knowledge, rather than on some engineering gears that need to be reassembled every time in more rigid workflows. Agents need an environment, yes, as we know, they need a runtime to execute code. The execution environment is

Certainly, here's the translation of the provided Russian text into English, following your rules:

This is, well, it's like a new structure for your infrastructure, relatively new, but you need to constantly think about it, because if AI writes code, then how it executes it and where it executes it, this is a very important question. First and foremost, a security question. And specifically in this case, the architecture, this is, by the way, the file structure, it was difficult to extract something super specific. I tried to extract as much as possible without violating the NDA. This is a file structure, where you can see that we have, you know, the worker itself, which is mounted into the container, and there is certain code that launches this, including MCP and including the task runner, yes. The logical diagram looks like this. We have the external world, we have Airsyn synchronization, outside of Docker we get a folder that is always up-to-date. Synchronization works there regularly via cron, yes, and we only mount what is needed into Docker, with the correct permissions. For example, in this particular use case, we were not allowed to change, meaning file changes were not required, so we immediately gave it access to this folder, because if, well, it's simply a policy of minimal permissions. And inside the task manager, which launches, in fact, agent SDK instances. And it is allowed to write its own scripts, to optimize. Very often, you can see in the process some request that requires data aggregation. For example, gather information about all marketing campaigns from the last year and do some specific analytics, right? So, RAG, well, it's unlikely to be able to do that. Code is directed on the fly. It simply writes a script, launches Pandas, rechecks its work, and returns some metric that it was asked for. All of this happens in the background. It can even reuse this script later. I'm not even mentioning that through skills and through the same scripting, but a more deterministic one, when we have pre-written necessary scripts, more optimal execution speed and greater reliability can be achieved. So, yes, why did I show this here? Because isolation is a super important concept. Do not just run agents on your local machine that execute code like you do, and always think about security. And here, there is even Network Isolation, meaning it is not allowed, it is allowed to visit certain domains, for example, to download certain dependencies for itself and interact with, in short, there is a whitelist of domains, everything else is prohibited. When does File First approach work? When does it not work? Actually, this is a very conditional table, but I think it will be useful to talk about it. When you have hundreds, thousands of files, actually in that project there were, well, I'm hesitant to say now, because the number of files is constantly growing, well, tens of thousands of files. And it handles it perfectly. Including because most questions relate to a specific client, and it has a specific folder, and the agent will simply not grep in the wrong folders, it will look where it needs to. Files are updated frequently. This is suitable, this is a green fact. This means that you don't need to re-index everything constantly, you need simplicity and speed. There are no resources for this data engineering pipeline, because with good skill and good resources, RAG can yield colossal results. But often, in my experience, teams run into data engineering problems. That is, something works perfectly for them with hundreds of files, but then scales poorly with thousands. That is, they don't understand how to build durable background processes, these workflows, essentially, how to do ETL correctly. This is the domain of data engineering, and it goes beyond the scope, well, it overlaps with engineering, but many lack the qualifications for it. Complex tasks, advanced reasoning, it's a perfect fit, like the example with Pandas and so on. And there is no fixed workflow, flexibility is needed for our case. When it doesn't work, when, well, simply on a scale there is a huge number of files, so I doubt that any, say, support agent, to answer a client's question, when they were creating their, you know, support agent, that they would create a similar system that is in isolation, in a sandbox, for each client, to answer, to do all these extra actions. So this is more and more about internal corporate scenarios. Semantic search is needed, also not a blocker. This can be bypassed in principle, because through the CLI, yes, and not through the CLI, you can do MCP, you have a huge number of ways, including using embeddings in such an agentic approach. When there is critical latency, yes, because it is clear that this optimized, directed graph will, of course, be maximally optimized for speed. Critical token cost is also not very suitable. A lot of users, as I said, it's hard to scale. Although a lot of users is a relative concept. For some, even 10,000 users is not a problem if there is proper orchestration. Well, and there is a clear workflow, that is, we have cases where we simply have a directed graph, we know exactly what should follow what, if there are any conditionals, they are quite transparent, understandable, there is no need. We are not trying to complicate it through this approach, simply because, well, it is more reliable. So. Yes, there is still a little time, I will tell you about MCP. This deviates a bit from the topic of agents, but I think it's interesting because it's related to this case. MCP, when they appeared, yes, and still, local MCP is something completely geeky for most business users. When you do it yourself, like, it's nothing complicated. JSON config and that's it. When you try to explain it to a business user, how to add MCP to their environment, for example, desktop code, it's rocket science for them. That is, well, it's understandable. Open JSON config, paste config strings there, set up environment variables, restart the application, install some dependencies. That is, this is not how a USB for models should work. This is not the UX we are looking for. We made MCP for this case. And it works, it worked, and it works perfectly. Many of you know MCP, and Valera also wrote about it. In essence, it's the correct MCP, as it should be. It's an HTTP-based MCP with OAuth 2 authorization. Firstly, it's secure, and secondly, it's super convenient. We did this through OAuth Kit and Fast API. It's funny, to connect this, we did it at an early stage, and we rewrote the project from TypeScript to Python simply because Python has Fast API and it has the necessary connectors. And in general, Fast API is so good that we even changed the programming language for this project from TypeScript to Python. It wasn't very painful, and it was at an early stage. Yes, no configs. I recorded a short video to show you how it works. So, now it's the beginning. We just add, we click, go to connections. This is on the example of Code, but it works similarly for others. An agent is added, well, in this case, an MCP, and authorization happens through OAuth, and that's it. So, this agent is available in our list of connections, and now we can make requests with this MCP just like with a regular tool. What's here? What advice can I give? OAuth, if you don't have it implemented yet and in principle it's not there. In this case, there was a situation where this project didn't have OAuth. It's very convenient to do it through Workflows and OAuth Kit. There is Keycloak and so on, there are standard solutions. The key here is, of course, the ability, the ability to, well, just a moment, okay, I won't go off-topic, I can't remember the term now. Remembered. DCR dynamic client registration, roughly speaking, OAuth Kit solves it in four lines of config, and it turned out to be so fast and convenient, and we get top-tier security. These are not some tokens that don't update. It's very reliable and very convenient. So I don't see any reason not to use it in similar scenarios. This same MCP can be connected, and that team connects it to CodeC, and to ChatGPT, and so on. So, there's a little time left, I'll tell you about the decision framework. So, so, so, build. We'll skip this. When to build, when to buy. Let's go back to the beginning a bit. And we understand, yes, that we have a huge number of features. There will be more. That's the catch, that we see a clear trend that providers will increasingly add features that are absolutely unique to their platform and, moreover, work maximally cool within their ecosystem. That is, Code + Code will always work better than Code + GLM, for example, or at least in the near future. When to build, when it's your core IP, your competitive advantage. And your unique data, you should build, use this right-hand column of the table and figure out how to do it yourself, how it could be replaced with your own controlled solution, and if necessary, with another solution. Buying is easier when it's an infrastructure task. For this company, for a marketing agency, it's not their main business. That is, knowledge management search, even advanced, even AI knowledge management and some AI scenarios are their secondary business. They are not going to resell it as SaaS, it's their corporate automation. And plus, many things are better bought, because you can't build such a search yourself as Google's, no matter how much you want. Or sandboxing now, I think it's cheaper to buy than, although this is also a question, in fact, it depends on non-functional requirements. Okay, I've covered this. So, I'm already on a tight schedule, so I'll wrap up and hand over the word to Kolya. Yes, Arif, thank you very much. As always, it's interesting to listen to you. Let's now throw in some questions. Some that I saw, some that came to my mind, and some that were already asked. The first question is, how much does a solution on Cloud Engine SDK cost in terms of tokens? Because Anthropic is quite an expensive pleasure. I can say that for this client, it was in the range of 10 cents to 70 cents per question. Even though some questions that were around 70 cents, the agent dug through them for a very long time. It wrote a script, it ran it, it evaluated and checked it. So, it could work for 10 minutes, and that's normal for such a task. And specifically for this client, characteristically, their hundred-dollar subscription worked. It was enough for all their tasks and is enough. So they don't spend these tokens via API. Damn. I didn't have access to their RAG expenses, so to speak, but he complained that it was also not cheap. Tokens there were also not free. It's not cool that it falls into the B2C subscription for business. That's cool, of course. Okay. A question about the File First approach. How problematic is it when there are multiple languages? I mean, the language of the documents, I assume? Yes, I think that the language of the documents and when the language of the document differs from the language the user is working in, is not a problem at all. Well, I assume there are certain nuances, of course, in this particular case, it was Dutch, which is very similar to English in its kinship, if they are from the same subgroup, it seems, and they constantly switch between English and Dutch. And there's no such problem. I think the problem with languages can only be if it's an unpopular language. Russian definitely falls into popular languages, because as far as I remember from statistics, the top looks like this: English is first, then German and Russian share second and third place, it seems so. So Russian is definitely in the top three most common languages on the internet by content. Listen, can you explain to me on your fingers how this works in the case of Russian, if, for example, our documents are in English, and the user asks a question in Russian? And also, there are documents in Russian? How does the agent understand what to grep there? Should it grep Russian words or English words? The beauty and convenience of this approach is that you have levers of influence on it. That is, you can prompt it in such a way that, for example, it considers the folder prefix. Or you can tell it in advance, that is, consider how to query, or you can even ask it to query in both languages immediately. That is, we tuned this approach to querying, how it made requests, precisely through prompting and checked it immediately. That is, you see the result, you have a very fast feedback loop. You give it an instruction, see how it works, draw conclusions, and either skip benchmarks. And often, the main thing is to make it understand in what language, that is, your preferences and what languages the knowledge base has. Agreed. I'm actually also using similar approaches on a couple of client projects right now. Sometimes custom agents, sometimes Cloud Code. And it's surprising how easily changes are made in the system. You can completely change the system's flow with just a couple of sentences. Yes, it won't be rigidly fixed, but it will still work stably because the agent always has this recovery system if something goes wrong. And here's another question. Someone wrote that bash is trash. The chances of the agent getting stuck are 99%. Can you explain how this works, why the agent doesn't get stuck, and if it does get stuck, how it recovers? Well, here, of course, there are some life hacks, maybe how to make the agent's work more stable? I can say that I haven't encountered this. And it would be better to hear the arguments of the one for whom bash is trash. And I don't understand how it can be trash. If it works perfectly in coding, and even in such scenarios, if it gets stuck, again, Code has a timeout instruction. If it hangs, it will try another approach. And all of this is finely tuned through cookies, if this is a regular problem for you. We had a problem with MCP timeout. That's real trash. That's a separate issue, how we made MCP think longer than the MCP client itself makes it think. For example, Code has 30 seconds, and that's an unchangeable value. So we had to resort to trickery to make it work longer. But with bash, there are no problems in practice. At least, we haven't encountered them. Okay. I also had a question about isolation. Actually, I see a lot of misunderstanding about how to do sandboxing in general. So, maybe you can share some life hacks here too. I saw you mentioned Docker, but maybe there's a comparison of Docker with other approaches. And why Docker was chosen, maybe some heuristics on how and in which cases to choose something other than Docker. Yes, isolation is indeed an interesting point, because the range of solutions is quite large, meaning you can get very involved, there is more native isolation, a finer part, for example, the same Bubble Wrap in Linux. There are specialized solutions like Daytona, E2B, and so on. There is the same Docker, which in this case we chose for its simplicity and better developer experience, because it was convenient to enter the container at the right moment, see what was happening there, and there was no need to spin up a new container for every little thing. This is one company, they have a common container that works on different tasks, they don't have billions of parallel requests. So there's no such problem. So I would choose this heuristic. If you highlight, if you have a need for, first, ask the question: is it a monolithic or microservice architecture? That is, if you, as here, for example, one agency and everything revolves around it, then it's one solution. So here Docker fits perfectly, and even quite possibly, it's one that doesn't shut down. If you have a huge number of clients, that is, some multi-tenant project, and you need to start and stop a huge number of containers constantly, then there are a huge number of solutions from Code Flare, from Daytona, and so on, they provide sandboxing as a service, meaning they provide a high-level API, and they allow you to launch these sandboxes on demand as much as needed. They execute, they launch, they shut down, and so on. Regarding sandboxing, I'll also say that Docker alone, meaning there are a huge number of tools. That is, you have a large number of network isolation options, how to mount read-only volumes or how to work with temporary packs, yes. Docker provides a huge number of tools. And I'll also say that there's a cool technique, it's proxying network requests. That is, you can route all your requests through a proxy, for example, Nginx proxy. And you, that is, for the agent, yes, and inside the container, it won't even be obvious that there is some environment variable being substituted. That is, it will be a normal request, and you intercept this request from the outside and inject, for example, an environment variable there. This is a very cool technique, we sometimes use it. It's a bit more involved to implement, but it's top-notch for security. That is, in principle, even if Code enters another folder or if it has no environment variables, it thinks it's making a normal request without a header, and the header is added outside the container. This is a very powerful technique. Damn, I highly recommend everyone who is concerned about security and plans to use something similar to re-listen to this part. I really like what you said. Let's ask the last question and move on to Valera. I'll combine several into one. The first was about how testing is set up. Were there any question-answer pairs, or was it an LM as the judge, how is such a system tested in general when it's not a classic RAG, but something more flexible and agentic? And also, this prompt engineering, right, which is essentially reviewing the result, was it manual or was it already an agentic evolutionary self-improvement theme, where the prompt changes based on some logs, as in the recent ERC3? With reflection. Yes, yes, yes, I understand. So, in order. The first question about tests and benchmarks. We made a very simple benchmark at the first stage, it was essentially with checkboxes, boolean ones. Found, not found. That is, we asked a specific question, and the client came up with 50 questions, and we checked with regex if what he was looking for was found. They were all meticulous questions, meaning non-obvious ones. And most of them, well, these types of questions failed in RAG one way or another. At least, due to output limitations. This is a fairly primitive approach. We have now connected, well, we connected almost immediately, rating and collect logs to make the next round and benchmark based on the real data that their agency uses, that is, real data. In general, benchmarking and evaluating agent systems is very complicated, it's more complex, many moving parts. Evaluating it by internal parts doesn't make much sense, because that's the point, you give the agent freedom. Externally, well, there are three directions, traditionally. deterministic, evaluation, as a judge, and human as a judge. We used deterministic and we, well, RAG-based, and we used human as a judge, meaning the agency employees themselves actively tested it and said where something was wrong. In general, as an evaluation framework for LLMs as judges, I am very skeptical about it. This is a topic for a separate discussion, probably, but in short, it's a separate topic. Second, about prompts. The first version of the prompt was written by Code based on input, including call transcripts, and having studied it without access to the dataset on which we tested, but with instructions on what scenarios might occur. Then we iterated together with the client in such interactive sessions. Great. Thank you, Rifat. The questions were very interesting, the answers were interesting. Let's move on to Valera. Thank you all. Bye. Bye-bye. Valera. Yes, can you hear me? Yes. Okay, I already introduced you and Rifath a bit at the beginning, but now a few more words. If Rifath spoke mainly about proprietary technologies and cloud solutions, then Valera will now talk about the opposite, about corporations that will never transfer data there, and about the specific cases he had and what he achieved. He had many cases. Valera, begin. Yes. Hello everyone. Let me share the presentation I prepared last time, but I probably wanted to. It's slightly updated. Just a second. So, let me briefly explain what we will talk about. As they say, I will have a monologue with myself and with you. I want to touch upon the topic of local agents. Rifath explained the cloud brilliantly. In reality, I'll tell you, I really liked a recent meme. Cloud Code SDK support was added to Ollama. And those who tried some quantized local models will never launch Ollama again and won't connect it to Cloud Code, just to, in principle, get a feel for it. And here, of course, there's this trade-off between using the cloud and getting maximum benefit from models right now. And using local, yes, locally available models, downloaded, yes, from Hugging Face, under open licenses, and trying to adapt them to our business. Accordingly, this will be even more, I would say, technically challenging now. I will try to explain in very simple terms, focusing more on the business side. There will be many complex diagrams, and I will stop at each one in more detail. I will have more time. We will go through a small historical overview. You know, like, let's go back a bit in history. And accordingly, in principle, let's look at some solutions that are currently in my head, as, as Sasha Abramov, Head of AI Principal Engineer, told me. Let's introduce ourselves again. My name is Valera, I am Head of Fire Metro. Currently, I have launched a significant number of different services, interacted with various industries in Russia. Specifically, companies that have their own products, goods, or in principle sell their solutions to a large number of clients for a long time in IT. Last year, I started developing an agent framework. I'll talk about that too. A brief historical correction. No, for 10 minutes. And you know what the question is? When I was sitting at this presentation again, I had this in my head: are there really agent solutions, yes, for any frameworks for local agents, as a core, because many like to talk about frameworks, about end-to-end solutions, about platforms. In the enterprise, everything is either a pshik or a platform. New platform, next platform, updated platform. Accordingly, I want to understand what needs to be known, because FOMO is catching up, and we are constantly missing something. A huge amount of news comes out daily, and you can't escape it all. There's always some cloud of tags in your head. Hardware, security, LLMs, yes, tel, sandboxes, MCP, volumes, everything Rifath talked about. But now imagine that we don't have a responsible party, yes, well, where a hypothetical provider has closed all of this from us, and, as they say, we've become vendor-locked into an API, and we've stopped developing ourselves, and in principle, we say: "Well, great, the MCP will spin itself, the depressch will work, and internet search, and I'm now sending my history there." Yes, I don't need to carry these huge arrays of messages myself and come up with something with them. And something with RAG has already been invented, and documents are unpacked themselves. Everything is great. But that would be a spoonful of tar, it wouldn't be as cool as if we understood that all of this would be available to us. In the local business, we, first of all, try to protect our clients, to do everything according to certain norms, standards, laws, yes, so that it's all internal. Plus, yes, having, well, experience interacting, for example, with banks or with more serious organizations, then

It will not be possible to take and create an agent on some Open ASDK AG, yes, to pass tools to it. And then we just ran in general somewhere into the cloud, ah, yes, to send Haiku requests. This will not work. Therefore, ah, I want to discuss trends with you a little now, right? So, we discussed the cloud, that there are such thoughts, how to build local agents of the core or platform in 2026 and what for this, well, basically, I would like everyone to go through, and maybe, somewhere fleetingly, in principle, to have heard, and it would layer onto your experience, and you would gradually become more pumped guys. Ah, this is still a presentation, probably, I would say, very jealous. At the moment, everything is changing rapidly. From Open AI somewhere in twenty-second year, when they talked about their experience of building RAG, and how much needs to be applied in aggregate, not separately, right, in order to achieve some high accuracy metrics on various measurement systems. Here you can see different approaches separately, there are potential documents, someone is definitely familiar with expansion, when we expand the user query, when we experiment with quantization, a sliding window, there we change vector models, we do some experiments, ah, yes, so we vectorize something, we make top queries, there, we somehow experiment with this, we install and launch re-ranking models. All this is done step by step, well, in the logic of things, it works with classic RAG, ah, it increases accuracy. But at the same time, yes, of course, as was also said, yes, it entails, probably, a huge amount of code, a codebase. And, uh, at first, when there were no ready-made frameworks, especially open source, for example, such as, RCKflow, Defy, Mastra, everything that Openor now, in principle, covers more or less out of the box. There are, of course, difficulties, challenges with adaptation, but now, for the current year, I would say that most of these things are automatically covered by open source frameworks. But there is one caveat, ah, probably from the point of view that if we work in a large organization and pull a framework, as soon as we move from experiments to the production-ready stage, then we will encounter such a thing as security. And, well, for some reason, it turns out to be a secret that any framework that is brought into a large, quite serious company must undergo, well, at least, there are checks, there are SAST, vulnerabilities, right, well, some such basic things. Ah, but given how quickly everything is developing, security, right, there is usually such a tab on GitHub for some solutions, it is probably present in ten percent of open source frameworks that truly observe and monitor some security in general, for multitenancy, for distribution, for ensuring that it is at least, you know, enterprise-ready. And such a thing, well, in most cases, is unavailable. Either Open Source Work makes money on this. That is, you get some, ah, you know, a dopamine hit from the fact that I will now take a lot of open source, apply it here locally, close all this, I will not engineer it. In my opinion, this is mistaken from one point of view, and from another, well, as it were, you can experiment with this in principle, but as soon as you start to reach a truly large scale, you will have to redo and refine a lot. Well, if we consider, right, classic RAG schemes, they really move away from architectural problems, problems with code and security to how much needs to be done, because, well, it is not available to us, it is not available to us. We have, what we have now is GPTOS and GLM recently came out, by the way, updated. Thanks to the guys. A thirty-billion model. Tell me there, how do you test it there with a board. I haven't gotten there yet, I'm still on GPTOS on queens. But, given the capabilities of these models, we can build such pipelines. I will not dwell on this, this is classic RAG, it is, in principle, naive. There are minor improvements. As well as such RAG, right, in which there is all the same. There is no React, it is not an agent. Everything here is on workflows. Moreover, workflows in code, not even a graph and not even some framework. It's just, in principle, in code, separate things: guard rails, planning, query expansion, document embeddings, some kind of router, hybrid search, right, there is a check, whether clarification is needed, launching another cycle. That is, all this is just a workflow that is advanced enough to run for about a minute on a local model and try to extract slightly more accurate data and work, for example, with multifab queries, when in one query we don't just ask how to get an office pass, but a whole bunch of possible search queries. And all this, in my opinion, in a corporate environment, is perceived by engineers as a tool. Now it's already some tool in MCP, behind which the entire architecture is hidden, either built on an open source framework, which runs somewhere separately isolated from the general environment, or it's a self-developed story, chosen from somewhere like Langchain, other frameworks and integrated into the system, ah, of the corporate environment, into the core, having passed all checks and its validity, right, when we say that we want to move away from classic folds to agents, then here we can consider different cycle management systems, systems of management, namely planning processes. But if we break it down, we recently did a stream with the guys, again on the sensational enterprise challenge, namely the agent one, the third one, it turned out that the simplest agent, right, on Open ASDK five lines of code, ah, with a slight overfitting, it has a distribution, right, for tasks, it takes first place, simply by taking a powerful model. Does this not seem to you or me that we often create some overhead or use too much, you know, this hardware and this onboarding engineer magic. That is, maybe everything should really be solved much simpler. We fit prompts for tasks, do a classic react loop, which has been around since twenty-second year, and let's go. Yes, we don't do anything extra here. But this is not the case. In reality, a lot of things have been invented that allow us to build more deterministic systems. After all, React is all about the fact that we win quickly, get good, good, acceptable quality of some solutions, even on local models, but it's not about 95, it's not about 98, right, 98% accuracy. And here I would like to draw attention also to the history, to the history of such a pleasant solution that makes LLMs return JSON to us. But if you are still prompting an LLM to return JSON, that's good, of course. In principle, cloud ones withstand all this and even sometimes don't sneeze and don't break on quotes and apostrophes. Well, and if we talk about the fact that we use local frameworks like VLM, for example, SGL, then specialized systems have long been invented for these frameworks. Here is a small history of how it developed. The structure was developed, and then function calling was added. And now, indeed, using function calling or structured output, we get very good working JSON objects that we can then use in our business logic. And this was preceded by really a lot of historical work by the Open Source Community, which developed systems like XGrammar. In general, in principle, I would say, systems that support grammar control within frameworks. That is, roughly speaking, when the model generates the next token, we don't just generate it and give it back to the user in streaming mode. We additionally check it, whether it corresponds to the declared scheme. And such an approach, in principle, generates a deterministic part of all this huge number of possibilities that you wanted to have. Ah, that is, function calling worked unstably at first, and this was, in principle, a long time ago, right, the historical reference will still continue, it's not for 2 minutes. In 2024, Open achieved indeed there 100% accuracy and somewhere, right, in twenty-fourth-twenty-fifth, they reached parity. That's all. And local models can generate JSON accurately. I didn't say without hallucinations. And, accordingly, we are now approaching the point where the agent core is indeed a sufficiently, ah, a combined path, where earlier it was necessary to write, probably, this much code. That is, now these branches are either collected into ready-made frameworks, or we have long passed this over the past years of engineering and understand, right, how to build RAG, how to connect MCP, how to write it on Fast MCP, which entic DK to choose. But I haven't written percentages here. If you decide to build an agent platform in 2026, then, in my opinion, approximately such a percentage distribution of employment should be. Moreover, I specifically did not set priorities here. What to do first is up to you to decide, what to do? Sandboxes, evaluations, choosing an SDK that will manage all agents, choosing some agent loop, which will be a graph-based React. Well, you can read about the competition between Openi and Langgraph, right? Well, in principle, you choose yourself what to build a bicycle from, and what will be your standard for LLM, how you will write MCP, declare for your different product directions or individual teams, and how it will work together. Accordingly, if we talk about choosing LLMs, then, probably, yes, there, now, as it were, I would say, six months ago, we still came and helped select models for different companies, that is, in principle, we created, ah, yes, internal evaluations, right, so we tested somehow, then now, probably, some such, well, as it were, the most popular leaders have emerged, which are quite broad, that is, roughly speaking, we constantly experimented at the beginning of twenty-fifth year. What will we do, how, what benchmark will we collect, how to accurately check that the model will respond well, then now we have an absolute leader in open source, right, it's QuT, there's still 4, but I would say now that GM, Gomka, right, and in principle, also fall into this category, judging by benchmarks, not yet by popular recognition, but Reptios is not for nothing in first place in various agent approaches. I confirm my practice here, probably, you know, not practice, but my observation of what people write, I confirm by what I saw that a sufficiently large team uses GPT 120B in a very huge project. And it is precisely it that is the core, right, of this system. And here I would say that we experimented with it at the start. And here it can be said that the model initially did not show good results due to problems with harmony. I myself tried to patch Volm, make some settings, in general, I returned to stable working Quen, so that everything simply worked. I also conducted various experiments with Gigachat. At the moment, ah, yes, so I will still experiment with GM, because I want to see how the thirty-billion model works and what it can produce. But at the moment, there is such a parity, I would say. We have several open source leaders. We are waiting for Anthropic to release something open source, I hope. Oh, I'm keeping my fingers crossed that they will also break into this competitive story with some small open source model. But if we talk about model sizes, right, that is, we have chosen a model, some, well, groups of models, then at the current moment I would distribute it like this, that for the agent core, you can use OS or KN for coding, again, according to insights, according to communication with different companies that are trying to pull it in and would like to make a remark, this is not that we place the cursor and connect it to Openrouter. No, this is us, ah, we will download Open Hands or, ah, install, write our own plugin for VS Code and install a model that can tap, do some React logic and write code. So this is approximately this range, that is, Quen 235B or Qen 480B, depending on the available VRAM, will be able to cope at the level, at the level with Anet fourth with coding tasks. that at the moment, probably, right, you see marketing materials that smaller models can handle large projects. Well, this is fundamentally not true. In reality, we simply have a situation where everyone divides differently what coding tasks are for them. For me, a coding task is when I can have four simultaneous terminals locally, right, and develop my local, ah, in parallel, four tasks, where the codebase has already grown to 3,000, right, lines. And there is a huge number of microservices and infrastructure. And someone considers a local coding agent to be one that can suggest a function. Well, why not, right? And this, in principle, works in business. It works in practice. Rena Renat touched on the topic of sandboxing, but here a very cool article came out on January 5th. This can be considered an update. About how to choose a real sandbox, especially in a local environment, and how to do it safely. Containerization, of course, is good, but if we talk about security, then a container has more or less the same core as the host system. Therefore, we return to the history of some years ago, when KVM appeared, full virtualization, right, from the kernel to the devices, which provides a higher degree of isolation. And now the top is microVM, for example, Firecracker, where a higher degree of isolation can be used. This will be very safe. By the way, VS uses this microVM approach. If you have used BDR Agent DK, as it turned out, for sandboxing, microVMs work, which start up in a few thousandths of a second. This is very lightweight virtualization that allows complete isolation of the kernel from potential injections, potential problems with, you know, the joke was with empty documents. The model read an empty document, a script was injected into it, it ran, and accordingly, all documents from the sandbox flew to, yes, there, to the attacker's servers. This worked, I think, at Anthropic for some time until they fixed some network issues. But this, of course, also relates to security. We want, after all, that the code agents and the core of our platform are truly secure. And we want to believe that we are indeed building it securely enough. Therefore, microVM, in my opinion, I haven't touched it myself yet, but I have studied this article very closely. There is a QR code here. Ah, yes, on the presentation, it is visible, you can read it. Very useful reading, a huge article from the developers or a developer from Mistral. If we have chosen an LLM, right, we have talked about sizes, we have talked about sandboxing, specifically local, right, that is, how it should look safe and what can be done with it, then I would like to note, you know, a certain phenomenon. It still doesn't stop in my head in terms of how it could be done. And I will, again, let's say, for 200 people plus, I think they will watch later, tell you that if you plan to build an agent platform this year and in the first streams there is not this system, approximately, approximately, for creating isolated benchmarks, score verification, task-level isolation, creation of API method emulation, so that you can run different architectures of different agents, different approaches, in general, in a local environment for your product, then you have probably gone somewhere wrong or you are being pushed too hard by deadlines, because at the moment, I think this is a precedent, and few have paid attention to it and in general, the recipe has passed and everyone has forgotten. And I consider this to be the second, you know, like Kaggle for ML models. Here is the example platform. I hope it will find the strength to improve it and make it available for uploading its benchmarks and using it for good purposes in general. But pay attention to the Ger platform in terms of how it is organized, how isolation is done, how sessions are done, what tasks there are. This is not about it being a golden bullet, I don't know, choosing an architecture. It's about the fact that by using such a system, you can potentially control the development of your agent or agent environment or agent in general, right, on, well, thousands and millions of runs. You have local infrastructure. Why not run an agent on a queen a thousand times, which will also rewrite its own prompt in isolation with some mock data, and the prompt will be improved, for example, by Anthropic, right, because we don't have anything from the corporate environment there. And then we will launch it in isolation on a local environment, on local models, already in our enterprise contour. And if we talk about hardware, then once again I want to remind you, right, we have, ah, yes, there is a sufficiently large, wide selection of graphics cards now. And I often see that consumer inference is indeed used, and this is a normal situation, because it is difficult to order, right, ah, ah, H-series, A-series specifically at your location. They are now even, in principle, occupied in clouds everywhere. That is, if we go now to rent a large number of A-series for a long term, more than six pieces, then you will not find them simultaneously, and it will be quite difficult. You will have to go through clouds. Sandboxes, microVMs, agent skills, react loop, tools, MCP, right, and, for example, well, my still top for inference is VM and EV platform. A small set. I just simulated all this through a cursor, made a small table. But in general, it reflects, right, now just a small summary, what we have talked about. Ah, well, how to solve the problem of building local agents, right, this is also a separate story. I specifically left a teaser there, Open Code. Ah, yes, there is also, in principle, ah, a solution, right, about codex, this is about some, well, open-source ones. Now, and probably, Nchain, Defy, Master, and Q can also be attributed there, so that, in principle, something can be built. And the most, in my opinion, problematic story, which I resisted, is that it is very difficult to get a team not to build complex bicycles on, say, Langgraph, Langchain. And coming in two months to launch some direction, you realize that the agent is this much codebase, the guys are on version 10 versions behind Langchain and in general, ah, yes, they can't even read normal logs, and they forgot to connect fuses at all. And here the thought came to mind or the idea, right, that the creations that were made with Iganko Core, which is now a framework, we released it on PyPI, it has Docker containerization, and now there will be a ready-made Docker image. Ah, yes, to assemble such a solution and distribute it in the company as a standard, well, again, not mandatory, turned out to be a somewhat winning situation from the point of view of support and from the point of view that, I gathered a small group of people who are engaged in the development of this agent framework. We ourselves built quite good quality agents on them, ran them through benchmarks, understood that they really work on the models we chose. We packaged this framework and started, in principle, to develop it further. Now we have a small core team inside that works well with our local solution, with our framework. And also, of course, if you can also build cool, right, there, within us on OpenAISDK or on other frameworks, it's generally welcome, but the essence was precisely that we can easily now transfer our expertise, quickly come, understand, there is no, a dependent codebase that is huge, which pulls, ah, the open source community that has accelerated to light speeds in commits. We have brought our framework to the minimum, possible, necessary functionality and continue to develop it. This, probably, was a distinguishing story. Well, plus we released it into open source. In fact, most companies did the same, but some forked chain, while remaining on some old versions, which are now being rewritten from scratch. This is, as it were, the thing that I didn't want to do. And we released it, right, gave access and configured it well for precisely the local models with which we work and which we deploy ourselves and which we deploy to clients. Accordingly, there are already more stars, almost a thousand, but this was a conscious, you know, choice of what to make a lightweight framework, right, not on Vipecoin, it was on Vipecoin at the start, now it's being done by engineers. And in principle, from some point, we started connecting, say, the cloud for writing some new features, like search by tool, right, for example, a more universal context system, and so on. But this is still our framework, we manage the backlog ourselves and know when we have a new release. And we distribute this release, you know, like a virus within the company, what's new we've brought. Yes, and we don't have to constantly chase open source frameworks in terms of updates, right, studying what has happened, what has broken, waiting for contributors, waiting for PRs, we manage it ourselves. And if we talk about the distinguishing features, right, of the framework, here it is simply, in principle, we applied a sufficiently complex approach with a reasoning tool, and for non-reasoning models with an instruction, this thing works. If you, right, plan to build a solution based on a graph framework with a reasoning model or a cloud model, you know, it will definitely work worse, because we refined it for small open source models and instructions. Accordingly, how such good control is achieved in our framework for us. We have made, right, several approaches with iterative planning, but without additional tools. That is, the agent plans within the tool call. This is the most distinguishing story that, in principle, right, out of the box, I don't know, well, it stands out from other frameworks. We have well-developed planning within the framework with small models. We have tested it, benchmarked it, checked it with a lot of guys from the community, confirmed the hypothesis, use it everywhere as an opener and a starting set of functionality. Our own agent framework, local, small, tested by all security service articles, which is compiled for, it has a PHP version, and that's enough for us. It turned out to be successful. Such a thing can be applied to quickly deploy, right, simple prototypes and bring them to the next stages of development through testing and evaluation and connection, for example, to different types of interfaces. Where we have already built this SG Core and with whom, there have probably been different cases. I will say this, that we first built them into our products. That is, all the products that we have now, right, in robotics, and which I oversee, agents work under the hood on this framework. This also adds value to the company and value to the products, which, in principle, well, as I would say, you know, it smells like marketing again, but it's marketing, and it works well. And this is the thing that I would like to share with you. Indeed, to use not what, right, is now at the top and open source, well, everyone can do that. But it was interesting, could we create our own framework and distribute it to clients, would it even work. It turned out that a Russian company, right, guys like me and our experts can do such a thing and sell it excellently to construction companies, sell it to large banks and truly integrate these solutions. And they also, most importantly, right, work. If we want to delve a little deeper into the details, then indeed there are RAG and concepts. I keep coming back to it, we talked about agents, but locally there are some problems, probably, when you want to build it from scratch again. And Refan, I remember, said to choose a file-first approach or RAG. I am currently precisely unable to answer what is really worth choosing at the start, when you have nothing. That is, you have neither RAG, nor an agent platform. Because to do Fire-First, you will still have to bring the agent core into your company and build this system with sandboxing, tooling, and so on on it. That is, if we are talking about some quick prototype solutions that are not scalable, of course, here you can download some ready-made solution from the point of view of Open DCK or other, right, agent frameworks and locally on your PC to show as proof, right, as a presentation, it's okay on local hardware. If we are talking about scaling your capabilities, then last time I said that, yes, you start with a first approach and indeed it will win you, probably, time quickly, it works well in the cloud. But if we are talking about, right, when we build RAG or when we build a first approach in a company, there are measurable things for labor costs, then if we want to make it so that a huge number of people can use this thing and it is protected, it is safe,

So, in principle, yes, I would start with RAG, but in the process, I would integrate the agent core at all levels of the platform or at all levels of the product, and immediately hire upon reaching some parity. With infrastructure for boxing and security, already a file-first approach, yes, in order to reduce the load and decrease database duplication. And if, you know, how to switch to what to ride on, right? So we talked about what our bicycle machine in general consists of, right? So, a different set of some engineering solutions that need to be brought in, made safe, made thoughtful, and, yes, with some trade-offs, then at the current moment I would say that we have, well, two leaders for sure, right, and many, I know, are refining them and definitely making some updates from the logic perspective. So someone patches VLM for specific settings, someone makes some changes in SG N that they need, for example, with specific data formats or their own fine-tuning models. But now, at the current moment, probably the most stable solution, and stable means at least four versions ago. And something is definitely written, tweaked with a file, right, with Belming - this is somewhere a few versions back, because they have a bit more unstable versions lately. Well, perhaps all this will be resolved, right, when the Linux Foundation takes over a bit better, it, by the way, was also transferred there, just like MCP was absolutely recently transferred to CPP, in my opinion, excellent, friends, install it on your laptop if you have, for example, a Mac on M1 with a large amount of memory or at home you have access to a graphics card with 16 GB of video memory and Windows. We download ready-made, right, images or KD or applications, download models and test some fast inference, right, of what the model can generally answer. But in my opinion, any model conversion from the initial format still carries some losses. Although the metrics say otherwise, my personal feelings are that when we are dealing with FTN with a model in its initial resolution, right, FP16, it is much more reliable in terms of long-term prospects for eliminating errors in production. If we are talking about a model, then at the current moment there are either ready-made frameworks that cover all this, or if you do not use ready-made frameworks, you can take a separate solution. So, we take, for example, the top of some formats that are used in our company. And we are not talking about pictures here. Pictures, scans are a separate story. Definitely, if they were in the company and you were involved in their recognition or, yes, locally, then there have long been pipelines that have been worked out. It's just that now OCR pipelines can be improved with, I would say, micro VL models, small VL models, so that, in principle, the OCR results can be sent to an L model and, for example, yes, somehow improve them. I've seen such pipelines, it works well. But when we are now talking about high-precision automated document recognition without human presence and additional verification, well, VL models still cannot cope with this with sufficiently high accuracy, yes, they create a sensation in terms of the fact that you can throw 5-10 documents into them for testing and say, wow, it recognizes, but then you start to figure out that somewhere in a long guest it confused numbers and so on and so forth. There will be a lot of this. Therefore, at the current moment, for a profitable local solution for multimodal recognition, yes, of documents, I would say that, well, loudly no, but it is being polished with a file by each company in terms of its requirements. Well, if we are talking about the classic format, like PDF, where we have tables, right, where we have text, well, some presentations, where there is also text, where it can be extracted, Excel files, right, and so on, then there are already long-developed favorites, you can, in principle, pay attention to them, or combine them into some, you know, a pipeline of fullbacks one after another, or choose one. For example, when I watched a presentation from Justy a very long time ago, the guys chose Doclik as their favorite in their teleplatform. When I watched what the guys, I think, were talking about somewhere at the Pedation of SNCOs, that they took, they took markers, they refined the marker [ __ ]. So, in principle, yes, I took the top that I heard about. If we are talking about MCP in local solutions, then let's also switch to the topic that, again, if I return to the previous speaker, right, Refat said such a thing that we had a repair in MCP, and it, in principle, clarified many points, many points on how it should look. And probably, I will say this, the main problem that existed, most fast, high-quality software that has all the functionality, is written in Python. And for AI, right, if we take Fast MCP, the fastest updating repository is fast MCP in Python. And if your company writes in PHP or writes in, I don't know, Jami, then you have to wait for ports or when their friendly repositories catch up with this functionality. Because, for example, OAU came out 2 months ago in the PON repository, and in PHP it comes out a month or a month and a half later. Either you have to refine it yourself, or wait for the guys to finish everything, based on the standards that were transmitted from above. This is the first thing. The second problem, probably, right, is that there is some misunderstanding of why we need MCP in principle. But I would like to explain here that at the current moment in local solutions, when we take and develop MCP, in my opinion, the only answer you should give yourself is that I am doing it so that a bunch of clients internally or developers who make agents, that is, for example, we have a CRM product within the company or a product, I don't know. Let's say. The HR department has its own API, for example, right, but all other departments want to connect to this API. All other departments develop agents to communicate with these APIs and so that, yes, we know for sure that everything corresponds to a single, well, business logic, that all tools are unified, right, tools are checked, tools have the correct business logic, name, that is, yes, the name, action, brief description, well, the classic naming approach, because if we give the HR department and 10 departments to write agents, then imagine that you will have 10 toolkits, they will all have different naming, they will all have different descriptions, and they will all perform this logic differently. This is where we come to this standardization, that we, in fact, wrap the API method in a single standard for other departments, that is, who will develop their agent based on the HR API. We will know for sure that when we call search employees, right, the same string will be generated for the description and the same tool description will be for all agents that are built within the company. I don't know of any other explanation at the moment, except if you want to offer your product externally to a lot of other clients. In principle, if there is no such need, then you can stick to one file, one folder of TOC, where the tools are written simply in code, where there is a tool declaration at the top, and Execution Choice is below. And here I want to tell a case from the perspective of Remote MCP, how, in principle, right, the procedure looked like in the development of MCP for вкусовал. I can say now, because the case had not yet been released for me. Initially, I made a prototype by parsing the site. You can look on my channel, yes, вкусл is probably the first retailer that released MCP in Open Source. So now you can, in principle, take, in Claude, in Cursor, wherever you want, throw in a URL and their MCs, find the products you need for, I don't know, any dish and make a basket from it, and from the basket, go to checkout, right, it performs only three actions. This is search, this is extraction of additional data, and making a basket. But it was indeed, of course, developed. So we initially did site parsing, packed them into Toolib, then packed them into MCP, after that we developed new Rest API methods, rewrote part of the business logic, and wrote a separate service that, roughly speaking, repacked another internal API. And in about 20-30 minutes, I wrote an example in Python on Claude, handed it over to the development team, they wrote it in PHP. Indeed, in local services, it is very painful that initially now, right, we would like to perceive the MCP protocol as exactly, you know, such a KPI gasket, but this is not always the case. And binary files cannot be put there at the moment. PDFs, pictures, right? All this will still have to be done through workarounds through skills or through direct upload to the server, where, for example, in the resources, there will be an instruction on how to do it. There is a slight cultural resistance always. Well, not always, but in my opinion, in the enterprise environment, it still exists and is spreading that we have made our own solution, it works, we will not try another one. And this, by the way, is not exactly a common problem, these are my observations. When a team is building an agent, it wants to get higher metrics, but it doesn't listen well to suggestions from outside. And here you have to lead by example, literally, this is, you know, my case, where I really don't just, right, tell, show some training materials, but literally come and rebuild the agent, right, on Open SDK or on SGE literally with wipe code in one evening. And for example, in the previous section, four requests from this team cost, right, $0.27 for using a cloud model, for example, for tests. And I, for example, am already using a cheaper model, right, with a much smaller number of tokens. And in principle, all this helps me, right, coding agents. Cultural resistance is that we have made our own solution, we will stroke it, caress it, and adore it. We won't even look at anything else. Therefore, at the current moment, well, this is probably happening everywhere with completely new AI, but this is just my personal example. I don't know, maybe someone resonates, but so far I manage to eliminate it by personal example. And this is a story, as it were, from the perspective of positioning who you want to be. Probably, I've already talked about the results, right? That is, what I use in my work constantly. Well, here, I probably won't repeat myself about it. Yes, so this is such a gentleman's set. Now cloud code has completely replaced cursor. In general, cursor only opens as a visual interface. I can't get away from that, but the terminals at the bottom are open, of course, with scon. Here we will share the presentations again with everyone who came. Newcomers will definitely get links. Here is another summary table of what we have discussed. about local infidс, how to build it, what to know, right, in 2026. And in principle, how I see the agent core, right, in principle, I would say this, my agent core does not differ much from the agent core that Fat talked about, from the perspective of, yes, there, how to do it. Well, here, probably, the history of local models, how to infer it, about, for example, 152 FZ, about, yes, cross-border, about security, code verification by the security service, and much, much more, because it is very important to know and in reality, well, you know, ignorance does not absolve you from the fact that you will have to rewrite your agent Slowain into some real ordinary code that clients or enterprises are currently doing. Thank you. How much did I stay within the time limit, is there still time for questions? I think so. Yes, Valer, go ahead. Since today is not a strict conference with a lot of speakers following each other, I think we can chat a little more. There are interesting questions here. And in general, I wanted to say that I am, of course, super close to this approach of not dragging huge frameworks, but making some small thing for tasks, especially when you know the subject area. Here, as far as I understand, correct me if I'm wrong, the strength of this SGR Core framework is that you have already eaten a dog on typical orders for Russia, and you understand what needs to be brought in and what doesn't, or am I misunderstanding this? That's right. Probably, I would also say that with the nag Core, with wipe code, I manage to show examples of working agents in react loop quite quickly, personally for me, using local models, yes, and closing, showing these use cases, that is, not saying that I will first develop the agent core or rewrite some for a local model, that is, there are already wooden tools described there. Super clear. Mts are very small, the flow is strict, deterministic, with a lot of checks, strict everywhere, the model cannot deviate left or right. That is, there is a constraint. And all this is out of the box. When you take a cloud model, you still don't have constraints or strictness. You try to remove it, because in general it also slows it down. But when we say that we are in a local environment, we want the model to do what you initially intended, a little more deterministically than to invent React. Therefore, at the current moment, the framework from Shared Reasoning has turned into a framework that works well with local models, which, in principle, performs business functions. We have written a lot of different convenient mps for it, and we are continuing to integrate it into business. That is, we have eaten, roughly speaking, dogs. And what else is interesting? We are now going to conduct quite a global training within the company on the use of the framework and, moreover, with passing the store benchmark at least 85% using this framework. This turned out to be a really cool idea. And everyone picked it up, including a competitive spirit within the company. That is, imagine, now 40 people on Idgar will go to torment Renat's platform again. I think that the store benchmark is really one of the coolest cases for training in general and engineering. By the way, in the comments, someone asked what to learn. I posted channels there. But besides channels, I highly recommend trying to solve the store benchmark in C3 in Renat's LLM under the hood channel. Yes, yes, let's go to the questions. Look, you say that this framework works very well with small local models, and you gave two columns, right, very small models of the 4B level and larger models. Can you roughly estimate some tasks for which you can, well, extract solutions from, yes, from, from this framework in conjunction with models, there, 4B and in conjunction with larger models? Well, yes, let me say it broadly, from the perspective of categorization, right? The main task is Deep Research, that is, in principle, well, there is also quite thoughtful tooling for integrations. We are working on it separately, how to compress the context, how to show models, how to, yes, artificially, break the step and transfer it to a new session. There is also such a thing, so that the model does not die on the tenth step. If we take QuB2507 Instruct, then here we can say that the model can quickly perform a small number of queries for Squarey Expansion in some API, for example, Git, if you need to search the database, summarize and give an answer to the next stage of the pipeline. That is, such small models are mainly assistants, as Max would say. That is, it is an assistant. That is, it is not a model that can do write, right, or delete, or create. That is, it is about support, about information search. Here we simply take on the shoulders of, hypothetically, document bending, query expansion, and a couple of other techniques, technologies, right, and approaches from advanced tracking that we have, and simply pack them into the model in a react loop. This is what they can do with this framework. And in fact, it simply closes it quite deterministically. Slightly larger models can do research on the codebase. This is directly proven. For example, you have a React agent Quen 3 30B 2507 A3B, which is a Mac. For reading, analyzing, and quickly writing code, we take a Mac code of the same type. And in conjunction, these two stories with the framework can do fast research quite well, right, on quite large repositories, simply grepping, searching, and, yes, making some, logical conclusions and analysis. Okay. That's about it. Uh-huh. Essentially, yes, 4B is enough for solving ordinary text tasks. That is, you can run it on your laptop. And I will have my own press. But for code, you need larger models. And here there are comments, why didn't you mention that Snimak and Numa recently released a model that is available for deployment. Look, the thing is, everything boils down to the entire infrastructure and how much cache you have available. Almost, few people in business run models with more than, let's say, 30 billion active parameters. This is not suitable either by time, it's not active, right? Because if we look at PTIOZB, there are 5 billion active, and Knatri, right, and all this fits well into the hundredth or H hundredth, right, I think, in the hopper, there is hardware acceleration. This fits well into the Osuna. That is, you don't need to have problems with launching on several, right, with in-linking, if you want to enable some speculative decoding. All this is not there. You have a thirty-billion model, one card with five parallel requests on Vilm. And if we are talking about a four-billion model, then two or three instances fit there. And this is with the maximum available, normal context. But it is not always, well, never used. The limits at which local models currently work in contexts, purely for summarization, are 80,000 tokens. Everything else is simply cut, well, in pieces and summarized with pieces. So here it's all about infrastructure. Yes, let's ask the last question about technical details and then we'll go to the philosophical ones. There's a cool question here. Yes, go ahead. On the technical side, look, you've talked about Asata, about Asota, and I remember you talked a lot about how you set up a cluster of 4,000, I think 90s. Yes, 4090. I can show it right now. I have this cluster open. I thought about showing a small demo on it. Well, these are the kind of graphics cards. Two RTX490s, then throttled. Valer, you can't see it. Your screen is open. One moment, and it will be clearly visible on the desktop. Now it's visible. This is what this story looks like. Three terminals. On the left, VLM is running with Kn 330B 3Brct 2507. On the right, we have a terminal with NVDS SM. And here you can see that it's actually an ordinary civilian computer with 62 GB of RAM. With a small amount of RAM. And I have my developed local desk open. I can, in principle, now run some small depressor using this Kn, and, in principle, choose such a temperature. Let's search for AI news today, especially find AMD's comment. I'm writing it incorrectly on purpose. And I'm sending the request now, and it has gone here. You can see in the logs, right? So here's my system prompt, everything has gone, inference has started, right? That is, the model is already searching for something, extracting, and, in principle, processing. Valer, but the question here is, why 4090 and not something like an RTX Pro 6000, which has 96 GB instead of 48? First of all, the 4090, in terms of its speed, is not much different from Asota in performance. Well, it's even more powerful. Availability. I don't need any industrial server. To be clear. Valer, Valer, I think the question here is about RTX cards, which are not Nvidia at all, as far as I understand. Well, it's expensive. Let's put it this way. No, RTX is Nvidia. RTX A6000, it's called Ada. Which, if I'm not mistaken, costs around 4.5 million rubles with 96 GB of video memory. If I'm not mistaken, if we are talking about such a 4090, now on the market you can buy it, in principle, for 440, well, let's round it to 4,000 dollars, 440,000 rubles, roughly, there, 450. And, accordingly, two such cards plus a server cost the R&D department, right, one server. That is, a deployed local Quen 330B Instruck now costs 1,200 rubles, because it's a civilian server, it's a four-unit rack case into which a civilian motherboard is inserted. An ordinary civilian, as I say, that is, an ordinary motherboard on a 550B socket. An ordinary Ryzen 7 is inserted there, which, well, you understand how much it costs, ordinary DDR4 memory, an ordinary 1000W power supply. And this server, for example, has been working for a very long time, for a very long time, in terms of cards. They were bought, I think, even a year ago. They were serviced once, and they are in the server room. So here, the question is, yes, it continues to search for news. The question here is, of course, much more complex, probably economic. I would say that at the current moment, I have probably managed to assemble the cheapest inference in Russia, based on the purchase price for 30 billion models. Moreover, it is currently deployed in 300,000 tokens of KV cache, which is not available. This means that this model can be overloaded with parallel requests, up to six requests of 80,000 tokens can be sent at once. Quite high performance. For a large number of people, this one server is enough for experiments. Many such servers have been purchased for coders, for VL models, and for other experiments. I don't see the point of putting more than two cards in one server. You see, under load, it consumes no more than 120 W. Therefore, I have talked about local cards many times. Of course, when we talk about higher load, stability, and we have proven the hypothesis, we buy Asota and hundred servers, six or eight pieces, and then we go to prove the cost. From the point of view of ROI. You are in the experimental stage, you don't know what hardware to choose. I think it's a must to buy at least one such server and, in principle, at least one such card and understand what can be deployed on it. How many requests can such a server handle in parallel? Parallel requests, well, look, when the model is loaded, it is loaded in size, you can probably even look, if you do it like this. Kv. Not like that. And if I'm not mistaken, I'll try to find it quickly, if not, then approximately 300,000 tokens are loaded and available. That is, in principle, divide them by requests. Here, now 9.1 request is Running. Simultaneously, you can send it, that is, you have 330 available, and you can divide them into small ones, as many as you want. For example, if your agent consumes 14,000 tokens at startup, well, that's a normal agent. Here, in principle, it is. Well, by the way, you can look at the speed. This is with caching, such a speed. 5,000 tokens input, output is plus or minus, the speed is also visible there, so it's normal, right? This is a lot. That is, the model is not bad. In short, you have access to the maximum context. In general, Kn, it's trained for, I think, 250,000 tokens. So I don't know, there are no such limitations either. And it's currently the most okay. GPT 120B is also deployed quite easily. Well, in general, the numbers are clear. Yes, let's go to the last question. Here's a philosophy. Where is all this going? Let's go, let's go. Will agent developers be needed by the end of 2026, or how long will such a role live? How do you see it? And what can this role be transformed into, if it hasn't already appeared? Listen, well, as they say, someone wrote to me on my channel, saying: "Why did I learn to code?" Yes, well, probably, it's all going towards the fact that, now, a large part of experimental verification code is definitely written and is already being written. That is, roughly speaking, I talked about local desk today, but in general, it is completely my development with the community's help. Not a single line of code, in my opinion, has been written by people. This is a portable desktop tool for working with models. I experiment with it myself, play with some Open Color, it allows you to conduct a mass of different experiments, right, up to consensus mode, running simultaneous tasks, and so on. And everything is on your desktop with its own sandbox. And at the current moment, well, I am a proponent of the fact that if you are not yet coding through AI, then those who code through AI have definitely overtaken you in terms of the number of unfinished projects. But if you approach it wisely and don't overdo it too much, because I met with guys, they tell me: "My sleep has been disturbed for the last six months." I say: "Are you so worried?" They say: "No, I send so many requests to the cloud. I think, now I'll go to sleep, I won't send anymore, I'll stop." But in the end, he sends more and more and more until 2 am, and it's time to get up, take the child to kindergarten, and so on. In general, everything is leading, as they say, to the emergence of new dependencies, new problems, but, in my opinion, it is a pleasant dependency if you have moderation in everything. Coding will not completely disappear from people. Just a part of it, yes, when people want to experiment, if they do it wisely, and when people want to understand the goal of what they are doing and can understand this goal, then, in principle, it leads us to achievement more often than if we didn't have it. That is, AI, in particular, agents, coding solutions, and in general, right, information search. I hear a lot of good, positive voices, that, yes, to please FOMO, people are really building agents that collect information and somehow accumulate it. And this is happening even without looking far ahead. I look at who reposts my channel posts, you go in, and there are six bots. Like this, you understand? They reposted someone who wrote such a solution for themselves into their channel. This indicates that the trend of writing ready-made services, right, by paying $20 for a subscription, a person can make a bot that will solve the problem of information gathering. It's already here. But there is also a big gap, that is, between those who really have access to all this most powerful, we are definitely power users and the future for us, in particular, but there are also those people who cannot get access to it and, for example, still have difficulties starting. Therefore, here I would, some interesting sound. Right. And I would, in short, say this, that it's great. I'm glad that it's generally appearing in our lives. And the philosophy boils down to the fact that there are both bad and good sides, but if you see the golden mean for yourself and you are comfortable with it, why not? Get involved. We have talked about all our channels many times. We try to review and make quite dry summaries of what guys, what Kolya from his experience, how we are moving through all this history, how we are moving. Therefore, I definitely suggest starting this year not only with runs, but also with installing cloud code in the terminal. And on this positive note, I think we can finish. With us today were Rifat Ametov and Valera Kovalsky. If you liked these reports, then I advise you to look in the description of this video. There is a link to the conference and the results of 25, where there are many more cool reports, both reports on development with artificial intelligence, and on the integration of artificial intelligence from the perspective of building processes and from the perspective of business. With you today was I, Kolya Shayko, as the host. We are finishing now. Have a good evening, good weekend, everyone. Bye everyone. Bye everyone.