Transcription
Hello everyone. And today we will talk about the topic of a month of development with CodeMax. The experience of best practices of AI Driven Development in Python. My name is Efektistov Stanislav. I am an ML engineer in the R&D team, where we are developing solutions based on artificial intelligence. In my free time, I participate in and win AI hackathons, and I am a speaker for the St. Petersburg Developers Club channel. And also, recently, I have become an open-source contributor. A brief roadmap for today's talk. We will talk about my developer background and the project on which I worked out my AI Dream Development approach. There will be a brief overview, the reason why I switched from Cursor to CodeMax, my version of the AI Driven Development paradigm, the main insights and mistakes that I would like to convey to you, and what are the outcomes of such an approach. Why this talk? Everyone has probably heard the term not only of coding but also of Driven Development. And it is undoubtedly popular now, many have even tried it, but nevertheless, there are not so many quality resources yet. No one highlights the problems that arise when scaling projects. And I will share my personal experience, my developments, which helped me to cope with these problems. Yes, to understand the starting point from which I began, let's talk about my developer background. 6-8 months ago, I was still stuck in notebooks, experimenting with ML there. And if I needed to wrap my solution in some kind of service, some other developer would do it. I didn't get involved in that. Then a situation arose when there was no one to transfer my code, and I started doing it. And, of course, I didn't start with competent AI Driven Development, but, of course, I started with web coding. At first, it was cool, of course, but after a while, I realized that it doesn't end there and you won't get far like that. Now I develop very reliably, consciously, and competently in the ID style. In a month, I develop a Python project of 20,000 lines of code, considering that I am not some experienced ML engineer. I am, first of all, a data scientist, an LM, NLP specialist, and so on. What kind of project did I develop? In a nutshell, it's an LM-based system for creating personalized educational material called Learn for AI4Battles, and a web service. You can scan the QR code via the link to GitHC. The project architecture and stack, meaning we have several services, it's a monorepository, FastAPI, Telegram, and so on. Well, in general, the entire standard Python development stack for agents. Let's not dwell too fundamentally on what AI Driven Development is in this talk. For that, we have the A Dialog channel and the VMST course, where we analyze the entire foundation in detail. Here I want to focus on more specific points, deeper practices. Nevertheless, let's do a brief recap, okay? My definition of AI Dream Development is a development paradigm using artificial intelligence, where a person acts as a technical director or architect, and code writing is delegated to an agent. And here's a quote: "Man builds systems, machine writes code." And how does it differ from AI Development? I would say that Dream Development is, let's say, a certain evolution of web coding. And perhaps it would be more accurate to say that web coding is bad AI Driven Development, and AI Driven Development is competent coding. Yes, I said that in IDD, a person acts as an architect, but here I would like to draw an analogy that an architect in classical development probably designs the architecture, processes, but does not write code himself. At least, probably, an architect in the classical sense, unless it's a startup where the architect is also a developer, also an engineer. But in AI Dream Development, the architect definitely reviews and sometimes even writes code generated by the LLM, because at this stage of large language model development, we cannot fully delegate code writing to a large language model autonomously. That is, it is still far from ideal. Some things are still beyond its capabilities, but who knows what will happen in some time. The essence of AI Driven Development is achieved through careful planning and project documentation. In general, there is no clear approach to documents; they overlap for many, but nevertheless, it all depends on the project, on the person. In my case, it consists of a global project vision, a business model, a roadmap, a backlog of tasks, and some ADRs for architectural decisions. And development agreements, although I don't actively use them. I'll explain how I adhere to them a little later. And at some point, I was developing work tasks using Cursor. After a while, I switched to CodeMax for certain reasons. Why did I switch? The main reason is, of course, money. In my experience, Cursors are very expensive. Yes, of course, there is a $20 subscription, but in my experience, it runs out in about three days of very active development, for example, full-time, and after that, payment for actual usage is activated. That is, you pay for tokens via API, and one model call can cost several dollars. I generally never paid for this in our team. That is, it was paid from our team's budget. Nevertheless, I got a large task. After a week, I saw such a bill in Cursor. And if I had completed this task on Cursor, at the end of the month, I would have presented our team with a bill of $1,000. And I was a bit embarrassed to do that, so I started looking for alternatives. I tried Codestop, but eventually came to CodeMax. There is a $100 subscription, CodeMax. And in addition to the basic $20 subscription. And as you can see, in a month, a little more, I would have spent $1,300 on tokens, but I actually spent $100 because everything is included in the subscription. That is, this subscription covers everything. But even here, there's a caveat: I didn't pay these $100. Bektemirov Sergey kindly agreed to pay for it, for which I am very grateful. He is essentially the sponsor of today's talk. He is an active participant in our closed Geek Club. So, if you wish, join us. Yes, let's briefly go over the valuable differences, then we'll break down each one in more detail. Differences between Cursor and CodeMax. The first, of course, is the type: Cursor is a classic IDE, some kind of VS Code fork, but, of course, with a built-in agent. CodeMax, on the other hand, is a classic terminal tool. Regarding limits, as I already said, Cursor has monthly limits included in the subscription, and then there's so-called usage-based, meaning based on actual usage. In CodeMax, there are five-hour limits with a caveat, because some abusers were found who shared accounts among several participants, and as a result, these five-hour limits were used all day. As a result, for a $20 subscription, one could spend $5,000 on tokens. And, of course, this all falls on Anthropic's budget. I would note that the paradigm of CodeMax is different. It's not just agent-based, it's multi-agent. Because there, the main agent can call some sub-agent, meaning it acts as a kind of orchestrator. And Cursor, of course, has background agents that were added after Codex, but nevertheless, an intuitive understanding has developed that CodeMax is more multi-agent. Checkpoints are available in Cursor. That is, you can send an agent with an implementation, and then click so that there is no need for confirmation. And after that, if you didn't like something in the implementation, the agent implemented complete nonsense, you can simply return to the previous checkpoint, and all changes will be rolled back. CodeMax does not have this. And working with the codebase, as Kudo brilliantly noted, in Cursor it's semantic search. That is, the model can ask a direct question in natural language into our codebase. It will return a relevant chunk, a relevant document. Well, essentially, classic RAG. And in CodeMax, it's classic console tools like GREP and find. Yes. Here you see Cursor. Well, Vadim, in fact, showed it, but I still inserted a screenshot. A classic IDE, but with a window for it. CodeMax is, first and foremost, a terminal tool. This is how it initially looks. But they did a very good integration of Cursor with CodeMax. That is, as you can see, I am in an IDE, in a convenient IDE with all its advantages, with all its features. Nevertheless, I have a terminal open in this IDE. And most importantly, I see the changes very conveniently, what changes my agent, launched in CodeMax, is making. And after that, CodeMax is no longer perceived as a raw terminal tool. And I haven't developed any particular intuitive differences. That is, it's as if I'm developing in the same IDE, with some minor nuances and trifles, okay. CodeMax limits. How long do these limits last? They reset every 5 hours. Of course, we can spend them within these 5 hours. In my experience, if you use the largest model, Claude Opus 4.1, it lasts for 2 hours of continuous development. But here I want to note that continuous development is when we don't think at all. It's probably even when we no longer review code, which I consider incorrect in AI Driven Development. If we go the more correct way, and we do planning first, then development, and all this together, then exclusively on the largest model, Claude Opus, development lasts for 3-4 hours. And if you combine Claude Opus and Claude Sonnet, Claude Sonnet is a smaller model, for example, not all tasks require a really large model. Many tasks can be delegated to a small model and not waste extra tokens. Then this approach lasts for all 5 hours. That is, you can develop without stopping. There is also such a convenient utility, CodeMonitor, because initially CodeMax does not show how many tokens have been spent, how many limits are left. They display a warning message that the limits will soon run out, and they display it after half an hour of use, even when 90% of tokens are left. Here we have a clearer scheme. It's not 100% accurate, but it can be used as a very good guideline. It even shows here when tokens will run out at the current usage. Yes. And as I already said, sub-agents in CodeMax are opened with such a command. You can create a new agent. There are custom agents, that is, those that belong to projects but not to a specific one. There are also project agents. For example, I pushed something like that in my project, and as a result, anyone can view and use them if they also have CodeMax. Here you can create a new agent. And, as was discussed, what was our question in the previous talk, that you don't necessarily have to give it a full system prompt, but describe the task in natural language. And again, the model will write the prompt. Everything is convenient and complete here. And here's how a document with an agent looks like, that is, the name, description, when the main agent CodeMax, which is in our chat by default, can call this agent. And below you see the system prompt begins. For many, it may seem interesting, but more on that later. And here's approximately how it looks. That is, we simply write in our chat: "Write an implementation plan for adding". I don't write that an agent needs to be called, and it, that is, our main agent, understands that an agent needs to be called and does so. Here it all displays beautifully. Checkpointing, yes, as I already said, in Cursor, we can click this restore checkpoint button and roll everything back. CodeMax does not have this, so we return to our good old Git, the version control system. That is, I heard about some extension in CodeMax to support this checkpointing functionality, but nevertheless, I still stick to clean development practices. I push commits, one feature, one commit, and thus CodeMax literally forces us to adhere to clean development practices. Working with the codebase. In Cursor, it's classic RAG, that is, we build an index of our codebase. In CodeMax, it's in the form of a similar command. And here I want to convey a very important point: in practice, I would say that these classic commands, that is, searching by word matches, work at least no worse, and in many ways even better, because probably the problem with RAG is the classic problem of embeddings. That is, it's not always possible to build a truly representative embedding that reflects the meaning of a chunk. That is, I believe that if we make an embedding of a piece of code, a piece of a document, where the business logic is not particularly reflected, that is, we don't have specific technical names, then this code is faceless, and therefore the embedding built on it is corresponding, for example, and it's not representative. Well, and in Claude, somehow, well, in practice, it really works better with manually directing agents to a document. I have used Cursor less often than when I used Cursor. Here I would like to note an important point about the chaotic nature of the CodeMax models. Probably many who have used these models have noticed this, that indeed, if you compare GPT with Gemini, for example, then Claude can be very, let's say, hallucinatory. They can do what they were not asked to do or not do what they were asked to do. And this is actually not an accident. It is indeed so. That is, they have very high initiative. But here I would like to note that their initiative is their blessing and curse, because their initiative allows them to succeed in agent programming. That is, what we need for AIIP, when they can navigate the codebase themselves, call tools, and so on. But this same initiative is the source of hallucinations, when they have to undertake a lot, and in the end, it turns out that they undertake it not when it's needed. In simple scenarios, this initiative is, of course, controlled by prompts. For example, communication in a dialogue. I have a very frequent scenario when I dictate some thoughts by voice, for example, about architecture or something else, then transcribe it and then upload it as text to the chat. And when I had a Claude Max subscription, I started, of course, uploading this, including to the large model Claude Opus 4.1, out of curiosity. And very often it started implementing something, writing code, or something like that. Although it wasn't in my words. I'm just throwing out my thoughts, and so on. And of course, in simple scenarios, this can be corrected by explicitly telling it. What not to do, don't implement anything, don't make changes, let's just work it out. That's how I write. But, of course, when we talk about the AI development approach, when we want to send our agent to implement an autonomous task, ideally, we don't even want to watch its code, meaning we don't want to review every change it makes. We want to see the completed task at the end. And, of course, this prompt is not enough. Even if we give it somewhere, after 50 tool calls, the model will certainly not remember it. And in that case, my tool became a quality implementation plan, following which the agent implements the corresponding functionality. That is, it is, of course, not perfected in all cases, but it is my main tool. And with the growth of the project, some prompt techniques were added, which I will also talk about. And in combination, it works even longer. Also, with the $100 CodeMax subscription, unlike the regular $20 Code subscription, access to the Opus model is unlocked, which is probably the best in agent programming right now. Many dream of trying it, as did I, before Sergey paid for my subscription. And a question: is there a difference? My opinion is that there is no fundamental difference. Both models are equally chaotic, in my opinion. That is, of course, the Opus model is better, therefore, it understands requests better, absorbs more context, and ultimately implements tasks better. As a result, in practice, tasks that can be given to Opus with a high-level description. For example, without code implementation, I have to write code for Sonnet and only then send it for implementation, for truly autonomous implementation. That is, without me watching every line of code. Now let's move on to my approach, my version of AI Driven Development. Task planning. We have, of course, documentation in the project, in particular, a roadmap, where we briefly outline all the global stages of project development. Examples of such stages you see on the right. A high-level stage, a milestone, is divided into some major tasks, initiatives. Each initiative is divided into specific implementation plans, and one implementation plan, one task, is one iteration of agent development. All of this, examples you see on the right again. Here's what the roadmap looks like. That is, on the left, we see our project development stages are very broadly outlined. Open source release, production platform, and advanced features. There won't be many of them. And in each of these milestones, we outline these major tasks. They are outlined here quite literally in bullet points, but in general, I don't see anything wrong with that. The main thing is to fix it and not lose the idea later. Of course, if you have any thoughts, you want to document them immediately, it's perfectly appropriate. You can adapt it for yourself. But I've come to the point where when an idea arises, I'd rather just fix it in bullet points like this, because if I want to fix it in more detail, I usually don't have that many ideas initially, and the models come up with these ideas themselves, which I consider incorrect. So, here, the main thing, well, in principle, as in the rest of the approach, the main part of the work should be done by you. The agent will undoubtedly write this document, but your understanding should be complete. That is, you should see the project from a high level, as clearly and in detail as possible. Next, mid-level. Based on these initiatives, we want to fix a major task. But sometimes, based on this thesis, it's not always possible to clearly understand what we need. Sometimes research is needed, to delve into details. Exploration. And this is also documented and recorded in the Explorations directory. For example, I need to implement a service, but I don't know how it should work, what are the nuances, and so on. Consequently, I need to work this out first, and this should not remain only in the chat with the agent. It's better to fix and document it. I record it in a separate Explorations directory. You can make a lot of edits, iterations, and so on there. If an important decision is made, it can be included in an ADR, which is an Architectural Decision Record, if any important decisions are made. But, of course, you can't always explore. You can plan immediately if you have a crystal clear understanding of what you want from the large task you are planning. The description of a major task is placed in the corresponding directory, and there is a general README for this task. And in this document, we describe this task more generally, more high-level. And in this same document, we can divide it into separate stages. So, literally slightly separate implementation plans for one iteration, we will implement further. Yes, here's what the task backlog looks like. For such a history, I have an archive folder and ongoing tasks. Here, you see, there is one task in both directories. And here, after implementation, instead of an implementation plan, we place a so-called post-implementation summary, because during implementation, some deviations may arise, and we fix them here. And the original implementation plan itself, for versioning purposes, is placed in the archive. And we also index all this change history in a special file, index.md. And here's what it looks like approximately. That is, we have the entire change history, beautiful, concise. I believe it's very important to maintain such a thing. At least, it's very important for me to have this in my project. What is important in an implementation plan? What should be in it? I am not accidentally adhering to such a hierarchy, such a structure. That is, here I have sorted in descending order, so from top to bottom is the most important. That is, in general, first I ask the model to reflect the meaning and goal of the task being implemented. That is, not just what needs to be implemented, but why and for what purpose. Next, the current project state. I indicate whether backward compatibility and tests are needed, because without this, our initiative-driven Claude loves to add some unnecessary backward compatibility for a project with no users and all sorts of tests that have never been in our project. Next, what should the implementation plan consist of? There are only two points, but I'll briefly go through them. Then we'll look in detail. It's the full workflow of new functionality, that is, a complete sequence of actions. And this is not for the agent, this is for us to review qualitatively. And, of course, edge cases that may arise. Of course, we can't account for all of them, it's simply impossible, but nevertheless, it's very good to understand what problems we might encounter. Next, requirements. I specify such a requirement that code should not be written in the implementation plan, well, conditionally, only contracts, that is, arguments, types, and so on. But, honestly, with the growth of the project, I'm coming to the conclusion that perhaps it should be written, because without it, sometimes even the large model doesn't work well, especially if the logic is complex, the project is large, there's a lot of context to keep track of, and so on. Here, I believe that when writing an implementation plan, the most important thing is to adhere to the YAGNI principles, that is, not to implement functionality that is not currently needed, and to stick to simple solutions. And there is also a requirement that the agent needs to do before creating the plan: study the implemented functionality, study the project stack. Yes, let's break it down in detail, let's look at it. Actually, yes, here, I hope it's quite large. Let's even do it like this. This is a prompt for the agent that I call to write an implementation plan for me. As you can see, the prompt is not written in Markdown format, as everyone usually writes, it's written in such an interesting XML format. Well, the format itself is not interesting, of course. Many have encountered it and work with it. But for writing prompts, it's only now gaining popularity. And what is its essence? The key difference from Markdown is that it is bidirectional. That is, in Markdown, in lists, they don't reflect the end, they don't mark it separately. But in Markdown, we have a closing tag that clearly separates. So, due to the names of these tags, due to the nesting level, we can convey the maximum meaning of the prompt. We can convey the maximum meaning of the task to the agent. Without this, we would need to write a lot of words in natural language. But this format is empirically proven to be easier for models to perceive. But bidirectional is needed for the underlying technologies of a large language model, self-attention, and so on. And as I already said, these tag names reflect this requirement, for example. Let's move on to the prompt itself. We have the project context. Here I обязательно indicate that we are at the MVP stage. That we have minimal complexity requirements. That is, we don't need to complicate anything. We write simply. Also, there are requirements that at the current stage, tests are not required. Next, we have a critical constraint. Well, as I said, I indicated that code should not be written unless explicitly stated. But lately, I've been asking the model to write code more and more often, especially if I've worked it out myself beforehand. Therefore, this requirement may be revised soon. Well, here, in principle, it's standard, we've already discussed this. The YAGNI principle, then adapt existing components. No backward compatibility, because, well, up to the current moment, my project is not even open-sourced. And, well, what backward dependency, it's unclear for what. Well, here, in principle, an agreement on how to save this task. Well, here, each project is set up independently. And here is the structure of the plan, the meaning and goal of the task, architecture, full workflow, interfaces, and so on. There is nothing fundamentally new here. Well, here I believe the most important thing is, of course, the meaning and goal of the task. Probably not everyone has this. Usually, it's not customary to write this in a dry formal document. Next, after our agent has written
We have an implementation plan, we bring this plan to an ideal state, because initially it is, of course, far from ideal. And what does an ideal plan mean? It is crystal clear to us. That is, there should be no details, no moments, no technologies that are unclear to us in the project, and we see them just as a word, without understanding the meaning. This should not happen, because if we have no understanding, then what can be expected from the agent. And further, there should be no contradictions in the plan, because it happens, especially after processing, after improving the implementation plan, we have some contradictions remaining. For example, in the main part, the use of a certain class is indicated, and at the end we forgot to correct it, and another class is mentioned, which was there initially. Of course, this should not happen, because even a brief mention can greatly affect development, and an agent, with the mention of one class, can implement it completely. And without the remaining artifacts from the previous refactoring, that is, from the initial plan that we edited. After this, we send the agent to implement. That is, as I already said, I do not monitor the agent, how he writes every line. I send, I give him the implementation plan and press Shift T, so that he, uh, makes his own changes. That is, I don't look at every line he writes. And after implementation, yes, here I will talk about practices that I did not have before the project exceeded, uh, the mark of 15,000 lines. But over time, they had to be added, because after this mark, even the best OPUS 4.1 model began to hallucinate very often, that is, to deviate from the implementation plan. Not significantly, but nevertheless, with the growth of the project, I think this is inevitable. That is, earlier, when the agent finished implementation, I started reviewing the code. And now, instead of reviewing immediately, I first ask the agent to do so-called self-reviews. That is, go through the implementation plan, describe what you haven't implemented from the implementation plan, what you implemented with deviations, and what you implemented beyond the plan. And further, a pair of improvements. That is, the agent writes what deviations, that is, we write to him, we make some minimal decisions, whether this or that needs to be refined, because our implementation plan may not account for everything. The agent may encounter some moments that are not described in the plan. And this is actually a normal practice. An ideal plan can never be made. As a result, with this approach, it became much more reliable and easier to work with. Hallucinations, their source is not now, but we have somehow coped with them. That is, I no longer have any fear in my head that my agent will start evolving a lot with the growth of the project and so on. And after we finished with the implementation, for thorough project documentation, I write such a prompt. That is, after we finished with the implementation according to the plan, update all relevant documentation and archive it yourself, index it, yes, into the history in index.md, and in its place leave a post-implementation summary. Where it is very important to describe not only what was implemented, but also what deviations from the implementation plan there were. The agent does all this. Here is a very important point. This post-implementation summary is very important when planning the next task within a large task. Therefore, I write all this one after another. That is, I don't write five implementation plans for one large task at once. I first write one plan, implement it, because, yes, some moments may arise in development. And thus, when we write task two based on the post-implementation summary of task one, we make this planning more thorough and, perhaps, more correct. That is, we plan not based on what we should implement, but based on what was actually implemented, right? And what are the main insights from AI Dream Development that I would like to convey to you now? A lot, at first glance, can be delegated to the agent. For example, you can not read code at all, not read documentation, not have a vision for the project. The agent will still write something, but it is very important to keep the main focus. That is, the first is understanding the project. It should be primarily with you, not the agent, your vision, plans, and current state. Because if you don't understand how you are going to manage the agent, then essentially, when you develop in the AI Dream Development paradigm, you are essentially the author of the project, its manager, product manager, developer, architect, and analyst. Further, it is very important to adhere to best development practices, because it is very important for the agent, for example, to be able to conveniently launch the project, service, to debug it, logs, because without this it is very difficult. That is, it is difficult for a person, and even more difficult for an agent. And dependency management – all this is part of it. Next, such an important aspect as a single source of truth, because now, yes, when an agent can generate a new document in 2 seconds, the main thing is to keep all documents synchronized and organized, so that there are no two documents dictating different truths. My case was that I was describing a large task, and there was some document that I didn't read at all. It was generated by a language model for the same task a long time ago, when I was just planning it on the roadmap, and completely different things were written there. And after that, my agent picked up this irrelevant implementation plan, synthesized something average based on it, and it turned out to be complete nonsense. And as a result of this concept, it is more important than ever to adhere to it. Well, and a very important piece of advice, I would advise everyone to study computer science. That is, at first glance, they shout: "Artificial intelligence will now replace all programmers." But I don't think so. It may replace some coders who exclusively write code and understand nothing higher. But a good developer is still a person who understands how it should work. Perhaps he even delves into architecture, not just the architect. And now, even I, as I said, am not a very experienced backend engineer, but nevertheless, I started learning, some architecture, some design patterns, development, and so on. And I feel it, it is very important, it greatly helps in AI Dream Development. That is, I believe that a strong developer, for example, Vadim with his global experience, will find it quite easy to become a really cool specialist in AI Dream Development, even if he doesn't have much experience in SVM development now, because he is a very experienced developer, which I am not, but I plan to become, of course. How effective is such development? In 120 hours, one person developed 20,000 lines of working code. At the same time, he did everything: architecture, code, design, documentation, even recorded the video logo himself. And here is a QR code to the project where you can make sure that all this works and launches. This is precisely the Learnflow AI system I was talking about. If you like it, please give it a star, I will be very pleased. Thank you for watching the report. I will be happy to answer questions and feedback. Yes, thank you for the report. Now for questions. Questions. Well, we have such, Nikolai relies more on his experience. I think we can just comment, agree. He says he does it. You see, right? Yes, I do. One task, one chat. That's right. I also, before I wrote the implementation plan, or rather, after I wrote the implementation plan and started implementation, I create a new chat so that the agent has a clean context, where there will only be the implementation plan and further steps. Everything is correct, right? I agree completely. Next is about Claude subscriptions. There is a cooler subscription. I don't know if you know. Well, yes, that's exactly what I was talking about, the $100 one. I had it this month. Well, initially I used the $20 subscription. Well, of course, with a $20 subscription, the limits are much smaller. And most importantly, it doesn't have this powerful model. But of course, not everyone has an extra $100. Well, for $20, you can also develop quite well, in principle. But of course, if you buy a $100 subscription, have these token limits, and know how to develop competently, I think it will pay off tens of times over if you know how to do it. [music] Next, about XML SQL, when you saw XML. How did I come to XML? Well, generally, I read the Telegram channel, let's say, of a prompt engineering specialist, Vladimir Ivanov. You probably know him, Alexander, and he analyzes in great detail why XML prompts are better. But are there any recommendations from Anthropic? I can't say. But now I know for sure that ChatGPT has included examples of XML prompts as a replacement for Markdown in their guide, because, well, it's precisely this duality that primarily attracts them, that in Markdown, sections can get mixed up for the agent. That is, in Markdown, sections are distinguished by a line break, and the agent might, let's say, not pay attention to this line break. But when there is a token, that is, yes, in XML, these words are converted into tokens, and tokens already carry some meaning, some vector. So, primarily, probably, this duality, the ability to delineate various sections. Well, and, as I said, this semantics, this meaning that you convey in the names and nesting of these sections, tags, and so on. If you ask the model, it will write such a prompt. If you just ask, like, write me a prompt in XML format, then, I think, something incoherent will come out. But in general, I am now, yes, working on developing a prompt, and there, yes, I give these instructions on how to correctly write a prompt in XML format. Therefore, if you give the model a system prompt where you describe how to write a prompt for XML, then, yes, I think it will write it very calmly. And this is an accurate option. I, in principle, don't write it manually, but I very carefully validate what it writes. Something like that. Thank you for the question. I would probably add, perhaps, a disadvantage of XML is that it is verbose, right? It needs an opening tag, a closing tag. And with models and this context and token consumption, it's not great. But XML has a cool feature that Gy and Yaml don't have – it's the possibility of attributes, right? That is, we can add some additional attribute to each element, that is, meta-information. Well, perhaps this is still, let's say, a real thing is being born right before our eyes. And everyone is trying something new, or reinventing something, and so on. Next question. Now, can I comment on your addition? Also, yes, a cool point about this meta-information, attributes. Honestly, I haven't tried it, but an idea just came up to try it. Regarding verbosity, I disagree a little, because, well, in Markdown, you need to write a lot of prepositions, articles, and in XML, probably, yes, you write the same words twice at the beginning and at the end, but nevertheless, you can omit many prepositions, probably. For example, here in the XML prompt, I just wrote one word between two tags. But if I had written it in Markdown, I would have written "project," I would have written "project." I'm probably leaning more towards the fact that in the end, we should arrive at some kind of Yaml, because in Yaml, you just write "stage: MVP," and you save so many characters. Well, Yaml, yes, but again, the lack of duality, the limitation, that's probably the minus. This is very, well, important for this transformer. Well, technology, yes, it's a type of carrier. Well, yes, and finding the end of a string is difficult. Yes, yes, yes. Okay, let's move on to the next question. About cloud functions. Integration of CLS GitHub. Hmm, probably since I can't answer, I probably don't use anything, and there, in terms of functions, like, can the KD access my repository, answer something about it, and so on. I just don't quite understand what tools there are. You can connect a GitHub repository for code search, if I'm not mistaken. Yes, but probably not. Although the feature is very cool, interesting. Well, Alexander will supplement the question now. Yes, if he supplements, we will answer. Next, also Nikolai with his opinion. Visual Studio. Visual Studio. Well, I don't use Visual Studio much myself, so the integration there is probably through a plugin and depends on the plugin developers, probably. Well, yes. Next. And it cannot maintain context. It will never replace a human. Never. But we'll see about that, of course. I myself hope that I won't be replaced by a large language model. But in general, the point about context is good. I strongly agree with it. That is, indeed, probably one of the fundamental limitations why artificial intelligence hasn't replaced many of us yet is context. Indeed, Nikolai is very right. And, well, given the speed at which they are developing, I think "never" also has some expiration date. Alexander supplemented the question about cloud functions. These are essentially, all sorts of reviews, I don't know, creating issues, that is, not old tools like the ability to create, yes, some function is given to the agent and it can call it on GitHub. I understood the issue, roughly. In general, no, I haven't used it, but essentially, an issue is when some third-party developer can open some issue on my project, right? Because there's a bug or some work to be done, if I remember correctly? Well, for example, you created, yes, yes, essentially a task, right? A bug or a feature. Essentially, you created a request, added a new feature, and your function can either review it or create, I don't know, you decided to put a block in the to-do list and create, ask the agent to create a task for you for the future. Like, I don't know, improve quality or something like that. Well. But not yet, I haven't used it, but I might. But again, the fundamental point is that I don't want to release anything fully autonomously without my review anywhere. Not even for review, well, okay. Definitely not for implementation, and not for documentation either, so my control is still needed. And here, probably, with this in mind, it won't boost things that much, so I don't know. Well, yes, actually, the technology is raw, it's like it only started gaining popularity somewhere in the spring, when they started giving access to various agents so they could go into the code and do things. ChatGPT also launched Codex. Next question. About acceleration, professional growth. I believe that I have never programmed so much, so powerfully, so complexly as I have since starting with the AI Development approach. That is, not that I, of course, don't write code mostly, but I believe I have begun to understand the essence of development with the AI Development paradigm. I did an event where I had to dive into all the services, protocols, and so on. That is, again, yes, as I said, I am not doing web coding, and as I gave advice, you need to study computer science. Probably, first of all, I gave it to myself, for myself. So, how much faster have I started learning new technologies compared to the classic approach? It's hard to say, but it's clearly several times faster, maybe four times, maybe. Well, something like that. Any recommendations for those who are just starting. For everyone, let's say, for beginner developers who can develop in the AI Development paradigm and grow quickly, what are the recommendations? Well, probably, don't be afraid and do everything consciously, that is, don't be afraid. That is, I usually think, now, for example, I want to apply some pattern, and I think: "Now I need to spend a couple of hours understanding the theory and so on." But in reality, theory is needed, but it's better to practice it. And I would advise not to be afraid, but at the same time, to do everything consciously with understanding, because without understanding, the agent will finish implementing, and everything will be forgotten. When you do everything consciously with understanding, it sticks with you, and in the future, you can make very cool decisions about architecture and development. Well, probably, these are the recommendations for now, but maybe I missed something, maybe I didn't add something. Actually, this is a lead-in to my upcoming report. I am also working on, well, reflecting on growth, on how to learn and all that. My report will be focused on this. Next question. This is probably about experience with Coder. Honestly, no. Besides Cursor and KD-Code, I haven't developed on anything else yet. I saw a video on the A Dialog channel about Coder. I watched it, and I'm interested in working with it, but in the last 2 weeks, I've been very busy with my pet project and haven't had time to try anything. But of course, I plan to try. But otherwise, I haven't tried anything yet, so I don't think about it. Well, Alexander sent a link to the video, he did a review, and probably, well, this creates some competition. I think it will benefit everyone. Next question about XML again. Yes, that's right. Exactly, well, this is the beginning of a system prompt. That is, they are sent to the LM in this format. That's right. That is, it's not converted into anything, but it's clear that under the hood, it, uh, turns into tokens. Here, probably, there will be such and such tokens, but they will be separated by all these. And yes, so they are sent. Next question. Apparently, about large context. Milus MCP context. I honestly don't know what context is. I know it's a vector database for RAG, if, of course, this abbreviation doesn't mean something else. MCP module context protocol. Well, I don't quite understand, but I can assume that CLD will probably store the context itself, probably not send it every time, but this needs to be, well, indexed in a database, accessed when needed, and so on. But this idea is interesting, that is, essentially connecting, well, the RAG that is in Cursor, but in Claude, in KD. Well, of course, yes, when large projects are needed, you need to come up with something like this, probably, this is one of the possible options, the question is in the approaches, in the quality, and in the specific methods. Perhaps, I had these ideas, that is, I said, yes, about the moment with the poor construction of building based on some piece of code. Well, I think, perhaps, if you mark it with comments somehow, so that embeddings are built, so that embeddings are built, you can give comments and build embeddings based on them. Our meaning is truly reflected. And then, I see, you can develop something very cool with this approach, but this is a separate, separate matter, it needs to be worked out separately. Yes. Sergey writes here that, well, essentially, like RAG, right, with indexing the codebase, so that it navigates the code better. Next question is about. Wait, one more time about XML clarification. For XML, is there no schema describing all fields? Can we set any fields? Or what fields are meant here, tag names? Well, probably, you essentially invented this schema yourself, right? Yes, it's completely abstract. You can set any fields. Yes, it can accept any words. That is, we invent it ourselves. It's not some kind of template. We can write any words here, even with mistakes. I don't know. It's not like, remember the structured output topic, right, when we give a schema, it fills it out at the output. Here, yes, it's not that, yes, it's just input, it's just, well, regular text, but yes, it's in XML format. Just, well, so. In a format understandable to the machine, right, to the model. Yes. Yes. Are there any more questions? No questions yet. And we actually had a question for the previous report. I will try to answer. Can a mobile application be created similarly with launch on an emulator? This is for Vadim's report, where he developed bots. I think there is low demand for mobile development, or mobile developers are skeptical about it, right, and they don't try to implement any practices. And I'm getting at this: I think if you try to develop a mobile application yourself with, I don't know, with an agent, it would be worth an article, a report, or some video. Because, actually, this experience is being gained right now. And I think there might be a problem with integrating, giving the agent some information from the emulator, because on these emulators there are some top IDEs like Android or something else, or there is iOS in Xcode. And even other IDEs can't surpass them. So, there are, let's say, top IDEs, and no one wants to compete with them. I think this is the nuance, that either you need some, I don't know, MCP or something similar, so that information is given from the emulator to the agent. Or rather, it requests it and receives it, like, is it drawn crookedly or not, as agents do from HTML. So. No new questions. Aha. Well, here they are calling for an article, yes, the idea is there in general on Habr, of course. Well, it just takes time. Well, yes, there are plans, of course. Well, yes, so, clear prompt markup in XML. Well, yes, in a concise format, let's say, yes. Sergey then says, with the ability to refer to specific tags. That's true. That is, if, for example, I can refer like this, somewhere else at the end of the prompt, I can write, like, look at the section. And I can refer like this. And it works very well. Because here, the name matches, the tokens will be the same. This will make the model's attention mechanism work strongly. Essentially, like for using variables. Yes. Well, yes, by the way, here, by the way, the guys in the chat remembered SOAP with XML. Now maybe it's worth telling about it briefly, what it is and why it was needed. So, we have, regarding questions, it seems everything is done. Thank you very much, Stas, for the report. Thank you for the questions.