Transcription
To the next question. The next question. Let's take the next question about using SPEC driven development. And you, you said you would talk about your flow. I'm here a little bit, well, maybe you can add your comments, then I'll add a little more comments, and we'll listen to your experience. This topic was born as a discussion in the chat, about how to use it at all and to look at different approaches that different chat participants use, because there is a hypothesis that it greatly improves the quality of agent coding, especially on medium or large tasks. >> It seems to improve the stability of responses, as far as I understand. The repeatability of responses even. The quality, the repeatability of the quality of responses. Well, let me give a quick introduction. Spect Driven Development is an approach in AI development where the specification becomes, in fact, the dominant artifact. And thus, the wipe coding effect is eliminated, when your context is constantly different, there is little of it in each task. In this way, by operating with context, you clearly control what is fed into the input, and, in principle, you can expect some guaranteed quality of the result. Uh, I divided this into two parts. There is a camp of people who believe that Spect Driven Development is based on tools like frameworks. Uh, there is functionality in Kra Spect Driven Development, uh, and there is in QPC Driven functional, where the AI itself creates a development plan, the design of your features, you approve all this, and then development begins. There are other guys who adhere to the fact that Spect Driven Development is, in fact, an approach. They are just tools. There, as far as I understand, there are now two leaders, maybe even three already. This is GitHub's SpecKit, which implements Spec Driven Development through standard scenarios, similar to how it's done in Cairo, spec, plan, tasks, execution, and the BM method, which involves different roles. Uh, what we were just talking about, it involves agents with different roles, and they try to process their respective tasks with different roles and produce the final result. I also found an interesting implementation, uh, more similar to Spec Kit - this is Open Spec. This little library. I'll send it to the chat. It, in principle, does the same thing as SpecKit. But in the end, as far as I'm concerned, all these libraries and all these approaches are about context management, about writing MD files, in fact, which are placed in the context. MD files on how to work with context. Well, Denis, now it's your turn. >> Well, yes. Uh, here I have an evolution, I also wrote in the chat, the evolution was from trying this approach in Cursor, because they promised that it gives great results, but in the end, Cursor, you used it through context 7, probably, right? Or >> No, I purely used Cursor, but that was a long time ago, back in the spring. Uh, the point was that it prepared a complete plan, and then it couldn't execute what was written in that plan. So it was pointless to use it. At least, I didn't get anything at all. And now I'm succeeding at work, there are guys there, they make agents, they don't write code by hand at all, and they have a kind of template. Or, let's say, I made a template based on the specs they write. They have one file, which contains, uh, a small research, uh, a small spec, small tasks, uh, and some other junk that everyone says needs to be added. So it's impossible to read what it adds. So there are like litigations, there are some estimates it writes, like, well, it writes code. With this thing, the coding quality has significantly improved, because you can work with this file and, in principle, develop small features like this. Uh, but the result didn't satisfy me, because the file is large and there's a lot of junk, it's impossible to review. Uh, my next evolution was that I used GitHub's SpecKit. Uh, the problem with GitHub's SpecKit is that it generates huge files. >> And what does working with SpecKit look like? Is it just recommendations, or is it a library that monitors files itself, fills them in itself? >> Well, in short, this BM and Spec Kit are like several elements, so there are prompt templates that are used for writing these artifacts like research, specification, and so on. Around these templates, there is a wrapper for agents like CodeLlama, Codex, and so on. >> So, like ready-made settings for specific agents, right? >> Well, there are like commands, it generates commands >> and then there is a wrapper in the form of some scripts. This is in Python, in Bash, in JavaScript, in BMDE. Uh, based on these scripts, it runs the flow of this spec driven development. But the point is that they all, both Spec Kit and BM, they want some kind of mega-universal flow that can be done for any project. This is super unclear. Well, I'm looking at this diagram, I don't understand anything. Well, this is a description of how a typical software team works in outsourcing. Like, >> well, like, yes, but it's complicated. I, uh, well, I needed a simpler tool. So, in the end, I took what SpecKit generated for me as final templates and final artifacts, and based on them, I created a similar flow, similar templates, but for my stack, for my projects, and maybe iterated on them a bit to refine these templates. And I have four commands left, like research. Research does something like a search, which does perplexity, or you do it yourself when you research a task. So you have a task like refactoring a project, using a new library, like some cache library. You say, like, here's this library, it needs to be used in these modules, in this project. Do a research for me, ask it some questions, what the library does, how it can be integrated into my project in the least verbose way, all that kind of crap. >> And you're using CodeLlama for this, right? Yes, it generates a research file, and it's like a canvas, similar to ChatGPT's canvas, which you interact with and say, like, I need this module, I need TTL, some other stuff, just like a brainstorm, like a rubber duck. >> I have a question right away. By the way, UmpuTun in the last episode of RadioTI highly praised Anthropic's skills and says that you can also use skills in CodeLlama and that a good selection of skills for a project almost replaces all Spec Driven Development approaches. Your agent just knows what to do. You just load the necessary scripts, the necessary description into the skills. Have you tried skills? >> I tried, but I had a problem that CodeLlama didn't write a simple enough skill for me. I struggled with it for a day. I needed to write an analog of a trace. So I have environments, I have entities, and in DynamoDB there is an entity, on S3 there is some derivative artifact from this entity, like an image, figuratively speaking. And I have this information on staging, and on production. I asked CodeLlama to make a skill. So I already had the full description for such a skill. I had a script, but my colleagues said, why add such a script to the main codebase, like re-engineering. I thought about making a skill. Well, and it struggled with this skill. In the end, it just didn't produce it. So they had a promise that you can give it the ability to write code itself. Well, it, uh, can't wrap it itself, and sit and figure it out, so basically, a skill, as far as I understood, is some kind of prompt, and you have to give it some super simple script, or something super clear. And if something is not very clear, then nothing works. So in the end, these skills, well, probably useful. And skills, well, in short, if we talk about SpecKit, then I had this thing that there are templates, templates are adapted to the project, there are commands that can be added to the project itself, and they will help you run through this flow. In the end, you work with MD files. Uh, in principle, the biggest stage for me now is research, uh, and everything else, in principle, is generated from it. My spec is super small, about 100 lines, which highlights some user stories, considers some race conditions. In the templates of GitHub's SpecKit, the prompt is configured in such a way that it also asks you for test cases, that is, it asks you questions when something is unclear to it on some issues, and you answer them while working on the specification. So there are useful questions. Plus, when working on the specification, you can see where the agent starts doing some crap. That is, it does not do what you want. So you fix it there. Then it makes an implementation plan. The implementation plan is actually code. You write in the template: "Write pseudocode for me, don't write too much, tests are like this, that and that." It makes this implementation plan for you according to this template. And based on the implementation plan, it then creates tasks. In tasks, it creates a graph of mini-tasks. So for a large task, there will be about 100 subtasks. And there, it can call sub-agents already, as it were, so it determines what can be executed in parallel. So, figuratively speaking, in the end, your feature task fits into what you write in research, you control that based on this research it made a spec, and an implementation plan that can be read, so it's about 100 lines. >> It's still unclear why all this was singled out as a serious separate name, Specflow. Well, before that, we had a term, which also tried to claim to be a single term. Now we have Specflow, although in fact it's just an iterative improvement of your prompt, which explains how to work with context. Isn't this actually prompt engineering, only prompt engineering with brains? >> Well, not entirely, so, uh, this research is prompt engineering, context engineering, which you describe. That is, a large document that creates the structure of your project for your specific task with specific file references, with, >> well, that is, the agent does the work, explains to the agent the places needed for work, shows the agent the most essential information needed for working on this project. Well, yes, but it's not new at all. >> Well, it's more about collecting context. And the spec is more for a person to understand the task. So, it, uh, you >> What is a spec in the standard development process, that's clear. >> Well, in agents, in agents, it looks like nothing new, just additional context about the project, automatically updated. I don't see this difference. I'm just trying to understand, is this some new direction, like, I don't know, a new way of developing AI that will be with us for another 10 years, or is it just a new name for ordinary human prompting, when you normally manually provide context and get tasks written by agents normally. Well, it's difficult here, here >> here >> now money will finish, then te >> Yes, the problem here is that when you have a complex task, a complex one, there is one that you, as a programmer, would do for a week or two, write, I don't know, a couple of thousand lines in a complex project, well, in a complex, medium one. And you have a problem that this feature is not solved by wipe coding. That is, you cannot solve it in a wipe coding session at all. That is, it, wait, what do you mean by wipe coding session? I spent 12 hours writing a prompt that made a medium-complexity feature for me in one run. Is this still web coding, or is it already? >> Well, this is more like a spec. What you wrote a prompt, >> based on which your code was generated, >> and this is what I would call the essence of this spec driven development, because in the end you get a large artifact of 1,000, 1,500, 2,000 lines. >> Right. >> Well, well, I disagree, Denis. Just now, from one prompt, from three lines, half an hour ago, I got 500 lines before your eyes. This is already this is not, wait, we are talking about different things here. I'm saying that spec driven, this flow, it allows you to get an artifact in the end. The artifact is a markdown file >> with a prompt, which you then give to CodeLlama. Understood. Inside this prompt, it is written which sub-agents should do what tasks in what order. >> Well, and for that, there's already a name for two years, it's called metaprompting, when you ask the model to create a prompt for you based on context that will be better suited for the task. >> Well, this is metaprompting, yes? So I think that But it's laid down in a methodology >> Aha. Okay. >> A certain one, that you have any task. You fit any task into four stages, there, research, implementation, >> design, >> no, not there, but research, spec, implementation, and tasks. And you can, >> well, a small to medium-sized task >> solve. At the same time, if you understand, well, if the task is clear to you, then this flow is super fast, and in the end you get the result that is described, >> that you expected, like a controlled result. >> Yes. And this is the main value. >> Let's go. >> Our other other methods did not give me this predictable result. >> Uh-huh. Thank you. Let's give Vitaly a word too. Vital, Vitalik, hello. You were asking me a question in private messages about Specflow. and about BM. Can you, can you tell us? >> Not about Spec, about SpecKit. Sorry. Yes, >> SpecKit is GitHub. That's how the team develops. Well, here I would probably say that Spect Development is one of the approaches to context engineering. And context engineering, let's say, is the practice of managing and limiting context for LLMs to get more predictable and adequate responses. Right. >> Uh, well, and accordingly, context engineering is a subset of prompt engineering. So all this is prompt engineering, but it's like a separate chosen direction, separate practices. >> But doesn't prompt engineering also include context compression? >> Prompt engineering is a very large, it's a very large area. So it's not a metric. Well, it's like LLMs are part of and are part of, well, more >> it depends on how you look at it, Carpati has completely abandoned the term prompt engineering and said that it's all context engineering. >> here. >> Because we need to manage context. We have an unlimited window, even people themselves, uh, well, just look at how engineers work, right? We break down the global task of an application into small subtasks and focus on a small volume, well, on a small task. We cannot simultaneously keep the entire application in mind and always program it. >> Listen, from that point of view, won't it be the opposite, won't prompt engineering be a subspecies of context engineering? Context engineering includes, among other things, engineering and prompt creation, and building context around the project, including for the operator who writes these prompts, and for the agents who use them. No, >> no, look, prompt engineering is quite a broad area, because it contains both one-shot and iterative approaches. Context engineering is more focused on an interactive approach. That is, in one iteration, we create a kind of artifact to make the next iteration. That is, we don't give the LLM a global task: "Develop this feature for me." Yes. And let's discuss and develop it immediately. We first discuss what this feature is. >> Okay. So, context engineering cannot be single-prompt, which are non-interactive single-prompts. Well, here's the approach, we gradually, by communicating with the model, create intermediate artifacts, based on which we then create code. >> And >> the same as >> as before, we did it with a team of people, right? So business analysts prepared the specification. Based on the specification, an architect or tech lead prepared a solution. From this solution, the team then planned, broke it down into tasks. Developers did it, testers tested. >> Okay. Let's assume we've more or less settled on the theory. I'll throw in a question right away. If anyone has anything to add to it, jump in. The question is, how much is this, how much is such a token-heavy approach. Acceptable, damn, I'm forgetting words. In short, how much is it, uh, suitable, it's not heavy, it's actually much more economical. >> Why is it economical? You have, you have a lot of things running to do a medium-sized task. >> Uh, Lesha, please tell me, how many attempts will you make to create one prompt that will do it for you in one go? >> Well, one. One. >> One, >> yes. >> To, like, a feature in a normal, like, serious project, >> right? Yes. No, we're just forgetting that we're not wipe coders at all. We make one prompt, and then we fix it. Nothing will break off me. I made a prompt and I'll fix it. >> Uh, >> well, look, it depends on, uh, what volume of feature you're making. If it's to fix, say, the name of a button or the color of a button, then, well, you really don't need to write a specification. No, it's not about the color of the button, of course, to write some feature in 30 files that you'll be writing for a day and a half, it will do it all for you. >> And you want to say that you can write a prompt that will execute it for you at once, and then you'll be sorting it out for several days? >> No, I'll be sorting it out for 2 hours. Literally, look, I have about 15 files generated in Cursor right now. It takes me an hour to check them. If I wrote them by hand, it would take about 700 hours. The prompt was literally three lines. So, if everything worked out for you the first time. 100% it didn't work out. There will be bugs 100%. But I know my code. >> And what will you do? Read it, debug it with these bugs, and fix it. >> Maybe by hand, maybe with additional prompts. Yes, of course. Well, I don't know, like, even in the past, even simple bugs required time to fix them, to debug them. You also need to look at how to fix it. So, if, of course, the logic is simple, then yes, easily. But if the logic is not simple, if you're making some kind of combine that processes data, well, where's the logic, well, there are equations there, well, a couple of hours. >> Hmm, let's hear more opinions from the guys with their hands raised. Let's start with Denis, then Valera. >> Well, in short, you, Alexey, don't see the problem, so it's difficult. But this is a common thing when you discuss it. In short, I wrote, as you describe, so I made some prompt, the code changed. And in principle, this approach works for small tasks that are understandable, and you know where to fix. So, instead of going to the AI to fix the code yourself, you ask the AI to make it very specific, detailed in certain places. As soon as there is some uncertainty, the AI makes a lot of a lot of a lot of code that is not needed. >> Uh, >> well, I had such a problem. >> A counter-question right away, just in case, maybe you'll answer it immediately, so it's clearer. Where does the uncertainty come from in a project that you own, that you're developing? Don't you know your project? >> Well, it turns out that I have, figuratively speaking, a task like this, uh, >> I have five projects at work, right? There are projects that I, uh, I wrote, there are projects that I didn't write, super badly written. And for example, there's a task to experiment with something. So I had some flow, I wrote it myself, and then I need to write validation. The validation consists of eleven stages. And I need to script these eleven stages. And, figuratively speaking, you say with a simple prompt: "Here are my eleven stages, the logic is here and there, make me a script." Well, and the agent goes to do it, to do some crap for you. It generates an unimaginable amount of, uh, unnecessary code. So you then try to fix it by hand, but this fixing process is very time-consuming. So it depends on the task, uh, how much it overdid it. So, if you have a large task and it's relatively new, then to generate normal code, not code that you'll have to rework for a long, long, long time, you need to provide good context as input. That's what we're talking about. >> Well, I have a team of different people. So the manager has their own set of requirements, one colleague has their own set of requirements, someone else will have another set of requirements. And when you generate with simple prompts, it gives you code that doesn't fit into their code. So I know what the code should be like for it to pass code review. But to get that code from the LLM, I need to spend a lot of time and tokens to achieve this goal. And in SpecDriven, this prototyping stage, which used to be in code and was generated, I don't know, by thousands, this code, it shrinks into a research MD file, and then in the spec, I discuss a bit with the LLM what additional code it needs to write, because it writes test cases that it plans to handle. So it plans to catch some errors and inserts them into conditions, it invents some cases. I tell it: "Don't invent anything at all." Well, not that don't invent anything, like, you need to handle these two cases, try to get the rest from the root somewhere. >> Understood. >> And then, as a result, I have colleagues who require a certain architectural structure. And I can control this issue at the stage of these implementation details. But the artifacts that I create, so you do it based on the codebase, so it puts the necessary context there, then you make a spec, where it asks you questions about what exactly you want at the level of functional requirements. So these are super short things. And it requires very few tokens. And the next stage is the actual implementation, where you can also write code. >> No, how it works is clear. We've discussed it many times and tried it. So, yes, >> yes, but the point is that the token consumption is super small. So colleagues who sit next to me, they try to use, uh, CodeLlama, they wipe code, that is, they just write prompts, then review the code, and then they don't like something, but their stage is not that they don't like it, but rather that you send it for review and there are 10,000 lines of changes, and, well, like projects can be thrown away and accepted, right? And in essence, you can't put this into existing projects anymore. So that's why a lot of time is spent, and people don't use this AI as much as they could. And on large projects, your agent can work for hours. Well, there's a colleague sitting next to me, he's working on a large C++ project, which is just millions of lines of code. >> Okay, let's, let's summarize, because Valera still has his hand up. >> Uh, yes, yes, to summarize, that you spend very few tokens. Your tasks are not very defined. Even if you, um, know the code, you need to indicate how to implement, what exactly to implement, what cases to handle, what not to. You can greatly simplify the solution at these stages. Uh, and so it's like a mental model for a person, which also greatly increases the quality of the result for LLMs. reduces token consumption. And, uh, what else will be there? Well, accuracy of the result, well, speed too, because based on this final artifact of tasks, it's very fast to generate code. So it's like >> there, I don't know, half an hour, an hour, and you have a whole project ready. >> Thank you, Valera. >> Ugh, I waited. Well, I've said many times that these are taxi costs, but in essence, what Denis and other guys have described, and what I want to say next, is that these are all just different approaches to this context engineering. And what I want to say, I've already written about it in the chat.
Recently, I don't remember which one, in an agent's one, it seems. Ah, but I suggest to you, guys, who practice this spectkit and other context engineering techniques, try to approach it the way I do. I literally wrote an article about this today and >> you'll find a link >> there's a link in the external content to the article on Likedes. >> Uh-huh. Uh, and there are two main points that I advocate for everyone, and I'll explain in more detail now. The first point is set boundaries. Specify limitations. And the second point is, uh, create context. Well, in short, first, I'll explain the rules. Everything that you, Denis, told about conditions, about some limitations from teams, architecture, and so on, all of this can be written into rules. And any request to the agent will know this context. >> When you say rules, do you mean rules, for example, in Cursor or just any of the mechanisms there, like System Prompt, it doesn't matter. >> The point is to set a box within which the model operates. So that it doesn't write this noisy code that you're talking about, where it starts drawing something all over the project. Limit it as much as possible, give it boundaries on how it should write, where, and what. Uh, and the second point, when I give it a task, well, I'm speaking from my experience, I give it, oh, if we take an abstract example, in such a way as if I'm handing over a task to a junior who is staying in my place when I go on vacation. I transfer the entire context of the task. That is, for this, I use voice input, free text, absolutely everything I know about the task, all the constraints, some things, you describe everything and that's it. One prompt, the task is ready for deployment to prod, as a rule, that's how it happens. These two points are very important and I don't know, they can even replace your approach. Just try to do it this way. >> Uh, okay, Vitalik, then Alexey, >> yes? Ah, well, in fact, he's right, but on the other hand, well, that is, Spectkit and similar things are precisely aimed at forming these rules, these boundaries. That is, with the help of prompts, with the help of interaction with the LLM, we prepare these boundaries. That is, we don't write them manually, as it were, but, and we formulate them according to the rules of prompting, right? And other prompts help us do this. That is, in principle, well, there is no contradiction here, and there is no, let's say, opposition. It's just a convenient way to create these rules, these boundaries. >> Uh-huh. Uh, and I also wanted to say, like, there's another approach to what spectflow or spectit does, and that's the BM method. Their approach is a little different. They don't do it through specifications, well >> they have a team of agents, right? >> Yes. They have, well, that is, spectkit is a sequence of activities. That is, we have prompts that describe activities in order to form a specification or these boundary rules. And in BM, there are agents, well, that is, there are agents who play specific roles, that is, this prompt framework is built on a slightly different principle. And if spectkit is more suitable for full-stack developers who develop from requirement elaboration, from task setting to delivery, then BM, in my opinion, is more suitable. And I don't have a specific example yet, but it seems to me that a separate specialist can work with a separate role. And, accordingly, team collaboration can be organized using this team of agents. >> It seems like BM is good for more complex and cumbersome tasks. No, >> it's hard to say. There, well, there's its own specificity, right? That is, you're like communicating with a specific, uh, well, that is, either you play the role of a manager, right, who interacts with each team member, right, and tells him what needs to be done. Or each individual specialist talks to their partner who helps them prepare artifacts for the next stage. >> Uh-huh. Yes, thank you. >> Well, something like that. Ah, >> Alexey, >> so, well, I probably won't strongly oppose anything, rather I'll tell you an alternative option. So, in my example, it's some greenfield project where I still fully own the codebase. My approach is to build the codebase in such a way that the LLM always has a good example to rely on. I practically have no pre-made prompts. I often look, understand that something can be restructured to add a new feature, requires a light refactoring. I do some smart machinery by hand first. Then I unleash it, and say: "Make me a second one next to it, a second folder with the same approach." I look at what it coded. Again, I identify some common parts that can be scaled and say: "Okay, now follow this pattern." >> Well, this works well when you own the code and when you have examples to show so that it can do it by example. >> Yes. Yes, it works well. In our case, >> in this approach, it's the opposite of preparing some pre-made prompts, specs, because the structure changes very often, it would be even longer. >> Well, there's also a third approach, to be fair, I'll mention it. Valik talked about it, by the way. Valik is still with us, probably already left for the workshop. He showed how he spent 5 or 6 hours creating a single prompt to implement a whole service for working with the store and PL/SQL. This is the micromanagement approach, when you describe the task from start to finish, almost entirely without code implementation, and the LLM simply and calmly writes code according to it. Well, this is when you know very well what you need, you know your codebase well, and you don't want to write code manually. Well, there's also that option, Denis. Well, I, in principle, have a similar approach to the previous speaker, that I already have a codebase, I already have patterns, it's just that there are many of them, so you can't fit them into a cloud MD or Wagons, it's too verbose. And it turns out that the method is quite flexible, and it also allows referencing these examples. The LLM extracts these patterns and puts them into its research or implementation plan. Uh, but I agree with Valery that it can be done simpler. That is, I read that he described it with boundaries, with context. In principle, I work with this method. That is, I have some kind of free prompt, into which I throw everything, and then I just run the flow. Probably, this idea that I need this flow, maybe I, as a person, need this flow to understand what I want to do myself, to reduce the scope, for review, and then, because I will know what others will demand from me, and to adjust and control it, but putting it in some external places. I haven't succeeded once. That is, either you have to write too much, that is, you have to do half the project, because we have features, but they are also of varying degrees of maturity. That is, there are some patterns, but there are many of them, because the tasks are different. That is, it's not that you just add features by similarity. And I also wanted to draw attention here, thank you everyone who spoke, I wanted to draw attention, we also have guys who are involved in web coding, who are engineers or not engineers, but don't deal with code. Don't you all feel, in principle, that SDD, and prompt engineering in general, is moving towards us moving away from writing code manually? All these methods are needed so that we move from writing code to explaining the task. From this point of view, then for web coders, and for engineers without programming education, software development becomes more open. It will possibly be, in principle, one of the directions of their work in the future. >> When that moment comes, then we will become unnecessary. Fear it. Well, in fact, when you describe the entire structure through architectures, specs, what else, uh, well, you describe everything in words, you don't write code there, you only describe the architectural vision, and often for not very complex tasks, you don't even need to review the code. It's either written well, or after several iterations, it's written well, not well, working. >> It's very important to have short iterations. The feedback loop should be clear to you when you go through the architecture and so on, you turn coding into a workflow. >> Well, it's interesting, it's interesting how much longer we will need to be able to read code. Not even write, but read. >> Can I add? >> Regarding what Valera said, we will be unnecessary. I think that's definitely not the case. And I think, firstly, the market is becoming wider, and secondly, yes, less coding, more engineering and doing. That is, before, I used to look at coding and drool, wanting to get into it, but I had to do other things to earn money, and I didn't have time to get into it. Yes, I started doing it. But now I realize that, well, I can't do it full-time. I have other responsibilities, there, in the company. And now, while I'm studying this topic, I'm waiting for them to post vacancies. I'm a web coder, I can deploy projects in a day. And I would gladly hire such a person. And moreover, I've already interviewed one person from the club, and, well, we're currently in negotiations. Great. >> Therefore, and, well, that is, again, the thought is that it has reached me as a representative of small business and there are many such businesses. And vacancies will only increase, and you'll have to write not enterprise projects, but maybe a local ERP system for a specific project. Well, why not? Now this is possible. >> I also like that, look, you can actually set up this specflow yourself. It's not some super technical thing. The only technical thing is to go to GitHub and read with your eyes how to implement it in Cursor or something, you said today, and you implement it. Your tasks are already at a higher level and performed more qualitatively. This is exactly what you asked at the beginning. Can you set up agent teams yourself somehow without engineering? Well, you can try to set up BM. You mean by that that a developer isn't needed, but you can set it all up and do it yourself? Yes. Yes, yes. >> No, but you still have to maintain it, and test it all. How much time does all this take? It's easier for me to delegate and pay for it than to focus my attention on all this and lose out elsewhere. >> Uh-huh. >> Yes, I can now. I'm already doing some projects. For example, I made a payment calendar, integrated it with our ERP, made exports. Now I can calculate finances. My warehouse doesn't have such a thing under the hood. But there's just no person yet. I need to look, I need to hire. It's not that popular yet. You ask many people, and no one knows what Cursor is, what code-code is. But hiring a standard specialist from the market is expensive, in my opinion. So I want this balance. >> They're already offering web coding services in our chat for MVP. Pay attention, Ada Oleg, >> I'll read all of this now. I'm not an ADHD person yet. I can't read three chats at once. But I'm developing it. I feel that these daily news are leading there. >> Yes. Okay. So, Denis, let's give you the floor again on this topic and move on. >> Yes, I repeat, I don't write code anymore because, well, I write these prompts and read them. >> Well, you read it, you read it, as it were, write. Okay, you still read it, you understand it. I read a lot, yes. I have a speed reading skill from school, I took courses, they were very useful. Now, uh, I wanted to say that non-coders also do quite a lot. The effects I observe are that people who know what needs to be done can now do it in an evening. Staff engineers, principal engineers create very useful tools that teams then use. Before, they would have hired juniors, mid-level developers for them, now they are effectively covering this with web coding. My manager also web codes in his breaks. That is, if he sees a clear task, and needs to move JSONs, compare them, do some CLI, then he does it very well. He's like a designer with a business focus, he understands very well how other designers work and so on, and can create a good, understandable tool without all this communication, that he needs to write requirements, that he needs to prioritize with someone. That is, he just sits with Cursor, >> and web codes >> and plus, he completes tasks that he would have delegated before, while we had free space within the team for low-priority tasks, now he can do it himself. That is, to install some security fixes, fix some minor bugs. That is, well, if it's good, if your manager is from a technical background. >> M >> Designer. >> Well, it's a technical background. In the sense, it's not someone who didn't grow up from business analysts, for example, or from tech writers? >> No, he made maps in Photoshop. >> Ah, ah, okay. Alright, I take my words back. Good. >> Well, he's technical, that is, he knows how to do and edit JSONs. Understood. >> And run the CLI in the terminal. But, well, that is, his specification is to draw maps in Photoshop. >> Thank you, Vitaly. Further, I wanted to add that, well, probably for simple applications like calendars or, say, statistics collectors or something else, or, well, some daily task trackers or something else simple enough, an engineer won't be needed. But there still remains a place for, uh, large systems, complex systems that must take into account many nuances. And such systems, well, cannot be web-coded. For them, you need to maintain specifications, documentation. And, well, in any case, engineers will be needed. Such systems are not built in an evening, or a lot of man-hours are invested in them. And plus, as I already wrote in the chat, well, and it doesn't need anything. As if it doesn't have a "stinger in its ass" to do something, to invent something. >> For now, >> yes, it won't, Lesha. If, if it needs to develop something, well, not today, it's not the thalamus and cerebellum, right? That is, there must be a system of acceleration and a system of braking. And this system must make mistakes. People, we make mistakes, because we are capable of making mistakes. We don't want AI to make mistakes. We want it to work predictably, to invent something new and do something differently. For this, it is necessary to make mistakes. >> Well, you've slightly changed the meaning. As it seems to me, we as developers, yes, we want predictability of answers and for the LLM to do what we say. But LLM developers want them to solve more complex problems. They don't aim to make LLMs respond interpretably. Yes, they want them to solve complex problems, but their users, well, that is, those to whom they sell their systems, they want these systems to give a 100% guaranteed result. >> Well, I, as >> which is always good, they won't accept it generating a bad result. And humanity has progressed in its development precisely because, well, that is, we made a million successful experiments because we made 10 million unsuccessful experiments. The model made unsuccessful experiments a trillion times during training. My point is that interpretability is not the main thing at all. The main thing is to provide new knowledge and quality solutions. And in this system, anything can happen with these things. We don't know how models work today. Doctoral theses are written on interpretability. Well, not doctoral, but scientific papers. >> No, the point here is not interpretation, but quality, some kind of >> quality, >> yes, no, in in freedom, well, uh, uh, in freedom of choice, right? That is, and a person wants something because he, um, is free to choose, he is free to do what he wants. It's impossible to give a person a task, right, well, that is, to say: "I need this." He already has a frame. Exactly the same as with AI, right, a frame appears. And then the one who set the task expects the result within this frame. >> Yes, I understand. >> I won't go further here, otherwise we'll get into philosophy. >> Well, perhaps, of course, models will want to do something someday, but now they react to, for now, we are operators, we are the trigger that models, >> yes, they react to our prompts. There's a question for Valera about his method. Does he have any templates, or do you just come up with a template for each task, boundaries, and then each time you throw in random context or dictate it. The thing is, I don't come up with boundaries every time. These rules are project-specific, that is, they are general project rules. No matter how many there are, it will still be less than the original super-prompt of the agent. Therefore, I describe the rules as specifically as possible, how to write code, what to write, what structural and architectural, uh, important things there are, which, if a new person came to the project, I would need to fully onboard them so that they know how to write this project. This is what I describe in the rules. >> Damn, I can't let go of the tasks. >> Well, I treat each task, as I've already explained, as a complete separate workflow that I also hand over to someone. And, operating with these two entities, almost every time, if the model itself doesn't get confused and mess up, it performs the result in one shot. >> Well, I have another question for you. Denis wrote a lot about this in the chat. Let's raise it vocally and discuss it a bit more at the end. Don't you think that specflow generates a lot of situational files that are only suitable for a specific large feature? And therefore, specs should not be stored in code, as is currently done in MD files, but, for example, somewhere in Jira, where business requirements are stored, and pull them with some MCP if desired. Plus, it will be much easier for non-technical personnel to work with these specs in Jira, who don't know anything about the IDE and don't navigate it. >> Well, I don't store them in code for myself at all, because they are too large artifacts. That is, research and these task MDs are very difficult to read. >> So how do you include them in the context if you don't store them in code? >> Gitignore. >> Ah, Gitignore. >> No, no, I have a separate folder, a separate repository where I store these specs and commit them there. That is, I don't store them with the code, with the code I have a PR formatted according to team rules. And the code should be clear about what it's doing, broken down, with comments if something is unclear or controversial. And the spec itself, well, it's not interesting to the team. That is, they don't want to read intermediate artifacts. And it's unclear how to use them at the moment. Well, in other teams, I've read, these artifacts are meaningless. That is, especially if the templates are not human-oriented, there's just an endless amount of garbage, which is written in the guides for the LLM to successfully complete tasks. Well, I clean it up from my prompts, but no one is going to read them. There's just this desire in some companies and some development teams to create a common infrastructure of MD files and create a supposedly common specrin approach that will suit both the frontend developer, the developer, and the tester. One repository from which you take the context file you need. >> I have little faith in such an approach. I fundamentally don't try to share any of my rules or anything else, because they really become outdated, they change from model to model. They need to be adapted. Each developer has their own approach to working with models. They all use different setups. >> It will just be a worse garbage dump than outdated documentation. >> In short, Gitignore or a separate repository. >> Well, I have an idea. I also want to web-code myself something that will commit to a separate repository, and, that is, I have, for example, a folder for the working project repository. And these files should not be committed to the main project, but to a side one somewhere, which is purely for me, purely for me. >> Understood. Okay. Then, if no one else has any topics, it's exactly 12:12. Thank you all for coming today. Special thanks to everyone who participated in the debates, both textually and vocally. Today was very productive, I think. And see you in 2 weeks, I think there will be Okay, I don't think so. Everything will be fine there. We'll meet in 2 weeks. Alright, bye-bye. Mishal. >> Thank you all. Bye. >> Bye-bye. >> Bye everyone. Bye everyone.