Transcription
To the next question. The next question. Let's take the next question about using SPEC driven development. And you, you said you would talk about your flow. I here a little bit, well, maybe you add your comments, then I'll add a little more comments and we'll listen to your experience. This topic was born as a discussion in the chat, about how to use it at all and to look at different approaches that different chat participants use, because there is a hypothesis that it greatly improves the quality of agent coding, especially on medium or large tasks. >> It seems to improve the stability of answers, as far as I understand. The repeatability of answers even. The quality, the repeatability of the quality of answers. Well, let me give a quick intro. Spect Driven Development is an approach in AI development where the specification becomes, in fact, the paramount artifact. And thus, the wipe coding effect is eliminated, when your context is constantly different, in each task it is small. You thus, by operating with context, clearly control what is fed into the input, and, in principle, can expect some guaranteed quality of the result. Uh, I divided this into two parts. There is a camp of people who believe that Spect Driven Development is based on tools like frameworks. Uh, there is functionality in Kra Spect Driven Development, uh, and there is in QPC Driven functional, where the AI itself creates a development plan, the design of your features, you approve all this, and then development begins. There are other guys who adhere to the fact that Spect Driven Development is, in fact, an approach. They are just tools. There, as far as I understand, there are now two leaders, maybe even three already. This is GitHub's Spekkit, which implements Spec Driven Development through standard scenarios, similar to how it's done in Cairo, spec, plan, tasks, execution, and the BM method, which creates different roles. Uh, what we were just talking about, it creates agents with different roles, and they try to process their own task with different roles and produce the final result. I also found an interesting implementation, uh, more similar to Spec Kit - this is Open Spec. This little library. I'll send it to the chat. It, in principle, does the same thing as Spect. But in the end, as for me, all these libraries and all these approaches are about context management, about writing MD files, in fact, which are put into the context. MD files on how to work with context. Well, Denis, now it's your turn. >> Well, yes. Uh, here I have an evolution, I also wrote in the chat, the evolution was from trying this approach in Cursor, because they promised that it gives great results, but in the end, Cursor, you used it through context 7, probably, right? Or >> No, I purely used Cursor, but that was a long time ago, back in the spring. Uh, the point was that it prepared a complete plan, and then it couldn't execute what was written in that plan. So it was pointless to use it. At least, I didn't get anything at all. And now I'm succeeding at work, there are guys there, they make agents, they don't write code by hand at all, and they have a template like this. Or, let's say, I made a template based on the specs they write. They have one file, which contains, uh, a small research, uh, a small spec, small tasks, uh, and some other garbage that everyone says needs to be added. So it's impossible to read what it adds. So there are some litigations, some estimates it writes, like, well, it writes klokod. With this thing, the coding quality has significantly improved, because you can work with this file and, in principle, develop small features like this. Uh, but the result didn't satisfy me, because the file is large and there's a lot of garbage, it's impossible to review. Uh, my next evolution was that I used GitHub's Spekkit. Uh, the problem with GitHub's Spekkit is that it generates huge files. >> And how does working with Spekkit look like? Is it just recommendations, or is it a library that itself monitors files, fills them in itself? >> Well, in short, this BM and Spec Kit are like several elements, so there are prompt templates that are used for writing these artifacts like research, specification, and so on. Around these templates, there is a wrapper for agents like klokod, codex, and so on. >> So, like ready-made settings for specific agents, right? >> Well, there are commands, it generates commands >> and then there is a wrapper in the form of some scripts. This is in Python, in Bash, in JavaScript, in BMDE. Uh, based on these scripts, it runs the flow of this spec driven development. But the point is that they all, both Spec Kit and BM, want some kind of mega-universal flow that can be done for any project. This is super unclear. Well, I'm looking at this diagram, I don't understand anything. Well, this is a description of how a typical software team works in outsourcing. Like, >> well, like, yes, but it's complicated. I, uh, well, I needed a simpler tool. So, in the end, I took what Spekkit generated for me as final templates and final artifacts, and based on them, I created a similar flow, similar templates, but for my stack, for my projects, and maybe iterated on them a bit to refine these templates. And I have four commands left, like research. Research does something like search, which does perplexity, or you do it yourself when you research a task. So you have a task like refactoring a project, using a new library, like some cache library. You say, like, here's this library, it needs to be used in these modules, in this project. Do a research for me, ask it some questions, what the library does, how it can be integrated into my project in the least verbose way, all that kind of crap. >> And you're using klokod for this, right? Yes, it generates a research file, and it's like a canvas, similar to some ChatGPT canvas, which you interact with and say, like, I need this module, I need TTL, some other stuff, just like a brainstorm, like a rubber duck. >> I have a question right away. By the way, UmpuTun in the last issue of Radioti highly praised Anthropic's skills and says that you can also use skills in klokod and that a good selection of skills for a project almost replaces all Spec Driven Development approaches. Your agent just knows what to do. You just load the necessary scripts, the necessary description into the skills. Have you tried skills? >> I tried, but I had a problem that klokod didn't write for me. a fairly simple skill. I struggled with it for a day. Uh, I needed to write an analog of a trace. So I have environments, I have entities, and in DynamoDB lies the entity, on S3 lies some derivative artifact of this entity, like an image, figuratively speaking. And I have this information on staging, I have it on production. I asked klokod to make a skill. So I already had the full description for such a skill. I had a script, but my colleagues said, why add such a script to the main codebase, like re-engineering. I thought about making a skill. Well, and he struggled with this skill. But in the end, he just didn't create it. So they had a promise that you can give it the ability to write code itself. Well, it, uh, it can't wrap it itself, and sit and figure it out, like, basically, a skill, as I understood it, is some kind of prompt, and you have to give it some super simple script, or something super understandable. And if something is not very understandable, then nothing works. So, in the end, these skills, well, probably useful. Uh, and skills, well, in short, if we talk about Spekkit, then I had this thing that there are templates, templates are adapted to the project, there are commands that can be added to the project itself, and they will help you run through this flow. In the end, you work with MD files. Uh, in principle, the biggest stage for me now is research, uh, and everything else, in principle, is generated from it. The spec is super small for me, about 100 lines, which highlights some user stories, considers some race conditions. In the templates of GitHub's Spekkit, the prompt is set up so that it also asks you test cases, meaning it asks you questions when something is unclear to it on some issues, and you answer them while working on the specification. So there are useful questions. Plus, when working on the specification, you can see where the agent starts doing some crap. Meaning it's doing something you don't want. So you fix it there. Then it makes an implementation plan. The implementation plan is the code itself. You write in the template: "Write me pseudocode, don't write too much, tests are like this, this and that." It makes this implementation plan for you according to this template. And based on the implementation plan, it then makes tasks. In tasks, it makes a graph of mini-tasks. So for a large task, there will be about 100 subtasks. And there, what it can do and call sub-agents already, like, it determines what can be executed in parallel. So, figuratively speaking, in the end, you have a feature task that fits into what you write as research, you control that based on this research it made a spec, and an implementation plan that can be read, so it's about 100 lines. >> It's still unclear why all this was highlighted as a serious separate name, Specflow. Well, before that, we had a ban that also tried to claim a single term. Now we have Specflow, although in fact it's just an iterative improvement of your prompt, which explains how to work with context. This is, in fact, prompt engineering, only prompt engineering with brains. >> Well, not exactly, you see, this research is prompt engineering, context engineering, which you describe. So a large document that creates the structure of your project for your specific task with specific file references, with, >> well, meaning the agent does the work, explains to the agent the places needed for work, shows the agent the most essential information needed for working on this project. Well, yes, but it's not new at all. >> Well, it rather collects context. And the spec is rather for a person to understand the task. So, it, uh, you >> not What is a spec in the standard development process, that's clear. >> Well, in agents, in agents, it looks like nothing new, just additional context about the project, automatically updated. I don't see this difference. I'm just trying to understand, is this some new direction, like, I don't know, a new way of developing AI that will be with us for another 10 years, or is it just a new name for ordinary human prompting, when you normally manually provide context and you get tasks written by agents normally. Well, it's hard here, here >> here >> now money will finish, then te >> Yes, the problem here is that when you have a complex task, a complex one, there is one that you as a programmer would do for a week or two, you would write, I don't know, a couple of thousand lines in a complex project, well, in a complex, medium one. And you have the problem that this feature is not solved by wipe coding. Meaning you can't solve it in a wipe coding session at all. So it wait, and what do you mean by "wipe coding session"? I spent 12 hours writing a prompt that, with one run, gave me a medium-complexity feature. Is this still web coding, or is it already? >> Well, this is more like a spec. What you wrote a prompt, >> based on which you generated code, >> and this is what I would call the essence of this spec driven development, because in the end you get a large artifact of 1,000, 1,500, 2,000 lines. >> Right. >> Well, well, I disagree, Denis. Just a while ago, with one prompt, with three lines, half an hour ago, before your eyes, I got 500 lines. This is this is not wait, we are talking about different things here. I'm saying that spec driven, this flow, it allows you to get an artifact in the end. The artifact is a markdown file >> with a prompt, which you then give to klokod. Understood. Inside this prompt, it's written which sub-agents should do what tasks in what order. >> Well, and for that, there's already a name for two years, it's called metaprompting, when you ask the model to create a prompt for you based on context that would be better suited for the task. >> Well, this is metaprompting, yes? So I think that But it's laid out in a methodology >> Uh-huh. Okay. >> A certain one, that you have any task. You fit any task into four stages, like research, implementation, >> design, >> no, not there, but research, spec, implementation, and tasks. And you can, >> well, a small to medium-sized task >> solve. At the same time, if you understand, well, if the task is clear to you, then this flow is super fast, and in the end you get the result that is described, >> what you expected, like a controlled result. >> Yes. And this is the main value. >> Let's go. >> Our other other methods did not give me this predictable result. >> Uh-huh. Thank you. Let's give Vitaly the floor. Vital, Vitalik, hello. You were the one who asked me a question in private messages about Xflow and BM. Can you, can you tell us? >> Not about Spec, about Spekkit. Sorry. Yes, >> Spec is GitHub. That's how the team develops. Well, here I would probably say that Spect Development is one of the approaches to context engineering. And context engineering, let's say, is the practice of managing and limiting context for LLMs to get more predictable and adequate responses. Right. >> Uh, and, accordingly, context engineering is a subset of prompt engineering. So it's all prompt engineering, but it's like a separate chosen direction, separate practices. >> But doesn't prompt engineering also include context squeezing? >> Prompt engineering is a very large, it's a very large area. So it's not a metric. Well, it's like LLMs are part of and are part of, well, more >> it depends on how you look at it, Carpati has completely abandoned the term prompt engineering and said that it's all context engineering >> here. >> Because we need to manage context. We have an unlimited window, even people themselves, uh, well, just look at how engineers work, right, we break down the global application task into small subtasks and focus on a small volume, well, on a small task. We cannot simultaneously keep the entire application in mind and always program it. >> Listen, from that point of view, won't it be the opposite, won't prompt engineering be a subspecies of context engineering? Context engineering includes, among other things, engineering and prompt creation, and building context around the project, including for the operator who writes these prompts, and for the agents who use them. No, >> no, look, prompt engineering is a sufficiently broad area, because it contains both one-shot and iterative approaches. Context engineering is more focused on an interactive approach. That is, we create a kind of artifact in one iteration to make the next iteration. That is, we don't give the LLM a global task: "Develop this feature for me." Right. And let's discuss and develop it immediately. We first discuss what this feature is. >> Okay. So, context engineering cannot be single-prompt, which are non-interactive single prompts. Well, here's the approach, we gradually, by communicating with the model, create intermediate artifacts, based on which we then create code. >> And >> the same as >> as before, we did it with a team of people, right? So business analysts prepared the specification. Based on the specification, the architect or tech lead prepared the solution. From this solution, the team then planned, broke it down into tasks. Developers did it, testers tested. >> Okay. Let's assume we've more or less settled on the theory. I'll throw in a question right away. If anyone has anything to add to it, jump in. The question is, how much is this, how much is such a token-heavy approach? Acceptable, damn, I forget words. In short, how much is it, uh, suitable, it's not heavy, it's much more economical. >> Why is it economical? You have, you have a lot of things running to do a medium-sized task. >> Uh, Lesha, please tell me, how many attempts will you make to create one prompt that will do it for you in one go? >> Well, one. One. >> One, >> yes. >> To, like, a feature in a normal, like, serious project, >> right? Yes. No, we're just forgetting that we're not wipe coders at all. We make one prompt, and then we fix it. Nothing will break off me. I made a prompt and I fixed it. >> Uh, >> well, look, it depends on, uh, what volume of feature you're making. If it's to fix, uh, say, the name of a button or the color of a button, then, well, you really don't need to write a specification. No, it's not about the color of the button, of course, to write some feature in 30 files that you would write for a day and a half, it will do it all for you. >> And you want to say that you can write a prompt that will execute it for you at once, and then you will spend several days sorting it out? >> No, I'll spend 2 hours sorting it out. Literally, look, I have about 15 files generated in Cursor right now. To check them, it takes an hour. I would write them by hand, well, about 700 hours. The prompt was literally three lines. So, if everything worked out for you the first time. 100% it didn't work out. There will be 100% bugs. But I know my code. >> And what will you do? Read it, debug it with these bugs, and fix it. >> Maybe by hand, maybe with additional prompts. Yes, of course. Well, I don't know, like, even in the past, uh, even simple bugs required time to fix them, to debug them. You also need to look, like, how to fix it. So, if, of course, the logic is simple, then yes, easily. But if the logic is not simple, if you're making some kind of combine that processes data, well, where's the logic, well, you have equations there, well, a couple of hours. >> Hmm, let's hear more opinions from the guys with raised hands. Let's start with Denis, then Valera. >> Well, in short, you, Alexey, don't see the problem, so it's hard. But this is a common thing when you discuss it. In short, I wrote, like you're describing, so I made some prompt, the code changed. Uh, and in principle, this approach works for small tasks that are understandable, and you know where to fix. So, instead of going to the AI to fix the code yourself, you ask the AI to make it very specific, detailed in certain places. As soon as there is any uncertainty, the AI makes a lot of a lot of a lot of code that is not needed. >> Uh, >> well, I had such a problem. >> A counter-question right away, just in case, maybe you'll answer it immediately, so it's clearer. Where does the uncertainty come from in a project that you own, that you're developing? Don't you know your project? >> Well, it turns out that I, figuratively speaking, have a task, uh, >> I have five projects at work, right? There are projects that I, uh, I wrote, there are projects that I didn't write, super poorly written. And, for example, there's a task to experiment with something. So I had some flow, I wrote it myself, and then I need to write validation. The validation consists of eleven stages. And I need to script these eleven stages. And, figuratively speaking, you say with a simple prompt: "Here are my eleven stages, the logic is here and there, make me a script." Well, and the agent goes to do it, to do some crap for you. It generates an unimaginable amount of, uh, unnecessary code. So you then try to fix it by hand, but this fixing process is very time-consuming. So it depends on the task, uh, how much it overdid it. So, if you have a large task and it's relatively new, then to generate normal code, not code that you'll have to rework for a long, long, long time, you need to provide good context. That's what we're talking about. >> Well, I have a team of different people. So the manager has their own set of requirements, one colleague has their own set of requirements, someone else will have another set of requirements. And when you generate with simple prompts, it gives you code that doesn't fit into their code. So I know what the code should be like for the code review to pass. But to get that code from the LLM, I need to spend a lot of time and tokens to achieve this goal. And in spec driven, this prototyping stage, which used to be in code and was generated, I don't know, by thousands, this code, it shrinks into a research MD file, and then in the spec I discuss a bit with the LLM what additional code it needs to write, because it writes test cases that it plans to handle. So it plans to catch some errors and inserts them into conditions, it invents some cases. I tell it: "Don't invent anything at all." Well, not that not at all, like, you need to handle these two cases, try to get the rest from the root somewhere. >> Understood. >> And then, as a result, I have colleagues who require a certain architectural structure. And I can control this issue at the implementation details stage. But the artifacts that I create, so you do it based on the codebase, so it puts the necessary context there, then you make a spec, where it asks you questions about what exactly you want at the level of functional requirements. So these are super short things. And it requires very few tokens. And the next stage is the actual implementation, where you can also write code. >> No, well, how it works is clear. We've discussed it more than once and tried it. So, yes, >> yes, but the point is that the token consumption is super small. So colleagues who sit next to me, try, uh, to use klokod, they, uh, they wipe code, meaning they just write prompts, then review the code, and then they don't like something, but their stage is not that they don't like it, but rather that you send it for review and there are 10,000 lines of changes, and, well, like projects can be thrown away and accepted, right? And in fact, you can't put something like this into existing projects. So that's why a lot of time is spent, and people don't use this AI >> as much as they could. And on large projects, your agent can work for hours. Well, there's a colleague sitting next to me, he's working on a large C++ project, which is just millions of lines of code. >> Okay, let's, let's summarize, because Valera still has his hand up. >> Uh, yes, yes, to summarize, that you spend very few tokens. Your tasks are not very defined. Even if you, um, know the code, you need to, uh, specify how to implement, uh, what exactly to implement, what cases to handle, what not to. You can significantly simplify the solution at these stages. Uh, and so it's like a mental model for a person, which also greatly improves the quality of the result for LLMs. reduces token consumption. Uh, what else will be there? Well, the accuracy of the result, well, the speed too, because based on this final artifact of tasks, it's very fast to generate code. So it's really >> there, I don't know, half an hour, an hour, and you have a whole project ready. >> Thank you, Valera. >> Ugh, I waited. Well, I've said more than once that these are like taxi costs, but in essence, what Denis and other guys described, uh, and what I want to say next, it's all just different approaches to this context engineering. And what I want to say, I've already written about it in the chat.
Recently, I don't remember which one, in an agent one, it seems. Ah, but I suggest you guys, who practice this SpectKit and other context engineering techniques, try to approach it the way I do. I literally wrote an article about this today and >> link >> there is a link in the external content to the article on Likids. >> Uh-huh. Uh, and there are two main points that I advocate for everyone, and I will explain them in more detail now. The first point is set boundaries. Specify limitations. And the second point is, uh, create context. Well, in short, first, I'll explain the rules. Everything that you, Denis, told about conditions, about some limitations from teams, architecture, and so on, all of this can be written into rules. And any request to the agent will know this context. >> When you say rules, do you mean rules, for example, in Cursor or just any of the mechanisms there, System Prompt, it doesn't matter. >> The point is to set the box in which the model operates. So that it doesn't write this noisy code that you're talking about, that it starts drawing something all over the project. Limit it as much as possible, give it boundaries on how it should write, where, and what. Uh, and the second point, when I give it a task, well, I'm speaking from my experience, I give it, if using an abstract example, in such a way as if I'm handing over a task to a junior who is staying instead of me when I go on vacation. I transfer all the context of the task. That is, for this, I use voice input, free text, absolutely everything I know about the task, all the constraints, some things, you describe everything and that's it. One prompt, the task is ready for deployment to production, as a rule, that's how it happens. These two points are very important and I don't even know, they can even replace your approach. Just try to do it this way. >> Uh, okay, Vitalik, then Alexey, >> yes? Ah, well, in fact, he's right, but on the other hand, well, that is, SpectKit and similar things are precisely aimed at forming these rules, these boundaries. That is, with the help of prompts, with the help of interaction with the LLM, we prepare these boundaries. That is, we don't write them manually, as it were, but, and we formulate them according to the rules of prompting, right? And other prompts help us do this. That is, in principle, well, there is no contradiction here and there is no, let's say, opposition. It's just a convenient way to create these rules, these boundaries. >> Uh-huh. Ah, and I also wanted to say, like, there is another approach to what SpectFlow or SpectKit does, and that is the BMT method. Their approach is a bit different. They don't do it through specifications, well >> they have a team of agents, right? >> Yes. They have, well, that is, SpectKit is a sequence of activities. That is, we have prompts that describe the activity in order to form a specification or these boundaries. And in BMT, there are agents, well, that is, there are agents who play specific roles, that is, this prompt framework is built on a slightly different principle. And if SpectKit is more suitable for full-stack developers who develop from requirement elaboration, from task setting to delivery, then BMT, it seems to me, is more suitable. And I don't have a concrete example yet, but it seems to me that a separate specialist in that role can work with a separate role. And, accordingly, team collaboration can be organized using this team of agents. >> It seems like BMT is good for more complex and cumbersome tasks. No, >> it's hard to say. There, well, there is its own specificity, right? That is, you are as if communicating with a specific, uh, well, that is, either you play the role of a manager, right, who interacts with each team member, right, and tells him what needs to be done. Or each individual specialist talks to their partner who helps them prepare artifacts for the next stage. >> Uh-huh. Yes, thank you. >> Well, something like that. Ah, >> Alexey, >> so, well, I probably won't oppose anything strongly, rather I'll tell you an alternative option. So, in my example, it's some Greenfield project where I still fully own the codebase. My approach is to build the codebase in such a way that the LLM always has a good example to rely on. I have practically no pre-made prompts. I often look, understand that something can be restructured, to add a new feature, a slight refactoring is needed. I do some smart machinery by hand first. Then I unleash it, and say: "Make me a second one next to it, a second folder with the same approach." I look at what it coded. Again, I highlight some common parts that can be scaled and say: "That's it, now follow this pattern." >> Well, this works well when you own the code and when you have examples that you can show so that it does it by example. >> Yes. Yes, it works well. In our case, >> in this approach, it's the opposite of preparing some prompts, specs in advance, because the structure changes very often, it would be longer. >> Well, there is also a third approach here, for the sake of fairness, I will mention it. Valik talked about it, by the way. Valik is still with us, probably already left for the workshop. He showed how he spent 5 or 6 hours creating a single prompt to implement a whole service for working with the store and PL/SQL. This is the micromanagement approach, when you describe the task from start to finish, almost without code implementation, and it simply writes code for you. Well, this is when you know very well what you need, you know your codebase well, and you don't want to write code manually. Well, there is such an option, Denis. Well, I, in principle, have a similar approach to the previous speaker, that I already have a codebase, there are already patterns, it's just that there are many of them, so they won't fit into Cloud MD or Wagons, it's too verbose. And it turns out that the method is quite flexible, and also allows referencing these examples. The LLM extracts these patterns and puts them into its research or implementation plan. Uh, but I agree with Valery that it can be done simpler. That is, I read that he described it with boundaries, with context. In principle, I work with this method like this. That is, I have some kind of free prompt, into which I throw everything, and then I just run the flow. Probably, this idea that I need this flow, maybe I, as a person, need this flow to understand what I want to do myself, to reduce the scope, for review, uh, later, because I will know what others will demand from me, and to adjust and control it, but to put it in some external places. I haven't succeeded once. That is, either you have to write too much, that is, you have to do half the project, because we have features, but they are also of varying degrees of maturity. That is, there are some patterns, but there are many of them, because the tasks are different. That is, it's not that you just add features by similarity. And I also wanted to draw attention here, thank you everyone who spoke, I wanted to draw attention, we have guys who are involved in web coding, who are engineers or not engineers, but don't deal with code. Don't you all feel, in principle, that SDD, and prompt engineering in general, is moving towards us moving away from writing code manually? All these methods are needed so that we move from writing code to explaining the task. From this point of view, for both web coders and engineers without programming education, software development becomes more open. It might even be one of the directions of their work in the future. >> When that moment comes, then we will become unnecessary. Fear it. Well, in fact, when you describe the entire structure through architectures, specs, what's there, uh, well, you describe everything in words, you don't write code there, you only describe the architectural vision, and you often don't even need to review the code for moderately simple tasks. It's either written well, or after several iterations, it's written well, not well, working. >> It is very important to have short iterations. The feedback loop should be clear to you when you go through the architecture and so on, you turn coding into a trifle. >> Well, it's interesting, it's interesting how much longer we will need to be able to read code. Not even write, but read. >> Can I add? >> Regarding what Valera said, we will be unnecessary. I don't think so. And I think, first of all, the market is becoming wider, and secondly, yes, less coding, more engineering and doing. That is, before, I used to look at coding and drool, wanting to do it, but I had to do other things to earn money, and I didn't have time to do it. Yes, I started doing it. But now I realize that, well, I can't do it full-time. I have other responsibilities, uh, in the company. And now, while I'm studying this topic, I'm waiting for them to post vacancies. I'm a web coder, I can deploy projects in a day. And I would gladly hire such a person. And moreover, I have already interviewed one person from the club, and, well, we are currently negotiating. Great. >> Therefore, and, well, that is, again, the idea is that it has reached me as a representative of small business and there are many such, and vacancies will only increase, and you will have to write not enterprise projects, but a local ERP system for a specific project. Well, why not? Now it's possible. >> I also like that, look, you can actually set up this specflow yourself. It's not some super technical thing. The only technical thing is to go to GitHub and read with your eyes how to implement it in Cursor or something, you said today, and you implement it. Your tasks are already at a higher level and performed more qualitatively. Exactly what you asked at the beginning. Is it possible to set up agent teams yourself somehow without engineering? Well, you can try to set up BMT, in fact. Well, you mean that a developer is not needed, but to set up and do everything yourself? Yes. Yes, yes. >> No, but you still have to maintain it, and test it all. How much time does all this take? It's easier for me to delegate and pay for it than to focus my attention on all this and thus lose out elsewhere. >> Uh-huh. >> Yes, I can now. I'm already doing some projects. For example, I made a payment calendar, integrated it with our ERP, made exports. Now I can calculate finances. My warehouse doesn't have such a thing under the hood. But there's just no one yet. I need to look, I need to hire. It's not that popular yet. You ask many people, no one knows what Cursor is, what code-code is. But hiring a standard specialist from the market is expensive, in my opinion. So I want this balance. >> They are already offering web coding services on MVP in our chat. Pay attention, Ada Oleg, >> I'll read all this now. I'm not ADHD yet. I can't read three chats at once. But I'm developing it. I feel it, these daily news are leading there. >> Yes. Okay. So, Denis, let's give you the floor again on this topic and move on. >> Yes, I repeat, I don't write code anymore - uh, because, well, I write these prompts and read them. >> Well, you read it, you read it, as it were, write. Okay, you still read it, you understand it. I read a lot, yes. I have speed reading skills from school, I took courses, they were very useful. Now, uh, I wanted to say that non-coders also do quite a lot. The effects that I observe are that people who know what needs to be done can now do it in an evening. Staff engineers, principal engineers create very useful tools that teams then use. Before, they would have hired juniors, middle developers for them, now they are effectively covering this with web coding. My manager also web codes in his breaks. That is, if he sees a clear task, and needs to move JSONs, compare them, do some CLI, then he does it very well. He's like a designer with a business focus, he understands very well how other designers work, and so on. He can create a good, understandable tool without all this communication, that he needs to write requirements, that he needs to prioritize with someone. That is, he sits alone with Cursor, >> and web codes >> and plus, he solves tasks that he would have delegated before, while we had free space for low-priority tasks within the team, now he can do it himself. That is, to apply security fixes, to fix some minor bugs. That is, well, if it's good, if your manager is from a technical background. >> M >> Designer. >> Well, it's a technical background. In the sense, it's not a person who didn't grow up from business analysts, for example, or from tech writers? >> No, he drew maps in Photoshop. >> Ah, ah, okay. Well, I take back my words. Okay. >> Well, he's technical, that is, he can do and edit JSONs. Understood. >> And run CLI in the terminal. But, well, that is, the specification is for him to draw maps in Photoshop. >> Thank you, Vitaly. Further, I wanted to add that, well, most likely, an engineer will not be needed for simple applications, like calendars or, uh, statistics collectors or something else, or, well, some everyday task trackers or something else simple enough, but there is still room for, uh, large systems, complex systems that must take into account many nuances. And such systems cannot be web-coded. For them, you need to maintain specifications, documentation. And, well, in any case, engineers will be needed. Such systems are not built in one evening, or a lot of person-hours are invested in them. And plus, as I already wrote in the chat, well, and it doesn't need anything. As if it doesn't have a "stinger in its ass" to do something, to invent something. >> While, >> yes, it won't, Lesh. If, if it needs to develop something, not today, it's not the thalamus and cerebellum, right? That is, there must be a system of acceleration and a system of braking. And this system must make mistakes. People, we make mistakes, because we are capable of making mistakes. We don't want the AI to make mistakes, we want it to work predictably, to invent something new and do something differently. For this, mistakes must be made. >> Well, you've slightly changed the meaning. As it seems to me, we as developers, yes, we want predictability of answers and for the LLM to do what we say. But LLM developers want them to simply solve more complex problems. They don't aim to make LLMs respond interpretably. Yes, they want to solve complex problems, but their users, well, that is, those to whom they sell their systems, they want these systems to provide a 100% guaranteed result. >> Well, I as >> which is always good, they will not accept that it generates a bad result. And humanity has advanced in its development precisely because, well, that is, we made a million successful experiments because we made 10 million unsuccessful experiments. Even more, the model made unsuccessful experiments a trillion times during training. My point is that interpretability is not the main thing at all. The main thing is to provide new knowledge and quality solutions. And in this system, anything can happen. We don't know how models work today. Doctoral theses are written on interpretability. Well, not doctoral, but scientific papers. >> No, the point here is not interpretation, but quality, >> quality, >> yes, no, in in freedom, well, uh, uh, in freedom of choice, right? That is, and a person wants something because he, um, as it were, is free to choose, he is free to do what he wants. It is impossible to give a person a task, right, well, that is, to say: "I need this." He already has a framework. Exactly the same as for AI, right, a framework also appears. And then the one who set the task expects the result within this framework. >> Yes, I understand. >> I won't go further here, otherwise we'll get into philosophy. >> Well, it's possible, of course, that models will want to do something someday, but now they react to, for now, we are operators, we are the trigger that models, >> yes, they react to our prompts. There is a question for Valera about his method. Does he have any templates or do you just come up with a template for each task, boundaries, and then each time you add random context or dictate it. The thing is, I don't come up with boundaries every time. These rules are project-specific, that is, they are general project rules. No matter how many there are, it will still be less than the original super-prompt of the agent. Therefore, I describe the rules as specifically as possible, how to write code, what to write, what structural and architectural important things there are, which, if a new person came to the project and I needed to fully onboard them so that they knew how to write this project. This is what I describe in the rules. >> Damn, the tasks are not letting go. >> Well, I treat each task, as I already said, as a separate complete workflow that I also hand over to someone. And, operating with these two entities, almost every time, if the model itself doesn't get confused and mess up, it performs the result in one shot. >> Well, I have another question for you. Denis wrote a lot about this in the chat. Let's raise it verbally and discuss it a bit more at the end. Don't you think that SpecFlow generates a lot of situational files that are only suitable for a specific large feature? And therefore, specs should not be stored in code, as is currently done in MD files, but, for example, somewhere in Jira, where business requirements are stored, and pull them with some MCP if desired. Plus, it will be much easier for non-technical personnel to work with these specs in Jira, who don't know anything about the IDE and don't navigate it. >> Well, I don't store them in code for myself at all, because they are too large artifacts. That is, research and these task MDs are very difficult to read. >> So how do you include them in the context if you don't store them in code? >> Gitignore. >> Ah, Gitignore. >> No, no, I have a separate folder, a separate repository where I store these specs and commit them there. That is, I don't store them with the code, with the code I have a PR formatted according to team rules. And the code should be clear what it does, broken down, with comments if something is unclear, controversial. And the spec itself, well, it's not interesting to the team. That is, they don't want to read intermediate artifacts. And it's unclear how to use them at the moment. Well, in other teams, I've read, these artifacts are meaningless. That is, especially if the templates are not human-oriented, there is just an endless amount of garbage, which is written in the guides for the LLM to successfully complete tasks. Well, I clean it up from my prompts, but no one is going to read them. There is just this desire in some companies and some development teams to create a common infrastructure of MD files and create a common spec approach that would be suitable for front-end developers, developers, and testers. One repository from which you take the context file you need. >> I have little faith in such an approach. I fundamentally do not try to share any of my rules or anything else, because they really become outdated, they change from model to model. They need to be adapted. Each developer has their own approach to working with models. They all use different setups. >> It will just be a worse garbage dump than outdated documentation. >> In short, Gitignore or a separate repository. >> Well, I have an idea. I also want to web-code something like this myself, which will commit to a separate repository, and, that is, I have, for example, a folder for the working project repository. And these files should not be committed to the main project, but to a side one somewhere, which is purely for me, purely for me. >> Understood. Okay. Then, if no one else has any topics, it's exactly 12:12. Thank you all for coming today. Special thanks to everyone who participated in the debates, both text and voice. Today was very productive, I think. And see you in 2 weeks, I think there will be Okay, I don't think so. Everything will be fine there. We'll meet in 2 weeks. Let's go, bye-bye. Mishal. >> Thank you all. Bye. >> Bye-bye. >> Bye everyone. Bye everyone.