Transcription
Hello everyone. Several studies have already shown that while some developers get a two-to-threefold increase in productivity using AI agents. Others, on the contrary, lose speed and quality. Moreover, this almost does not depend on their skills, seniority, or the language they program in. The main difference is in the experience of working with and tools in general. Usually, people understand how to correctly set tasks for an agent somewhere after 80 hours of working with it. But what does it mean to correctly set tasks for an agent? It means when you understand that this particular task needs to be broken down into several parts, or that this piece is better done manually than handed over to it, and here it's better to specify the exact file instead of hoping the agent will figure out to look into it itself. Even the most experienced guys I've talked to often feel these things quite intuitively and just instinctively understand how to act in one situation or another. And I really like to formulate specific criteria. by what qualities, looking at a task, I make a decision about what to do with it at the moment. In this video, I want to try to save you these 80 hours and explain five key qualities that I pay attention to when analyzing a specific task to understand whether it needs to be broken down, reformulated, or perhaps some part of it should be done manually. By the way, my name is Danila. I've been in development for 16 years, the last 8 years mainly as a CTO co-founder in various startups. And for the last year, I've been programming with AI agents on a daily basis. And even before we get to the juicy part, the qualities we will discuss do not depend on the specific tool. It doesn't matter what you use: pure Cursor or Cursor Supercode, pure-code or a swarm of sub-agents, maybe OpenCD, Cline Rcode, it doesn't matter at all. The qualities I will mention boil down to the fundamental principle of how LLM models work or how modern AI agents for development are structured in principle. Let's go. So, the first quality I look at when assigning a task is how unique it is. What is meant here? It will probably be easiest to explain with an example. Suppose I have a front-end project in React with some user table where I display information about users from the database row by row. And I ask the AI agent to add sorting to this table by clicking on a particular column. This task, despite happening on my specific codebase with my table, is absolutely typical and standard. This is a task that has been implemented in millions of projects before me. A million different tables on the internet have the ability to sort by clicking on the column header. And in this sense, the AI's solution will be a very standard pattern. it won't be difficult for it. It has seen how this task is solved a million times. And the probability of it making a mistake here is practically zero. Now let's look at another, but similar task. For example, I have the same user table for my online store, and I want to add sorting by a specific score to it. I want users with VIP status to be displayed first, then users who made a purchase no more than one day ago. And for the remaining users, we will calculate a specific score. We will multiply, say, 0.9 by the number of purchases they made in total and add some activity coefficient for the last month. And based on this score, we will also sort the remaining users in our table. Each part of my task that I have just described is not complex. Each piece of this task is not difficult in itself. Each formula and each branching is not a problem. But when I combine all this, it turns out to be a rather unique cocktail. And if the AI, especially in some reasoning mode, manages to decompose the task into subtasks and implements them one by one, then everything will be fine. If it fails to do so, there will be big trouble. And this is exactly how this uniqueness criterion works. When you look at a task, you need to try to understand how much the solution you expect from artificial intelligence fits into some typical standard pattern that it might have seen tens, hundreds, thousands, millions of times before. And if the task does not fit into this pattern, it will most likely be very difficult for it to solve. What to do if you think your task is very unique and most likely has not been encountered anywhere before? Break it down into subtasks. First, implement each piece separately, and then as a separate task, ask the AI to combine them into a branched structure. Most likely, you will not encounter any problems, and it will cope completely from start to finish. If you give it a whole chunk, a huge, unique combination of your business processes, layers, and how your specific project is structured, the probability that it will stumble at one of the implementation stages, go in the wrong direction, and do something wrong, is enormous. And you need to try to avoid this as much as possible. Simply by paying attention to such a quality of tasks as the uniqueness of their solutions, you can already significantly improve the agent's work simply by using the decomposition I described. But this quality works perfectly with the next one. So, quality number two. Depth of immersion. How deeply will the agent need to immerse itself in your project's codebase to complete this task? It's important to understand that, like the previous one, this is not some binary criterion, yes or no, that if it needs to immerse itself, it will definitely fail. Naturally, this is absurd. If you have worked with agents even a little, you know that they are perfectly capable of reading even a very large number of your project files, hundreds, and then proceed to execution. But the more the AI needs to immerse itself in your project, the lower the quality of its final solution can be. The fact is that when reading files, it does not build some internal hierarchical tree of connections between the components of your project in its virtual head. It studies your codebase quite linearly, sometimes even in reverse order if the files appear to it that way. And all this affects the quality of its solution. Somewhere it might stumble and not read something important, simply not realizing that some important information is in that file. Sometimes it can overload its context and simply get confused by interfaces and terms, and sometimes it might reinvent something new because it simply couldn't find it in your codebase. You shouldn't rely on this quality. So, at all costs, you need to provide maximum information directly in the prompt so that it doesn't delve into the project at all. There will be little sense in that. It will simply stop saving you time. But giving it tasks that require reading half of your entire project's codebase is also not the best idea. With each new file read, the probability of its error increases, well, roughly by two percent. If it reads five files, then with a 90% probability, it will give an excellent result. But if it had to read 30 files, the probability of error increases manifold. And by error, I mean here not so much a non-working solution in principle, but rather a solution that will simply differ from what you specifically wanted to get. And the solution to these problems is approximately the same as for the first point. Decompose. If you see that you are giving a task in a too general style like: "Figure out how our project works" and add authorization to it. Such a broad formulation can be problematic. It's better to break down such a task into specific stages. For example, you can allocate a design stage where you ask it to do all the same, but before starting to write code, prepare a detailed plan on how exactly it intends to do it. Then you will have the opportunity to make two or three edits to correct its path if it missed something somewhere or, conversely, wants to go in the wrong direction. Often we set such tasks because we already have the entire context of how our project works in our heads, and we are simply too lazy to explain all this again. We want it to read the code itself and do everything for us, but unfortunately, the quality suffers greatly from this. Therefore, this manifestation of laziness must be fought, or at least the task should be broken down into several stages so that the AI genuinely tries to understand the project itself. But if it makes a mistake somewhere, you can correct it. One of the most typical scenarios of problems that the AI will encounter if you give it broadly formulated tasks is that during the analysis of your codebase, it will simply miss some files. And this is very similar to the same class of problems that arise if you ignore the third quality, which we will now discuss. So, the third quality. Does this task require knowledge that is not present in the project's codebase? As comical and banal as it may sound, I think this is one of the most common problems I see when helping someone figure out why an agent writes bad code. People proceed from the context that exists in their heads. For someone who is involved in a specific subject area, some things are so obvious that they don't even think to describe them additionally when they give a task to an agent and generally have an idea of the world we live in. But some specific narrow nuances of a particular field may be completely non-obvious to them. What's more, they might be non-obvious to the person you are trying to assign the task to. The problem of synchronizing one head with another is, in my opinion, one of the main problems of management. And given that working with an AI agent is largely about management, this problem smoothly transitions into the realm of human-machine interaction. Put yourself in the shoes of another person, in this case, in the shoes of artificial intelligence. Does it have all the knowledge that you possess? Is some nuance in the subject area as obvious to it as it is to you, or should it be discussed separately, for example, saying, the typical user journey on our site is like this, because the user wants this and this. In general, the process of creativity and feature description should ideally be separated from the development process. You can brainstorm and be creative in one chat, package it into a specific task, and then pass it to the agent that will write the code. But sometimes these processes are difficult to separate. And writing entire detailed documents for each task is a bit unpleasant because it takes away all the joy of delegating some routine to the agent. If you spend 30 minutes writing a task, what's the point? You could have done it yourself in the same 30 minutes. Therefore, for things that are already obvious from your codebase, it might not be worth overdoing it. But still, don't forget that some nuances of your subject area are best added directly to the prompt or to some global rule for your project. In the second quality, where I talked about the necessary depth of the agent's immersion in your project, I explained that it's related to how many files it needs to read and that this generally worsens its result. This quality has a mirror twin. Quality number four. In how many places does the agent need to make changes for the task to be completed? I think you already understand that the principle here is exactly the same. Even if the AI has come up with a competent solution and understands what exactly it needs to do, if this change is required in a whole bunch of files in your project, it can simply miss some of them. And this all falls into the class of refactoring problems, where even a seemingly elementary change like renaming a variable actually requires the AI to go through a whole check for attentiveness and not miss a single file in the project where it is used. Specifically in the case of renaming, it's not so painful. It can use grep and won't miss a single place. But when it's some really complex logic that requires, perhaps, not such a difficult change, but in a large number of places, the number of errors and omissions that the AI will make will be directly proportional to this very number of places. You can assume something like this: for every 10 files, it will miss one. Therefore, if you want to perform a refactoring or changes that will affect 20 or 30 files, you can safely count on it making mistakes. If your project doesn't have some system that automatically highlights this, tests, or linting, or something similar, it will be very painful for you. How to solve such problems? Obviously, with the same decomposition. However, especially when it comes to refactoring tasks, it's quite difficult to decompose them. Even the smallest change can affect a huge number of files. I just gave an example of renaming a global variable. An elementary edit that can affect your entire project. This will be the first time I advise you: do such things manually. If it's something not complex, you tried to give it to the agent, but it failed several times, most likely you've fallen into this trap where the model simply cannot fit into its context the information about all the branches and all the places it needs to consider, and from time to time misses something. This happens rarely. Try to isolate a specific small piece that you need to do manually, and do it. And the rest of the task can also be tried to be given to the agent. Perhaps after you've done the most crucial part, it will cope with it perfectly. But there is one class of problems for which there is practically no cure. This will be the fifth quality that I will describe to you. But before we move on to it, I would like to state one important thesis. Artificial intelligence today can handle any task. The only question is how detailed the implementation details you will need to provide it with to achieve success. And here lies all the magic. If it takes you half an hour to explain a task that you can do yourself in 1 minute, it is obvious that there is no point in this and you will simply waste time. On the other hand, if you can complete a task in a minute that would take you half an hour to do yourself, then the miracle happens when the AI greatly speeds you up and saves you time. And all these qualities that I am describing to you today are precisely intended to help you understand whether to give this task to artificial intelligence now or if it's easier to write it manually. The moment you start to feel this very well, based on these qualities or your internal intuitive feeling, you will start to get that incredible boost. Because the tasks on which the AI would perform poorly, you will do yourself, and they will take you absolutely as much time as before. And for all other tasks, which, in my opinion, are about 90%, you will be able to delegate them to the agent qualitatively and get huge time savings without sacrificing quality. And now that I hope I have conveyed this key thesis to you, I want to mention the fifth quality, which, in my opinion, is one of the most annoying, because this is precisely the class of tasks with which artificial intelligence performs terribly. Well, quality number five. How many different implicit states of the system does the agent need to consider when solving this task? It's formulated a bit difficult, so let's use an example. You are creating some form on your front-end, quite a large one, and its visual appearance changes depending on the parameters you have chosen in it. For example, it's some form for obtaining a loan. And depending on whether it's a legal entity or an individual, the composition of the form itself can change. Slightly different controls, slightly different inputs. You are working with it on your computer, and the agent has made it exactly as you want. Everything works perfectly. Now you open it on your phone and see that the form has adapted very poorly to the small screen width. You perform a new iteration, the agent corrects it for you, now everything looks good, you look at the desktop and again there is some problem. Another iteration. Now it seems like everything is good on both desktop and mobile, but when you start transitioning to a specific state of the form, you notice that another control works poorly on one of the systems. Another iteration: the agent seems to have tried to fix everything, but, for example, you turn on the dark theme and realize that everything is unreadable, the font colors are hardcoded, and it's impossible to use. You fix it again, going through several iterations and errors, when the agent corrected it on one platform but missed it on another. After that, everything seems to be working, but you start scrolling this form and notice that the header on your site sticks to the top of the screen when you scroll down. And this header overlaps important form fields in certain situations. And this is a perfect illustration of how different states of different parts of your system begin to affect each other. And for the agent, taking into account all such intersections is a very difficult task. Another example from the same series is any parallel asynchronous operations. For example, if in the same form, when you click the "submit" button, you display a nice loader on the button while some information is being processed on the backend, will the agent consider what will happen if the user clicks this button again? Will it block it so that the operation is not repeated? Or how will it be handled? Two operations will run in parallel and will conflict for the state of this form. So, all these situations where the AI needs to consider how different components of the system will implicitly affect each other depending on their internal states, or when even the same system component, running in parallel threads, can be in different combinations of these states, all these situations are incredibly difficult for AI agents. And even if you explicitly ask them to consider, for example, by explicitly writing down all variants of such combinations or writing how these components can affect each other, it still works very poorly, because AI agents think very linearly, and such perception requires a certain hierarchy to keep these abstractions of different branches in which we can find ourselves in mind. For agents, this task is quite difficult. If some models in reasoning mode can indeed brainstorm with themselves, write down all these states and try to consider them, then weaker models or models without reasoning will most likely produce absolutely terrible results. And what to do here? Unfortunately, there is no universal pill here. You can try to separate the implementation process from the planning process and ask the AI to analyze your entire system and estimate how components can affect each other or in what states one part or another can be. But, unfortunately, you will inevitably encounter the fact that in complex parts of your application, it simply will not cope. In this case, you will have to arm yourself and write some of the most important part, the core of your component, manually, so that the AI doesn't have to deal with all these varieties of states. This doesn't mean it's completely useless for these tasks. It can still help you write 90% of the code for how a component behaves in a specific isolated state. But this mechanism of how they intersect, overlap each other, it's better for you to do personally. Well, let's move to the finale. When I describe all these task qualities during my lectures, I really like to conduct exercises afterward. I take some prompt, some specific task from my practice or from the problems people have approached me with, and together with the audience, we start to analyze it based on all five qualities. How unique does it seem? Or could it have been encountered in millions of other projects before? How much does it require the AI to deeply immerse itself in the project's codebase? Are there any specific subject area knowledge that was in the author's head but which they forgot to mention in the prompt itself? Does it require the AI to edit many files in this project? So many that it might stumble somewhere and miss something. And how many different states can the component whose implementation we want to entrust to our AI agent have? By answering each of these questions, it becomes quite obvious: what specifically needs to be done to make things better? Perhaps break it down into a couple of tasks? Perhaps add additional context to a global rule? Perhaps some part of this task should simply be programmed manually and not torment the poor AI agent. By repeating this exercise, say, 10 times, spending on it, I don't know, two hours of your real work with the AI agent, I think you will start to understand much more accurately and deeply why it failed with some tasks and succeeded wonderfully with others. And you will generally start to develop this skill of how to correctly set tasks for AI agents so that they can handle them. I hope this video was useful for you. The topic of development with artificial intelligence is very important right now. I often hear from developers that they are worried about the fact that AI is writing code better and better, but at the same time, when they try some tool, give it a task, they say, it pondered for 5 minutes and wrote some nonsense. I would have been better off doing it myself, I would have spent less time, it would have been better. But at the same time, studies are coming out that show that some people achieve very good results with these same tools, which causes many to fear that they are missing something, not understanding something, and are afraid, roughly speaking, of being left behind. Therefore, if this video was useful for you, share it with your developer friends. I will be glad if as many programmers as possible understand how to effectively manage AI agents, instead of just hating them or fearing that they will one day take away your jobs. System architecture skills are not going anywhere, they are still damn important, and you are the architect of your system. But AI can save you thousands of hours simply by taking on routine tasks. Therefore, you should not avoid working with it, you should understand its basic principles and simply learn to use it effectively. Thank you very much. Subscribe to me here on YouTube, subscribe to me on Telegram. I often post news and information there about how I use AI agents for development. And in general, it's cool and great there. Have a good evening. y