Transcription
Hello everyone. Several studies have already shown that while some developers achieve a two- to threefold increase in productivity using AI agents, others, on the contrary, lose speed and quality. Moreover, this almost does not depend on their skills, seniority, or the programming language they use. The main difference lies in the experience of working with AI tools in general. Usually, people understand how to correctly set tasks for an AI agent after about 80 hours of working with it. But what does it mean to correctly set tasks for an AI agent? It means understanding that this particular task needs to be broken down into several parts, or that this piece is better done manually than handed over to the AI. Here, it's better to specify the exact file instead of hoping the agent will figure out to look into it itself. Even the most experienced guys I've talked to often feel these things quite intuitively and just instinctively understand how to act in one situation or another. And I really like to formulate specific criteria based on which qualities, when looking at a task, I make a decision about what to do with it at the moment. In this video, I want to try to save you these 80 hours and explain five key qualities that I pay attention to when analyzing a specific task to understand whether it needs to be broken down, reformulated, or perhaps some part of it should be done manually. By the way, my name is Danila. I've been in development for 16 years, the last 8 years mainly as a CTO co-founder in various startups. And for the last year, I've been programming with AI agents on a daily basis. And before we get to the juicy part, the qualities we will discuss do not depend on the specific tool. It doesn't matter what you use: pure Cursor or Cursor Supercode, pure-code or a swarm of sub-agents, maybe OpenCD, Cline Rcode, it doesn't matter at all. The qualities I will mention boil down to the fundamental principle of how LLM models work or how modern AI agents for development are structured in principle. Let's go. So, the first quality I look at when assigning a task is how unique it is. What is meant by this? It's probably easiest to explain with an example. Suppose I have a front-end project in React with some user table where I display information about users from the database row by row. And I ask the AI agent to add sorting to this table by clicking on a particular column. This task, despite happening on my specific codebase with my table, is absolutely typical and standard. It's a task that has been implemented in millions of projects before me. Millions of different tables online have the ability to sort by clicking on the column header. And in this sense, the AI's solution will represent a very standard pattern. It won't be difficult for it. It has seen how this task is solved a million times. And the probability of it making a mistake here is practically zero. Now let's look at another, but similar task. For example, I have the same user table for my online store, and I want to add sorting by a specific score to it. I want users with VIP status to be displayed first, then users who made a purchase no later than one day ago. And for the remaining users, we will calculate a specific score. We will multiply, say, 0.9 by the number of purchases they made in total and add some activity coefficient for the last month. And based on this score, we will also sort the remaining users in our table. Each part of my task that I've just described is not complex. Each piece of this task is not difficult in itself. Each formula and each branching is not a problem. But when I combine it all, it creates a rather unique cocktail. And if the AI, especially in some reasoning mode, manages to decompose the task into subtasks and implements them one by one, then everything will be fine. If it fails to do so, there will be big trouble. And this is exactly how this uniqueness criterion works. When you look at a task, you need to try to understand how much the solution you expect from artificial intelligence fits into some typical standard pattern that it might have seen tens, hundreds, thousands, millions of times before. And if the task doesn't fit this pattern, it will likely be very difficult for it to solve. So what do you do if you think your task is very unique and likely hasn't been encountered anywhere before? Break it down into subtasks. First, implement each piece separately, and then, as a separate task, ask the AI to combine them into a branched structure. Most likely, you won't encounter any problems, and it will handle it completely from start to finish. If, however, you feed it a whole chunk, a huge, unique combination of your business processes, layers, and how your specific project is structured, the probability that it will stumble at one of the implementation stages, go in the wrong direction, and do something wrong is enormous. And you need to try to avoid this as much as possible. Simply by paying attention to such a task quality as the uniqueness of its solutions, you can already significantly improve the agent's performance just by using the decomposition I described. But this quality works ideally with the next one. So, quality number two. Depth of immersion. How deeply will the agent need to immerse itself in your project's codebase to complete this task? It's important to understand that, like the previous one, this is not a binary criterion, yes or no, that if it needs to immerse itself, it will definitely fail. Naturally, this is absurd. If you've worked with agents even a little, you know they are perfectly capable of reading even a very large number of your project files, hundreds, and then proceeding to execution. But the more the AI needs to immerse itself in your project, the lower the quality of its final solution can be. The fact is, when reading files, it doesn't build an internal hierarchical tree of connections between the components of your project in its virtual head. It studies your codebase quite linearly, sometimes even in reverse order if the files appear to it that way. And all of this affects the quality of its solution. Somewhere it might stumble and not read something important, simply not realizing that some important information is in that file. Sometimes it might overload its context and simply get confused by interfaces and terms, and sometimes it might reinvent something new because it simply couldn't find it in your codebase. You shouldn't rely on this quality. So, at all costs, you need to provide maximum information in the prompt so that it doesn't immerse itself in the project at all. There would be little point in that. It would simply stop saving you time. But giving it tasks that require reading half of your entire project's codebase is also not the best idea. With each new file read, the probability of its error increases, say, by about two percent. If it reads five files, then with a 90% probability, it will provide an excellent result. But if it had to read 30 files, the probability of error increases manifold. And by error, I mean here not so much a solution that doesn't work at all, but rather a solution that is simply different from what you specifically wanted. And the solution to these problems is roughly the same as for the first point. Decompose. If you see that you are giving a task in a too general style like: "Figure out how our project works" and add authorization to it. Such a broad formulation can be problematic. It's better to break down such a task into specific stages. For example, you can allocate a design stage where you ask it to do all the same, but before starting to write code, prepare a detailed plan on how exactly it intends to do it. Then you will have the opportunity to make two or three edits to correct its path if it missed something somewhere or, conversely, wants to go in the wrong direction. Often, we assign such tasks because we already have the entire context of how our project works in our heads, and we are simply too lazy to explain it all again. We want it to read the code itself and do everything for us, but unfortunately, the quality drops significantly because of this. Therefore, this manifestation of laziness must be fought, or at least the task should be broken down into several stages so that the AI genuinely tries to understand the project itself. But if it makes a mistake somewhere, you can correct it. One of the most typical scenarios of problems that the AI will encounter if you give it broadly formulated tasks is that during the analysis of your codebase, it will simply miss some files. And this is very similar to the same class of problems that arise if you ignore the third quality, which we will now discuss. So, the third quality. Does this task require knowledge that is not present in the project's codebase? As comical and banal as it may sound, I think this is one of the most common problems I see when helping someone understand why an agent writes bad code. People proceed from the context that exists in their minds. For someone who is involved in a specific subject area, some things are so obvious that they don't even think to describe them additionally when they give a task to an agent and generally have an idea of the world we live in. But some specific narrow nuances of a particular field may be completely non-obvious to them. What's more, they may not be obvious to the person you are trying to assign the task to. The problem of synchronizing one head with another is, in general, I think, one of the main problems of management. And considering that working with an AI agent is largely about management, this problem smoothly transitions into the realm of human-machine interaction. Put yourself in the shoes of another person, in this case, in the shoes of artificial intelligence. Does it have all the knowledge that you possess? Is some nuance in the subject area as obvious to it as it is to you, or should it be discussed separately, for example, saying, "The typical user journey on our site is like this and this, because the user wants this and this." In general, the process of creativity and feature description should ideally be separated from the development process. You can brainstorm and be creative in one chat, package it into a specific task, and then pass it to the agent that will write the code. But sometimes these processes are difficult to separate. And writing entire detailed documents for each task is a bit unpleasant because it kills all the joy of delegating some routine to the agent. If you spend 30 minutes writing a task, what's the point? You could have done it yourself in the same 30 minutes. Therefore, for things that are already obvious from your codebase, you might not need to overdo it. But still, don't forget that some nuances of your subject area are best added directly to the prompt or to some global rule for your project. In the second quality, where I talked about the necessary depth of the agent's immersion in your project, I explained that it's related to how many files it needs to read and that this generally worsens its result. This quality has a mirror twin. Quality number four. In how many places does the agent need to make changes for the task to be completed? I think you already understand that the principle here is exactly the same. Even if the AI has come up with a competent solution and understands exactly what needs to be done, if this change is required in a whole bunch of files in your project, it might simply miss some of them. And this all falls into the category of refactoring problems, where even a seemingly elementary change like renaming a variable actually requires the AI to perform a thorough check for attentiveness and not miss a single file in the project where it is used. Specifically in the case of renaming, it's not so painful. It can use grep and won't miss a single place. But when it's some really complex logic that requires, perhaps, not such a difficult change, but in a large number of places, the number of errors and omissions that the AI will make will be directly proportional to the number of places itself. You can assume approximately that for every 10 files, it will miss one. Therefore, if you want to perform a refactoring or changes that will affect 20 or 30 files, you can safely expect that it will definitely make mistakes. If your project doesn't have some system that automatically highlights this, tests, linting, or something similar, it will be very painful for you. How to solve such problems? Obviously, with the same decomposition. However, especially when it comes to refactoring tasks, decomposing them is quite difficult. Even the smallest change can affect a huge number of files. I just gave an example of renaming a global variable. An elementary edit that can affect your entire project. This will be the first time I advise you: do such things manually. If it's something not complex, you tried giving it to the agent, but it failed several times, it's likely you've fallen into this trap where the model simply cannot fit information about all the branches and all the places it needs to consider into its context, and from time to time misses something. This happens rarely. Try to isolate a specific small piece that you need to do manually, and do it. And the rest of the task can still be tried to be given to the agent. Perhaps after the most crucial part is done by you, it will handle it perfectly. But there is one class of problems for which there is practically no cure. This will be the fifth quality that I will describe to you. But before we move on to it, I would like to state one important thesis. Artificial intelligence today can handle any task at all. The only question is how detailed the implementation details you will need to provide it with to achieve success. And herein lies the magic. If it takes you half an hour to explain a task that you can do yourself in 1 minute, it's obvious that there's no point in it and you'll just waste time. On the other hand, if you can complete a task in a minute that would take you half an hour to do yourself, then the miracle happens where the AI greatly speeds you up and saves you time. And all these qualities that I am describing to you today are precisely intended to help you learn to understand whether to give this task to artificial intelligence now or if it's easier to write it manually. The moment you start to feel this very well, based on these qualities or your internal intuitive feeling, you will start to get that incredible boost. Because the tasks where the AI would perform poorly, you will do yourself, and they will take you exactly as much time as before. And for all other tasks, which, by my estimation, are about 90%, you will be able to delegate them to the agent qualitatively and achieve huge time savings without sacrificing quality. And now that I hope I've conveyed this key thesis to you, I want to mention the fifth quality, which, in my opinion, is one of the most annoying because it's precisely the class of tasks that artificial intelligence handles terribly. So, quality number five. How many different implicit system states must the agent consider when solving this task? It's formulated a bit difficult, so let's use an example. You are creating some form on your front-end, quite a large one, and its visual appearance changes depending on the parameters you have selected in it. For example, it's some form for obtaining a loan. And depending on whether it's a legal entity or an individual, the form itself can change. Slightly different controls, slightly different inputs. You are working with it on your computer, and the agent has made it exactly as you want. Everything works perfectly. Now you open it on your phone and see that the form has adapted very poorly to the small screen width. You perform a new iteration, the agent fixes it for you, now everything looks good, you look at the desktop and again there's some problem. Another iteration. Now it seems everything is good on both desktop and mobile, but when you start transitioning to a specific state of the form, you notice that another control works poorly on one of the systems. Another iteration: the agent seems to have tried to fix everything, but, for example, you turn on the dark theme and realize that everything is unreadable, the font colors are hardcoded, and it's impossible to use. You fix it again, going through several iterations and errors, where the agent fixed it on one platform but missed it on another. After that, it seems everything is working, but you start scrolling this form and notice that the header on your site sticks to the top of the screen when you scroll down. And this header overlaps important form fields in certain situations. And this is a perfect illustration of how different states of different parts of your system start to affect each other. And for the agent, considering all such intersections is a very difficult task. Another example from the same series is any parallel asynchronous operations. For example, if in the same form, when you click the "send" button, you display a nice loader on the button while some information is being processed on the backend, will the agent consider what will happen if the user clicks this button again? Will it block it so that the operation is not repeated? Or how will it be handled? Two operations will run in parallel and will conflict for the state of this form. So, all these situations where the AI needs to consider how different system components will implicitly affect each other depending on their internal states, or when even the same system component, running in multiple parallel threads, can be in different combinations of these states, all these situations are incredibly difficult for AI agents. And even if you explicitly ask them to consider, for example, by explicitly writing down all variants of such combinations or writing how these components can affect each other, it still works very poorly because AI agents think very linearly, and such perception requires a certain hierarchy to keep these abstractions of different branches we can end up in in mind. This task is quite difficult for agents. If some models in reasoning mode can indeed brainstorm with themselves, write down all these states, and try to consider them, then weaker models or models without reasoning will likely produce absolutely terrible results. So what to do here? Unfortunately, there is no universal pill. You can try to separate the implementation process from the planning process and ask the AI to analyze your entire system and estimate how components might affect each other or in what states a particular part might end up. But, unfortunately, you will inevitably encounter the fact that in complex parts of your application, it simply won't cope. In this case, you will have to arm yourself and write some of the most important part, the core of your component, manually, so that the AI doesn't have to deal with all these varieties of states. This doesn't mean it's completely useless for these tasks. It can still help you write 90% of the code for how a component behaves in a specific isolated state. But this mechanism of how they intersect, overlap, is better done by you personally. Well, let's move towards the finale. When I describe all these task qualities during my lectures, I really like to conduct exercises afterward. I take some prompt, a specific task from my practice, or from the problems people have approached me with, and together with the audience, we start analyzing it based on all five qualities. How unique does it seem? Or could it have appeared in millions of other projects before? How much does it require the AI to deeply immerse itself in the project's codebase? Are there any specific subject area knowledge that was in the author's head but which they forgot to mention in the prompt itself? Does the AI need to edit many files in this project? So many that it might stumble somewhere and miss something? And how many different states can the component have, the implementation of which we want to entrust to our AI agent? By answering each of these questions, it becomes quite obvious: what specifically needs to be done to make things better? Perhaps break it down into a couple of tasks? Perhaps add additional context to a global rule? Perhaps some part of this task should simply be programmed manually and not torment the poor AI agent. By repeating this exercise, say, 10 times, spending on it, I don't know, two hours of your actual work with the AI agent, I think you will start to understand much more accurately and deeply why it failed with some tasks and excelled with others. And you will generally start to develop this skill of how to correctly set tasks for AI agents so that they can handle them. I hope this video was useful for you. The topic of development with artificial intelligence is very important right now. I often hear from developers that they are worried about the fact that AI is writing code better and better, but at the same time, when they try some tool, give it a task, they say, like, it thought for 5 minutes and wrote some nonsense. I would have been better off doing it myself, I would have spent less time, it would have been better. But at the same time, studies are coming out that show that some people achieve very good results with these same tools, which causes many to fear that they are missing something, not understanding something, and are afraid, roughly speaking, of being left behind. Therefore, if this video was useful for you, share it with your developer friends. I will be glad if as many programmers as possible understand how to effectively manage AI agents, instead of just hating them or fearing that they will one day take away your jobs. System architecture skills are not going anywhere, they are still damn important, and you are the architect of your system. But AI can save you thousands of hours simply by taking on routine tasks. Therefore, you shouldn't avoid working with it; you should understand its basic principles and simply learn to use it effectively. Thank you very much. Subscribe to me here on YouTube, subscribe to me on Telegram. I often post news there, information about how I use AI agents for development. And in general, it's cool and great there. Have a good evening. y