Transcription
Hello everyone. I am sure that you have long been using various AI assistants, whether it be some kind of co-pilot or cursor, or cloud services, or anything else. And you are surely generating code with the help of these tools, perhaps some documentation, tests. Well, in this presentation today, I will try not just to tell you how you can use them, because you already know that, but how to use neural networks effectively so that the results obtained are better. and require fewer edits, let's say. The goal is not just to show how they work, but to explain why they work one way and not another, what their limitations are, and how to turn them from simple assistants into reliable partners. To achieve this goal, we will consider four key aspects. Strategic prompt engineering, fundamental principles by which AI processes code, practical methods for integrating these tools into a professional workflow, and finally, a critical analysis of the risks and limitations that must be considered when working with artificial intelligence. This approach will not only increase productivity but also ensure the quality, safety, and reliability of the software being created. Therefore, as Yuri Gagarin said, let's go. Typical scenarios for using AI in development include autocompletion, generation of ready-made solutions, and code review. However, in order for these scenarios to be maximally effective, it is necessary to master not just writing requests, but their strategic construction. Effective prompt engineering goes beyond simple commands and becomes a methodological approach. The first and most important principle of effectiveness can be called clarity and detail. An artificial intelligence model works better when it receives a clearly formulated specific task. For example, instead of a general request "Write a summary of the book Pride and Prejudice," a much more accurate result will be given by the request: "Write a summary of Jane Austen's book Pride and Prejudice. Focus on the description of the main characters and the development of the plot." For programmers, this means the need to specify not only the desired functionality but also the programming language, the framework used, dependencies, and the preferred coding style. This level of detail narrows the context for the model and allows it to generate more accurate and relevant code. The second principle is defining the role. Assigning a specific role to AI, for example, "you are the best programmer in the world" or "you are a DevOps expert," also significantly improves the quality of responses. This approach helps the model focus on the relevant area of knowledge and apply the appropriate style and tone. As noted in various expert discussions on the internet, you can find them yourself. For the GPT4 model, for example, such role-playing can increase the "IQ," if one can say so, of the responses, as it directs the model to find the best solutions in a specific narrow area. The central principle, which is often overlooked, is the so-called "one task rule." Research shows that you should not ask AI to perform multiple tasks in one request, as this can overload the model and confuse everything. When it receives a complex multi-component request, it tries to process multiple ideas simultaneously. This increases the "noise" in its context, well, let's say, it cannot focus on one thing. This effect can be explained by the fact that AI functions based on attention mechanisms, which allow it to focus only on the most relevant parts of the input data. It turns out that when several unrelated tasks are included in the prompt, the model is forced to distribute its attention among them, as if not understanding which of them is the most important. And this can lead to a lower quality result. For example, if you ask AI to write code and test it in one message, in one prompt, it may generate working code, but the tests will be incomplete or contain errors. Therefore, separating into two sequential requests, first generate the code, and then write the tests, allows the model to fully concentrate on each individual task, which significantly increases the accuracy and reliability of the final result. And moving on to more complex methods, let's consider two powerful prompt engineering techniques: few-shot prompting and chain-of-thought prompting. And let's start with few-shot prompting. This is a method in which one or more examples of desired interaction are included in the prompt, which helps the model understand the required result, style, and logic of the response. And unlike traditional fine-tuning, when you pre-configure the model on some specific data, which requires a larger dataset for training and is more complex and expensive, the few-shot prompting approach helps the model generate behavior based on just a few examples provided directly in the request. This is especially useful for developers in tasks where a strict input or output template is important. For example, when you write tests or something else. That is, you can feed into the request how the model should, how the result from your function should look. That is, you want a function from the model, you describe the function itself, and say that I want the result to look in this format. If you use any programming language where you can create models, like TypeScript, Python, Java, in principle, any programming language today, then you can create a model and feed this model as what you want to get as output from the function. And if you provide all this information, then the model will correctly generate the function, the test if necessary, and anything else, and it will have a defined style for both incoming and outgoing data, and it will process this data internally as you describe. Chain-of-thought prompting is a technique that forces the model to break down a complex task into sequential logical steps or to "think aloud." This significantly increases the model's accuracy and reliability of responses, or rather, the accuracy of responses, especially for multi-step tasks requiring reasoning, such as arithmetic calculations, logic, planning, anything. And for developers, this means that this approach turns AI from a simple code generator into a practically real partner to work with. For example, instead of the request "fix this code," you can ask the artificial intelligence to "analyze this code step by step, find errors, and explain why they occurred." This approach allows not only to get a solution but also to see the process of finding it, which will help to debug, fine-tune, improve the prompt itself, and understand where the error was made. Well, and also, if you don't understand how all this works well, and the error itself looks scary to you, then this approach will essentially help you, most likely, to figure out what actually happened and how to fix it, because it will write to you step by step how it did everything. Essentially, since most neural networks today work best with English, you can simply add one such phrase to your request: "Let's step by step." And this command already prompts the model to generate intermediate steps on its own, which often leads to a more accurate final result. Applying chain-of-thought to debugging makes the process more transparent and manageable and allows you, as a developer, not just to get working code, but also to understand the root cause of the problem. And now, in order to effectively use neural networks, let's understand how they process information. This is also important. AI does not understand code in a human sense. It does not run it, it does not track variables, it does not interpret its logic. Although, of course, you can get its runs and error tracking in the same cursor, for example, but still, it doesn't just read texts with its eyes and understand what's happening somewhere in its head. Instead, AI works with abstract representations. Let's start with the first stage of code processing – tokenization. This is the process by which the source code is broken down into recognizable groups called lexemes or tokens. For example, you can take a string, say, "networks = assets - liabilities." And this string is converted into a stream of tokens, such as, for example, an identifier, which will be "networks," an assignment operator, which is "=" in our case, an opening parenthesis, then another identifier, then a minus sign, then another identifier, a closing parenthesis, a semicolon, and so on. This approach allows the model to ignore superficial details that do not affect the program's behavior. And such not particularly important superficial details would be, for example, variable names, because it doesn't matter at all what the variable is called, or separator symbols. The same semicolons are not always needed, and they can simply be ignored if they are present. A deeper level of understanding of code by the model, which it "parses," is achieved using an Abstract Syntax Tree, or AST. Unlike a simple sequence of tokens, an AST represents a hierarchical structure where internal nodes correspond to operators, and leaves correspond to operands. And converting code to an AST allows you to identify dependencies and semantics that tokenization might otherwise miss. And understanding how, even superficially, AI processes code is crucial. Again, the model does not understand execution logic like a human. It works exclusively with such abstractions. These are either tokens or ASTs. And this feature explains why AI excels at tasks requiring pattern recognition, such as boilerplate code or autocompletion. However, the model can "hallucinate" very strongly when solving unique and logically complex problems that were not present in the training dataset. Realizing that the model is essentially a plagiarism machine or a statistical engine or a set of if statements, however you want to perceive it, and not true intelligence, allows developers to structure prompts in a way that best suits its internal mechanisms. For example, if you are refactoring using AI, and if you provide not only the code but also a description of the architecture and key dependencies, it will help direct the model to a deeper analysis. The AST will lead to a higher quality result. Continuing, the key parameter determining the model's ability to process information is the context window. This is a kind of working memory for the model, which determines how much text, measured in tokens, it can consider when generating a response. And here it is important to understand that a token is not a single letter; a token can be a whole word. And usually, models measure the size of their knowledge in tokens. Since the appearance of the first transformer models, this parameter has grown thousands of times. It started with 512-124 tokens. Now it's hundreds of thousands, and sometimes even millions of tokens. And impressive growth is observed in models like GPT4, Claude, or Gemini 1.5 Pro. And it sometimes seems that the larger the context, the better the result. But in practice, the paradox of a large context window arises. Although the theoretical capacity of the model grows, experienced developers, again on the internet, in articles and videos, and I can also tell you, note that providing too much code can lead to a decrease in the quality of responses and even sometimes to foolish decisions. You have probably noticed, if you have used the same cursor on a large existing project, that you ask it to do something, and it starts doing it in many files where perhaps it shouldn't, and adding some functionality that you sometimes didn't ask for. This is because the attention mechanism, despite its improvement, cannot always effectively process giant amounts of information. As a result, important details located in the middle or at the end of a long prompt can be lost. This reduces the coherence and relevance of the generated code, and simply copying the entire codebase into the prompt is not a guarantee of success. If you provide the entire code of a large project in the context, then most likely everything will be very bad. And effective use of neural networks requires a strategic approach to context management. The developer must carefully select the necessary information, reduce noise, and direct it towards solving a specific task. Therefore, if you are working with a large project in a cursor, when you write some code, specify the specific boundaries in the code file, or perhaps specific folders or specific files as context, rather than letting the cursor work with all the files in your project. And let's move on to AI agents, because neural networks no longer just supplement code, they are becoming autonomous partners. And editors like Cursor or agents like Code, represent the next evolutionary step, allowing complex multi-step tasks or even simple, mundane ones like writing boilerplate code to be delegated to these agents or code editors. Examples of task delegation to an agent: we already know that we have agents, we have some workflows that can work without human intervention, and examples where we can integrate AI into our workflows include large-scale refactoring. Instead of manually renaming variables or changing function calls in hundreds of files, you can give a command to a hypothetical agent like "optimize performance in this module and add docstrings," and it will independently analyze dependencies, make changes, and perhaps even write comments. In general, you won't be doing such primitive tasks that simply take a lot of time. You can also add new functionality. You can ask it to complete a theme on the settings page, for example, dark instead of light, or add dark to light. You can also ask it to create some function and add it to the interface. And the neural network will independently analyze dependencies, make changes, find the necessary files, create some styles if needed, update the interface, add some switch. In general, in theory, it will do all this independently. Of course, it's not perfect yet. You will still have to manually review and correct it, but in general, all this is more or less working. Or the integration with new libraries. For example, you can feed the library's documentation to the agent, and it will integrate this library into your code, with the functionality you request. Also, complex debugging is when you can work on some problematic code, you highlight it and simply ask the agent: "Find a memory leak in this module, analyze the cause of some error, and suggest a fix." This will allow the artificial intelligence in the form of an agent to analyze your entire project if necessary, or just one file. Remember that the context window should still be more or less limited. And after that, the artificial intelligence will possibly, and most likely, offer you some fixes, changes, and patches, often even correct ones. And some advanced tools today, such as Code, for example, allow you to create specialized sub-agents with clearly defined roles, context, and toolsets. Essentially, you can create not one big Jarvis agent, but you can create, for example, Kevin, the architect, who will be responsible for planning and refactoring, a tester who will exclusively generate and predict tests, then some Andrew the programmer, who will write all the code. And this approach helps you break down complex problems into manageable parts, which increases reliability and efficiency. Code review with artificial intelligence. You should remember that artificial intelligence does not replace human judgment in the review process, but complements it. Therefore, models can be used to excellently solve some routine work. For example, AI tools can check code for syntax errors, find null pointers that you missed, suggest adding JSDoc or documentation strings, check for compliance with style guides, and so on. Although, of course, all this can be done without AI today, but we can also add AI to it. And all this will allow automating what is already automated. And in some ways, because, for example, AI can analyze code for security errors, for the overall flow, for compliance with all other files in the project. And this can reduce the time that normal people, ordinary living people, spend on code reviews, because it is usually very expensive. The people who do code reviews are developers, and they have high salaries. So, if such a basic or even not so basic inspection and analysis of files is done by AI, it can save a lot of money. Therefore, if we add AI to do code reviews, we are essentially freeing people from looking at trivial things. They can delve into some serious logic and check it, rather than whether there is a semicolon at the end of the line or not. At the same time, such neural networks for code review exist today on various platforms. There are already them in GitHub, in GitLab, there are built-in ones, and there are external ones that can be added as plugins. Therefore, I would recommend using them. It's actually a very cool thing. And here is a small list in a table of places where AI can be used today, at what stage of development. And here, in fact, all stages from planning to monitoring and logging of what is in production. Therefore, essentially, today, if done correctly, we can integrate neural networks into all processes in software development. But seniors, and not only seniors, who work with AI today, need to understand not only the capabilities but also realize the risks that neural networks carry. And one of the most serious risks today is code security. A study conducted by universities, the engineering school of New York University, showed that about 40% of the code generated by GitHub Copilot, and I think this can be applied to any other artificial intelligence, contains vulnerabilities or design flaws that can be exploited by attackers. That is, essentially, 40% of the code written by neural networks is dangerous and hackable code. And this leads us to the following conclusion. Code generated by neural networks should be considered as third-party library code, which you are not particularly familiar with. When you use some third-party library, for example, Node.js, you download an npm library, you are not particularly confident in its quality, in its security. That is why we are facing a huge number of hacks through libraries recently. So, AI should be treated the same way as libraries. That is, it is unsafe, uncontrolled, and security audits must be carried out very carefully. For this, you can use various static analysis tools. For this, you can manually, essentially with your eyes, go through the code that the neural network provides, and pay attention not only to the logic and the beauty of what is written, but also to potential vulnerabilities. Moreover, if you integrate an agent directly into the application's workflow, it's an even bigger problem, because there are prompt injections, and in theory, your product can be hacked through prompt injections. Therefore, using AI for code generation and as part of the product itself is very dangerous and requires careful attention to everything. So, once again, the code you receive is like third-party library code. Treat it carefully, check for all risks. And another serious problem that is part of the risks today when using artificial intelligence is licensing. There is an opinion that tools trained on public repositories may generate code fragments identical to those protected by licenses. And this creates a risk for companies, because if you use such code that was actually protected by a license, then you are violating copyright and can receive a court summons. So, not always, and even more often, AI is trained not only on open-source code but also on proprietary code. Or it may be free to use for some purposes, and for others, you have to pay for its license. But since AI does not know this, it simply knows that it is a piece of code, it can give it to you. And this situation illustrates that technology is outpacing the law. And of course, another problem that exists and has always existed is hallucinations. Neural networks, especially large language models, those LLMs, do not possess true knowledge. They are probabilistic machines that generate the most plausible sequences based on the data they were trained on. This means that essentially they can create, for example, code that looks logical but is actually incorrect or contains hallucinations, information that does not correspond to reality. This once again emphasizes that critical human oversight remains, if not the most important, supervision over what AI does. And that is, AI is a powerful, cool, but very unreliable assistant. As a developer, you must always check the generated code, test it, ensure its correctness, so as not to get, in the code, the use of functions from some third-party library, in which these functions do not exist. And the library itself may also not exist. That's all for now. In this video, I did not aim to show you specifically how to write code using neural networks. I only gave you food for thought, so that you can think about how you use all these neural networks. And always remember that large language models are a rather complex tool. And you must know how to use it correctly to get the best answers. M.