Transcription
Almost 200 dollars for Claude, over 200 dollars for AP Openrouter, and a 200 dollar subscription to GPT Pro so that it doesn't ban me during intensive prompting. I spent all of this to analyze 4,000 scientific studies on prompt engineering that occurred in January, February, and March of 2025. I downloaded all these studies, extracted the most necessary and useful information from them, and kept only 300. 300 studies represent maximum usefulness and applicability. I thoroughly studied everything, tested it for 3 weeks. And now I will tell you about the most necessary concepts that I learned from these studies. At the end of the video, I will show you how to access these studies, how to get the bot that I wrote based on these studies. Let's get started. The studies indicate the importance of specifying not just a role, but a specific role that can help precisely in the given solution of a question. For example, we say not just "be a mathematician," but we say: "Be an expert in linear algebra," if we know that we need to solve some highly specialized request. But this is not enough. And here's a cooler tip: you need to use a multi-agent approach, where we divide the task into several parts at once and assign a role for performing this task. Look here. Help me plan a vacation. As a travel agent for families with children. Here is the first role. As a financial consultant. Evaluate the cost of the route. Second role. As a travel logistician, the third role. And act as a travel blogger, so that he describes how the vacation will be spent day by day. That is, four roles are used here at once. Depending on your tasks, you can connect other necessary roles for the same question. Here, a travel blogger is connected, who summarized all this and uses it. Made introductory remarks. As a travel agent, he starts advising, as a financial consultant, he makes a table, like a typical financier, a travel logistician. Plans for the days. And here's the travel blogger, he took all this, and the travel blogger described it all, how it will be every day. Thus, by connecting several roles at once, we can work through our task from different angles. Moreover, here we have serious aspects, like a financial consultant. Well, in the end, we, as travel bloggers, conclude everything so that it looks clear and interesting, so that we can imagine this vacation. At the same time, it will adhere not to emotions, but to all these aspects that we have designated. That is, logistics, finances, interest for children, and this is just a wrapper. And if we had only one wrapper left, it would describe it to us only from some incomprehensible, uncontrolled point of view. Here, with the help of these roles, we activated the knowledge that we need for this trip, which we consider important. And when the neural network combines all these elements into one, it understands better and conditions less. The most royal method in prompt engineering is, of course, the chain of thought. Chains of thought are so cool that neural networks have started creating their own models that are trained specifically on reasoning using chains of thought. This is simply a very powerful prompt engineering method, especially when these specialized models did not exist. People used this technique to extract powerful knowledge from neural networks. In modern research, there is something to add to this method. So, look, the most important thing here is gradual reasoning, deriving intermediate steps, identifying cause-and-effect relationships, and self-checking during reasoning. Here it is, look, the basic code. Solve the problem and reason step by step. So, I open a non-reasoning model, and it automatically becomes reasoning because I told it: "Reason step by step." But this has been known since 2023. This is quite an old technique. Here it reasons, step one, step two, figures everything out. Here it figured it out and calculated. Everything. The next option is not just reasoning step by step, but I have to write down the steps for reasoning instructions myself. Look, here, here's an analogous task, and we set it in the instructions. For each step, formulate the reason, then formulate the consequence, indicate the connection, and check all consistencies. Here we control the steps. We tell it to act step by step, but we keep each step under control. Just like in this example, look, we tell it to analyze a certain passage and set the steps. What step? This step. Everything. And how can we identify these steps, by the way? Here, this study shows that preliminary improvement, improved understanding of the task, increases the quality of reasoning. That is, we can initially give the task to understand what is needed to answer this question. Write down the steps, then insert the steps, and it will follow them. Look, let's take it. So, I gave it the task and at the end told it to identify all the steps needed to find the correct answer. Here it identifies the participants. Step two rewrites the conditions in the form of logical implications. Here it starts to break it down and everything. And it suggests proceeding to verification. I can just write "Proceed." That is, I already see all the steps and can influence this process if, for example, I don't know how to solve this problem. Here it came to an intermediate conclusion. Well done. And then it forms an in-depth analysis. Everything, Anna took the candy, Anna is revealed. And here is my favorite method - the chain of thought, but with the help of program code. It's very cool. Look, the key here is that it implements the solution in the form of Python pseudocode. It should now form a function by which it will be necessary to solve it. That is, in text form, literally on a piece of paper, it will write this function and, by looking at the solution option without running Python, it will solve it. Does it have the ability to run Python? Yes. Let's see now. If it runs Python, I will see it. So, there will be a Python execution window here. And it should just write it in a line. And look, I have model 4o, which is a regular model. And because the prompt is formulated this way, it thinks for a very long time. It thinks like a chain of thought thinks when you give it. Here it started forming equations, steps, here it's just writing Python code and step-by-step execution. It solves it in pseudocode. Look at the example. I told it to count the vowels in the word "vodozasposobnost." It says: "Khvyat." There are actually eight. Let's see if it can solve it using the step-by-step method and using Python. Uh-huh. Here it formalizes the task. It defines what vowels are. Second task. Aha. That is, it says, this script would start counting, but it doesn't count, it just drew it. Uh-huh. The counter has gone. Voila. It solved it. There are eight of them. It wrote a function, so to speak, wrote a function, but it didn't actually use Python. That is, if I had told it: "Use Python," it would have just counted. Look at the difference. Here, please, I wrote it to use Python. And here is the code execution. Yes, Python started. The word is such and such. Vowel is a vowel, and that's it. Here it calculated this sum. This is precisely the calculation by running Python within itself. And this is an option for organizing thought. That is, it doesn't calculate by running Python itself, but it wrote it down, and it knows how it's solved. It doesn't need to run it because it sees the structure and it sees the solution. Therefore, if I need to solve complex tasks involving calculations and analytics, I use this method because it provides maximum accuracy, especially for financial analysis, it's maximum. Be sure to use this approach. Next, the knowledge tree method is also a very powerful method. The essence of the approach is that we break down tasks into several alternative development options and give it the opportunity to return if necessary. For example, we break down the task into subtasks, and for each subtask, it needs two to three possible solutions. It needs to evaluate the effectiveness of each branch. And if it gets stuck, it needs to return to the branch above and go along the correct path again so that it doesn't go down the wrong one. So, it needs to go through the tree and solve this task. Well, here it goes, breaking down into subtasks. Here we go. Here it goes, goes, solves the task. Here. And it came to the final schedule of all these speakers. Used all alternative methods. This method is similar to a method from another study. It is called "make the correct and incorrect answer." A very cool technique, by the way, that allows the neural network to focus on the fact that there is an incorrect answer. And because it exists, it better forms the correct one if there are incorrect ones. Look, this is called contrastive reasoning. We ask the language model to generate both correct and incorrect answers. Moreover, we can use this contrast in combination with the chain of thought so that it uses the step-by-step method. That is, and thanks to this, it activates critical thinking. It generates opposite answers, incorrect answers so that it doesn't go down the wrong branch. And also thanks to this, it is more focused on making the correct answer, unlike the incorrect one. Well, let's look at an example. By the way, always conduct all tests in a new chat so that the context is not leaked. By the way, when it starts to annoy me that it often praises me for no reason, like a three-year-old child, I switch to Monday mode, and Monday is ruthless. Let's see how Monday will not praise me. Well, like, look, I have a sub-colloquial and managed to divide. The world is not yet doomed. I like this more than it constantly praises me in a rosy light like a fool for micro-actions. So, look. The correct answer is time equals distance divided by speed, and the incorrect answer is distance multiplied by speed. This is, well, a fundamentally different approach. It can also be used for logical relationships, for business tasks. And here researchers say that it improves by 5-10% specifically in complex tasks. Luck works, in my opinion, it's very similar to the tree of thoughts. In the tree, we give it different alternative paths, and in the contrast method, we tell it specifically correct or incorrect. And here we say variations of answers. And here's another cool technique - the chain of drafts. With this approach, we tell it to make minimalistic generalizations of five words. Just as short as possible. Let's ask Monday for an example right away. Please. Task. For each step of reasoning, use no more than five words. This is the root of this method, so that it acts step by step and uses no more than five words, so that it is as concise as possible. Here we go. Cost, markup, everything. That is, it throws tags. That is, it doesn't just write fragments, it really compresses this information into five words. But not always five. Although here, plus is also a word, it's six. Well, okay. Everything. Here's the answer 73125. Look, the next example is a chain of reasoning. We use multiple paradigms to solve this task. Look at what to solve the task using this method. Understood? First, use natural language reasoning, then algorithmic reasoning, and symbolic reasoning. This method, by the way, is very similar to multi-role prompting, when we say: "First, think like a psychologist, then like a marketer, and then like an SMM specialist." And let's make some conclusion. Similarly, here we tell it: "First, use this method, then use that method, and then use this one." And it will be a compilation of approaches, due to which the task will be worked out more deeply. Here's a simple example of this method. I tell it to solve the task and follow three stages. Well, naturally, uh, reason step by step. Then analyze, evaluate your answer, find possible errors and omissions, and present an improved solution. That is, here I embed the need to conduct an analysis, to conduct self-criticism. We turn on self-criticism mode, due to which it improves the result. Look, reasoning. Then analysis. It conducted an analysis, rechecked, everything is logical, no errors, made some simplified version. And here it made an improved solution. Chain of verification is a more advanced method, using chain of thought and self-criticism. So, now let's solve a marketing task, where an advertising manager is launching a campaign on three channels: Google Ads, with such a budget, cost per lead, Instagram, and so on. The solution requires solving the task, showing each step, and performing verification, checking if all data conditions are correct. Here are five verification conditions. And then we move on to the third part, correction. Thanks to this verification block, it should, it should very thoroughly check all information on five aspects and only then correct and solve. Here is the first part, it's calculating something. Step one, step two, distributing leads. I am interested in verification. So, well, it wrote that everything is correct, everything is correct, it makes sense. No corrections are required. Apparently, the task is simple and it didn't correct anything. But that's good. If there are no corrections, it means it is maximally confident in the answer. Possibly, it has started to answer better thanks to the fact that it knows that it will have to go through this entire chain of verification and self-criticism. By the way, here's another example of working with self-criticism - the confidence indication method. Let's see. I tell it to answer my question as accurately as possible, rate your confidence in the answer on a scale of one to ten. By the way, there is a study that says that it is necessary to use exactly these numbers, at least from one to 10, and even better from one to 100. Then the distribution gradients are much clearer. Look, if the confidence is below seven, suggest an alternative answer. And here we go. Look, it gave several answers. Life on Mars has not been discovered. It is 10 out of 10 confident about this. Next, it is 6 out of 10 confident that microbial life may exist. And here it has 9 out of 10 confidence that Mars was potentially habitable in the past. And the type of information is reliably scientific knowledge. This is a very cool technique by which we see its degree of confidence. I used this technique when analyzing scientific research data. I asked it to be confident in assessing the relevance of each study, and this yielded very good results. The placement of key information in the prompt matters. Researchers recommend placing the most important information at the beginning or end. Therefore, we have arrived at a combined method. This is a meta-sandwich, where important information is placed at both the beginning and the end. At the beginning, it is mandatory, because in this way the context is set, and the necessary neurons in the neural network that will answer your question are activated. At the end, it can be duplicated, moreover, in different words, to avoid misunderstanding. Look at the sandwich effect example. At the beginning, we place the information to be scientifically accurate, contain three key recommendations, use simple language. At the end, we remind of the same thing, but in different words, thesis by thesis it is repeated. The next approach is simply to place the most important, the most key information at the beginning, and then just distribute it by priority, indicating what is the highest priority, what is the medium priority, and what is the ordinary priority. Thus, we set the priority for each block of information, and the neural network perceives it better. You can also go the route of a narrowing funnel of detail. For example, as indicated in this example. Here we set a general task: to provide information about possible causes of headaches - this is medical consultation. Then we designate the target audience, an adult patient. Then specificity, the narrowing occurs. And the last is the detailed content requirement. Thus, we took the general and narrowed it down to the specific, and it built it like a pyramid. This is the basis of the didactic method, due to which very complex logical tasks can be solved. Next, the intermediate summarization method, when we ask the neural network within the prompt to make summary conclusions after significant information. Look at the smart home system project, functional requirements, and then a summary. Here will be a certain list, and here we make a summary, we summarize the limitations ourselves. We summarize the limitations so that it does not miss this large list that will be here. Budget calculation. We summarize each block of information for it, and thus it will navigate these summary blocks, especially if we mark them with specific bullets or letters or numbers. Next example is placing blocks in the prompt in a specific order. This is somewhat similar to this medical consultation, the cascade method. Only it includes not a narrowing cascade, but certain blocks that influence each other, which are more or less equivalent. That is, we have a certain natural order of the thought process for each person. And this entire process is permeated with texts on which the neural network was trained. All our knowledge is somehow subject to a certain chain of thought process. And researchers have more or less come to the conclusion that context, task should come first, and then additional requirements. Many experiments have been conducted in this direction. And look, here are the results that high response quality is when the role is at the beginning, then context, then instructions, then format, and then some limitations. And if, look, at the beginning, you set the format, some start with limitations, like "write a text of 500 words," that's wrong. This will be low quality. Do not start with the limitation "write a text of 500 words" or do not start by breaking the text into paragraphs. Start with the role, context, then instructions, and then everything else. And here there are different types of tasks that were tested, for example, for analytical tasks. If a role is set at the beginning, it gives better results than if context is placed at the beginning. For creative tasks, context first, then role is better. For educational tasks, there is also role, target audience. The most important thing to take away from this is that order in the prompt matters. Perhaps you will have a task where, by recombining elements in the prompt, you will get a more accurate result on an even playing field, simply by rearranging them. And it will be like this. Everything needs to be tested. Everything needs to be tested, even if the researchers told us here, yes, you need to move all these elements. I often change different elements in my prompts, but I never put this format, these additional requirements at the beginning, because if you first write: "Write concisely," and then about what? It activates its neurons responsible for some kind of concise representation of information. So, compare the result on the left and on the right. First, I asked it to be concise and laconic, and in the second option, I asked it to describe the history of computers. It is also short, but here is a completely different approach. Look, it turned to the origins. Early 19th century, Turing. And here began the first generation. And here it immediately started with the forties and fifties and so on, here we go. Here it activated its conciseness and laconicism. And here it immediately understood that I need to know the history of computers. But then I slowed it down with the next word, so that it would be concise and laconic. But some information about the history of computers has already gone into it. from the very origins. And with this phrase, it initially acquired the format of conciseness, laconicism. There is no such thing as right or wrong, depending on what you need. Keep in mind that the placement of each sentence greatly affects the result. There are many such methods. I have created a generalized PDF book in which all these methods are attached. Here is the table of contents and a brief description of each method. You can familiarize yourself with each method, read the key conclusions, and go to the study itself. In addition to the PDF book, I have created thematic articles that comprehensively review different methods, including those used in this video. This is what they look like. You can go and see what two-stage prompt creation is or 10 prompt engineering techniques from these studies that are presented here. So, these that are attached here reveal various techniques in more detail with examples. If in this PDF book each study is attached and how it works, then in the articles I have included more practical cases and combinations of approaches. For example, the top five summarization techniques from the study. Several methods are considered here. Top five methods, in fact, there are five more methods by which you can summarize text. All of this does not fit into the video, so I divided it into articles. Follow the link in the description to read how to access these articles. I have also created a Telegram bot that works based on these studies. You can ask it questions about some method. For example, you decided to learn more about this method and write to it. Look, it briefly explains what this method is. And a small prompt example of how it works. Now I am showing this as implemented in VN, but it will be packaged into a Telegram bot. And you can write directly to the Telegram bot. You can, by the way, write like this. Look, you write to it that you, for example, are a lawyer and advise on five prompt engineering techniques for my daily work. Here you go, it suggests hierarchy and decomposition, chunking, breaking down legal documents into fragments. Chain of reasoning, please, here it is. Using, processing pause marks. Here you can briefly imagine which methods would suit you from this study. The link to the bot will be in the description. The conditions for accessing it will also be written there. Look, the bot works on the 4o model. This is not a super, as you understand, cool model, not a super smart one. It will give everything in a very concise format, but I will be able to present this bot to the public. Thanks to this, you can work with the book, with the articles, and consult with the bot specifically on these studies, so that it helps you interpret and summarize them. 4o compresses everything, yes, and you can ask it to provide a link to the study and its more precise name to find it in the book. Here you go, it provides a link to the book and gives the name. Well, here you go, it gave a link to the name. Thanks to this, you can get a brief consultation here, then clarify what exactly was meant, get a link, download this study, it's large, yes, and upload it to GPT. Look, you get the study. This is most likely inevitable. If you like some method that you want to dive into thoroughly, you will need to get this study to work with it thoroughly, because in my vector database, there is still a summarization. Good summarization, but it's a summarization. And with the help of the bot, you can extract a specific name, how it all works, a link, and work with it in detail. Here you go, everything, it provides it. The bot provided the necessary link, yes, and you can continue working with it. First, of course, I made just this table with key findings, ratings, and started reading this table myself. I realized that some things are just hard to perceive in the form of this table. Some things, of course, can be perceived, but overall, it's not so easy. Then I started creating the book. I created the book from this Excel, wrote a script using GPT according to some algorithm to extract and form such a booklet. Then, in addition to the book, I decided to create a table of contents. That is, you can just go through the table of contents and see brief conclusions. And if something appeals to you, some method, click on it and go directly to it. This will be very convenient. Then I made a lot of generalized articles in HTML format, in PDF format they are also presented here and in Markdown format so that you can feed this Markdown to GPT and work within this study. For example, instead of searching for a study, you can just upload one of these results here. And here you go, it just takes the method that is presented and forms prompt variations so that you understand how to work with it in practice, because just reading is generally uninteresting. You need to be hooked by some information, and then you should start working with it in GPT. And one of the simple ways is to feed it context in the form of a ready-made article and work with this article. So, work with the table. The table will also be provided with PDF documents and with my bot so that you can master these modern prompt engineering methods. This is my first digest element, which I plan to release regularly. I will update the bot, fill it with new scientific research, and periodically release digests of updates on new methods, new approaches for working with neural networks. All these studies greatly improve prompt engineering skills. When you work, even the methods used by researchers are often very original. And even if the conclusions relate to some programming that is not relevant to you, you can apply the method itself, which they came to, with which they worked, in your activities, in your chat ecosystem. There are many useful methods here for automation. Yeah.