Transcription
If you are tired of your agents getting confused, if the reasoning mode doesn't help you, if you want to make all your automations as accurate and verified as possible, where you use generative artificial intelligence, agents, or queries to models, then this video is for you. Be sure to watch it to the end, because I will be talking about one of the most successful, most accurate architectures, on the basis of which you can build very cool automations that are suitable for business simply because they always provide the most accurate and relevant answer at the output. And we will talk about something like SGR, Schema Guided Reasoning. I'll say right away that I am not the author. I saw this from another author, a developer. His name is Renat. And I was inspired by his materials, his developments, studied his articles, which he kindly shared on his channel. And I implemented it in N8N, slightly changed it as I see it, as it seemed to me, it would be clearer to show. And most importantly, how it can be successfully applied. So, what is SGA? Schema Guided Reason. You know perfectly well that there is now a new generation of models that can reason. And it seems that this should give them additional accuracy, because they start to accelerate tokens, reason before giving some final answer. But there is one nuance. Sometimes these very reasonings, they can go off track, they can lead the model in a completely different direction. And thus the model gets onto such a false trail and it leads it astray. In the end, this is an irrelevant output. What, in my understanding, is very good automation for business? The most important thing is the predictability of the result. It is the constant, stable, non-decreasing quality of this result. It is always the same output format, which completely satisfies the business, completely satisfies the stakeholders or beneficiaries, let's say, of this automation. And there is no randomness. If you throw the same prompt into different chats, starting them from scratch, for example, analyze a candidate for a given position, for a given vacancy, and you throw it into a resume, then most likely, you will constantly have different assessments, constantly shifting, yes, conclusions. And these are not the automations that business expects, this is not quite what everyone dreams of, and for such things, as a rule, they won't pay much. Schema Guided Reason offers an alternative. It does not assume that the model will have an internal reasoning block working, and instead, chains of prompts will be fed to the model, and it will give the subsequent steps itself. And the most important thing is that the model will always respond in a structured format, in JSON format, which will be parsed later. And first, it will help the model perform tasks in a loop, even if the number of these loops is not predetermined. And second, that the entire process of subsequent processing by other models will go strictly in the same format always, and the model will not steer where it shouldn't. Another important point is unpredictability. It is precisely the Schema Guided Reason approaches that also level out this drawback, that the model paid close attention to something once, the second time it simply missed it, the third time it touched on some topic superficially. Schema Guiding Rizing also solves this problem. And this means that quite stable systems can be built, even on some critical segments. I will give an example of contacting technical support. We must always have a clear step-by-step plan for how the model should behave in a given situation. It must identify the problem, understand what specialist is needed, and what the solution should be. And do this not randomly, but according to a clearly prepared scheme and apply only the necessary level of escalation of the situation. If a person, for example, says, my laptop turned off, then the model can tell him, charge the battery, for example, and solve the problem, figuratively speaking. If a person says, my laptop exploded, there was smoke, then the maximum level of escalation is required, depending on what protocols, what scripts the company has. All this should always work stably, we should always route to the correct segment, to the correct block. And such automations, and they, unfortunately, will always work poorly if we use only a bare prompt in the hope that the model will guess something itself, will think something up. This is pure luck, and business does not really want to rely on luck. And let me show you three patterns that the author of this method had. The first pattern is cyclical. Let me start it right now and explain as we go. Within this example, I have cases for risk analysis, financial risks, and decision-making to mitigate these risks for project work. That is, we have some context, there is a new project that we have come up with, and we want the system to analyze the risks. The most important thing that happens here is that we have an input agent that has a specific step-by-step plan on how to identify these risks. The second important thing is that it has a structured output. Here we completely collect the structure of how the model should respond. That is, we submitted our project as input, and you can see what our output was. First, we get Project Overview, risk factors collected into an array. And the risks are always clearly predetermined. technical risk, that is, a description of the risk itself, its degree of influence, probability, and what strategy to mitigate it. Then it is resource risk, security risk, compliance risk, time risk, business risks. And the model will always clearly output these objects, strictly according to this structure. And depending on what the model gave, we can then make decisions. One of its tasks is to come up with recommendations for some proactive methods or actions to combat these risks, to reduce them, let's say, initiatives to reduce them. After we receive it, we take this array of initiatives and break it down into several different objects. And we launch the processing in a loop, detailed processing of actions to work with these risks. That is, we feed the action here, we feed the additional context of the risk itself. And the model starts to spin it out in a loop, however many there are. If there are 2, there will be 2 cycles. If there are 20, there will be 20 cycles. And in this way, we process all these actions in detail, simply because we have a clearly defined response structure, yes, all collected. It will always be the same, because the model is forced to respond in JSON. If it doesn't work, we connect automatic JSON correction, and we arrange everything correctly in JSON. So, the first component is a well-written prompt, where specific input parameters are clearly defined, and a framework, methodology, approaches to be applied are defined, and the response structure is very strictly defined in the code. Second is the response structure itself, that is, the schema itself, in which everything is written as detailed as possible. Yes, and it is this structure that we receive. Well, and third, when the model finishes working, we will see that the third component will be assembly, dynamic assembly, but using JavaScript, dynamic assembly of the next prompt for the next step, depending on what the result was from the previous model. That is, we take everything we have in JSON as input here, and then we start arranging all the components. Here we can add various conditions, for example, if it is a critical risk, then we use it and add it, mix it all with some additional prompts, with contexts. That is, we get total control over the integration of real business approaches, expert knowledge of specialists, entire departments into this system. If we have figured out how people solve this problem, if we understand exactly what business approaches are, we can implement them here. Essentially, any integration of such automations, well, it is obvious that what I am showing is not particularly suitable for some chat systems, for dialog systems. Rather, these are some background processes that are built into some CRM or ERP. But it is obvious that if we understand, we can implement it. And another thing I wanted to say is that integration is impossible without deep business involvement. We, as developers, cannot get a technical specification and go and build a project risk analysis system. And come back in a week and say, here, guys, it's ready, everything is great, nobody needs this, and it won't work. Any risk assessment specialist will reject this system, saying it will be totally random. Yes. Here we have collected a prompt for validation, that is, we have arranged everything here, collected the necessary context, and thus it is passed to the next agent, whose task is already different. We feed this collected prompt, generated, not randomly generated, but assembled in code, based on the results of previous models, into it. Here we have, in fact, the task to form the final answer, the final report based on the initial analysis, to take into account all initiatives and actions that can be devised to somehow mitigate the impact of these risks, to reduce them. And when this model finishes working, well, it has a rather large prompt. By the way, all models have large schemas like this. Because all this is necessary to construct prompts. Because here the prompt is constructed from here, and from here. After the model responds to it, we also create an HTML file in a very, very convenient way, into which only the necessary components from the answers of all models are substituted, we also solve the problem of reports themselves. That is, they will always be the same, they will always be in the same structure, everything will always be stable, the same, because each element of our pipeline, our conveyor, is strictly typed, and we just arrange them. That is, if we get some numbers, some coefficients, we can calculate all this on a calculator. If there is an understanding of how to do it, if there is access to experts who can say that we apply these formulas, these calculations. And thus, there is a real opportunity to transfer human expertise here, in full or in part, and create truly useful automation. And for immersion, for your immersion in such processes, I think the check will be much higher than for some guy who said, I know how to write prompts in ChatGPT and now I will automate everything for you. This is a little different, as you have probably already understood. Now we will wait for the model to finish working, it failed to fit the schema, so here model 4.1 is rewriting it in JSON, it does this well, the responding model here. By the way, here, when you use such an approach, you can save a lot on tokens, because even mini-models often cope, because the tasks are atomized, they are specific, and due to the fact that we sometimes go into these loops, there is not a very large cognitive load on the model. Our model responded through JSON output parser. It didn't fit right away, but then another model helped it. We eventually executed this JavaScript, and with its help, we generated the report itself. That is, this report looks like this. It will always be the same, with the same number of sections. That is, we have a table here, everything will be divided into pages. Each report segment will always be on its own page. Nothing will be here, no randomness. And people will get exactly what they expect every time. And then it's a matter of competent prompting and transferring expertise at the prompt level. If there is an understanding that the model is not coping, this task can be divided into subtasks and solved in the same way very effectively. Yes. This approach, this cyclical pattern. But we have other patterns besides the cyclical pattern. Let's start the second pattern now, I'll tell you about it too. This is the routing pattern. The principle here is exactly the same, but here we have an initial agent who, in fact, determines the level of escalation of the situation. It analyzes the input data. Here I have implemented it using an example of a person contacting a medical institution. If they have some symptoms, some indicators that indicate a high risk, for example, risk of heart attack or stroke or something else urgent, then our agent makes a decision about the highest level of escalation. It then contacts another agent who handles all the necessary preparatory work, what needs to be done. Now I'll show you the prompt of our agent briefly. So, here are the principles of triage. There is, well, like a triage doctor, a triager, yes. There are routes, emergency, specialist, primary care, and self-care. Yes, and essentially, it chooses where. Now it has decided that this is an emergency, there is a high risk threat, so we went for the maximum level of escalation. Yes, and here we further, I'll show the prompt. We plan what to do next. After planning, we collect all this into such a complex object, where all the necessary equipment, all the necessary actions, everything that needs to be taken from personnel for a visit to this person. Because there is a suspicion of a severe situation with a myocardial infarction. Therefore, all necessary medications, all monitoring equipment are here. That is, everything is prepared, this is the maximum level of escalation. Then it goes separately to specialists, to the team. A paramedic is needed, something else is needed. And in this way, the model distinguishes what plan we have. We also generate a report and calmly we see, here is our specialized medical care plan. We see that it is urgent, we see what research is needed, preparation for admission, we see immediate steps, we see information about specialists, that is, who we will need. Here is a clear step-by-step plan, everything according to the protocol. If it is a medical institution, then all the content of the protocol can be transferred here. Yes. If we, for example, sent a prompt, here, for example, here is such a prompt, just a person, he has a dry cough, he doesn't feel well, let's just try to run it and see what level of escalation was here. And how it will all work further. That is, our agent now accepts the input prompt. After the model finishes working, it shows the lowest level of escalation, that is, self-treatment, and we don't go anywhere else to any models, and we see that here is a man, temperature, just generally with a lemon pie, and it will cost you 0 rubles, and everything will be fine with you. That is, this principle, this routing pattern, and it allows to determine different approaches. It is not necessarily the level of escalation, it can be routing, for example, you need to go to some technical specialist, somewhere else it can be applied well for technical support, some help desk, service centers, any medical institution inquiries. That is, the spectrum of application is very large. And let me also show the third pattern, which is called cascading. The cascading pattern works a little differently. That is, the cascading pattern implies that we have a linear architecture, there is one agent, this agent does something, also responds in a rather extensive, complex JSON schema. After it has done the initial work, we transfer its result and construct a prompt on the fly for the next agent. We also get a certain variability due to the fact that we construct prompts on the fly. In this case, I have test cases here. This is analysis of results, analysis of resumes and vacancies. Yes, in order to automate this initial matching of a candidate with a vacancy. And I have several agents here. The first agent is HR, that is, its task is to understand the basic information about the person. Then we have skill match, that is, how well the candidate's skills match the vacancy. Here too, we have a structured response. And here, by the way, I'll show you, yes, after the first agent responds, we collect all this in code and then feed this constructed prompt to our agent. Now I'll show you. That is, this is the constructed, dynamically constructed prompt, which uses elements of the previous response. Everything is arranged in fields. And similarly, we have an analysis of, let's say, cultural fit, how well they will fit into the corporate culture, into the team, and so on. Here we also extract everything according to the schema, according to how hiring is done in the company, by what criteria to evaluate, if some scoring is applied, we also take all this into account, and in the end we give it to the main decision maker, who receives everything as input, but receives it, let's say, in the format he needs, analysis of the resume, achievements from the first agent, skill match, cultural fit. That is, we have constructed all this from the fields that the previous agents provided in order to maximize the work. All this is also context engineering, when we don't just throw answers and requests from one model to another, but we manually craft, create as prompt engineers using code or models, create new prompts, mix them, arrange certain values in them so that we have the highest quality result, and at the same time it doesn't cost like an airplane. At the output, we have such a recommendation, I ran it about 10 times, it was always 78-79-80%, that is, it never flew off into some randomness, it works stably. And each time this report will be the same, accordingly. It can be made a short version, yes, just for a quick overview and not to read, if the candidate is not suitable, not even to waste time on it. Yes. And this pattern is also very well suited for such linear tasks, when our entire business process is built on the fact that it all comes to one person, they do something, then to another, to a third. And the results of the previous work are inherited and reused later. Yes, and I believe that these three patterns can be used in a huge spectrum, in solving a huge spectrum of tasks. This is marketing, that is, high-quality lead qualification, this is the quality control department, it can be used in production to automate documentation in absolutely any areas related to documents. This is precise substitution for filling templates, analysis of various contracts, analysis of projects, breakdown of some documents into constituent parts. This can be used to automate initial inquiries and identify the right specialist for booking, for reservations. That is, an infinite number of cases can be closed using such an approach. But if you thought it was difficult, yes, it is difficult. As you can see, it is not difficult to program in Python or N8N. The most important thing here is immersion in the business process. Without it, no one will do anything normal, I think, because you don't understand how a particular company can solve problems. If you take some remote beauty salon, you make them some WhatsApp bot, they are happy, then medium and large businesses will not go for automations that constantly generate something unclear, random. And the difficulty is, first, to agree with the client and make him understand that our joint participation gives us a much higher chance. And second, that you will have to immerse yourself and talk to people, communicate, find all these approaches and turn them into such patterns. If you are interested in a detailed guide, if you want to get a workflow from me on how to do it, there is a link in the video description. Give it a like, subscribe to the channel. There are many more cool videos ahead. See you next time. Bye.