Transcription
Good day. They say that supposedly an AI has appeared that thinks for itself. Not just reasons, but truly thinks on a given topic, solves a complex assigned task, using an unprecedented volume of reference data. Have you heard of this? Have you really not heard? A new hype is being generated around this. Although you and I know that no AI can think yet. I know my listeners will not be surprised. And many will immediately say: "What's new here?" Flagship bots already reason like this and show their thought process. And few people can do that. However, it seems that engineers have indeed figured out how to extract an analogue of the thought process from a neural network. So what have they invented? How do models think now and how can they be made to think using this new engineering solution. To begin with, how a reasoning model thinks by itself, yes, just like it does everything else. In the giant training data, there are many examples of reasoning, about which it is also stated that they are reasoning. Well, and purely statistically, the model guesses how to reason. This, as they say, is by default, based on the results of preliminary training. Then, during fine-tuning, reasoning models are specifically further trained on examples of reasoning, on entire arrays of training data and continuous reasoning. In addition, there is a mandatory reinforcement learning stage, when a human or another model evaluates the answers of the trained models. At this stage, only answers containing reasoning are selected, i.e., reinforced. Among the answers containing reasoning, those with the most complete and correct reasoning are reinforced, and so on. I am, of course, simplifying somewhat. Well, roughly so. Next. No magic. The same linear process of transforming a query into a generated set of words. Any query comes to a model trained for reasoning, but along with it, instructions are loaded on what to do with this query. For example, when to reason and when, under what conditions, to search for additional data in external sources for context, using the same RAG. For those who suddenly don't know or have forgotten what RAG is, watch my video on this topic. We will mention it again here a little later. You can even instruct the model to check what has already been generated, but it does not have addressable reasoning steps internally. Therefore, you need to directly tell the model: "If, for example, these phrases appear in the context, call RAG" or check the generated text against internal knowledge. LLMs generate output text token by token, i.e., word by word, continuously. Each generated token is added to the context, so that the next token is generated taking into account the previous one. It's not like something is brewing inside the neural network, brewing, and then suddenly an answer is printed. All the model's reasoning is usually part of the generated response. And only the interface settings determine whether to show them or not. It must be said that the reasoning of modern models is impressive, but these are all statistical probabilistic results, albeit very impressive. I once said that the future of practical AI application is not in bare probabilistic language models with their statistical map of language, but in hybrids of LLMs and external logic. By the way, if the logic external to the LLM operates automatically according to given rules and conditions, it is called agentic. There even arose a meme, agentic AI, essentially contrasted with generative AI, but we will talk about it separately, it is worth it. And finally, about the main hero of this video. Engineers from MIT have proposed a framework called Recursive Language Model (RLM), in which an LLM can solve complex analytical tasks based on very large data that exceeds the context window of any model. This framework they called Recursive Language Model. Recursive Language Model RLM. This is precisely what some are trying to call autonomously thinking AI, although it is just another variant of agentic wrapping around an already familiar LLM, and not a new model at all. But if an LLM beautifully guesses the reasoning process and the answer, then RLM performs a recursive process, processes practically any volume of external data step by step, returns to what has been processed if necessary, stores intermediate results and checks them, and then puts everything together. And all this is outside the language model, but with its help, economically using its context window. Since RLM stores intermediate results and logs their receipt, one of the problems of modern AI is solved. The process of obtaining an answer can be made repeatable and explainable. And this is the path to the application of AI. In systems where, with an abundance of input data, it is necessary to precisely state why a particular solution is proposed and what its justification is. This is the path to the very explainability of the result, which a pure LLM lacks. Now, how does it work? I have provided a link to the original publication in the description of this video. Moreover, I am simplifying in some ways, and in others, on the contrary, developing the developers' ideas. So, everything consists of an orchestrator, a regular computer program, and again, one or more models that perform three roles: planner, executor, and critic. These roles are performed sequentially, so it can indeed be one model. Although it seems to me that for different roles, models should be configured and trained differently. All communication happens through the orchestrator. He receives a human task. He asks the planner to create a plan for executing the received task. It is very important that all instructions, all the logic of the orchestrator, all settings, limitations, priorities are embedded in it by the developers. All its autonomy is equal to an automatic program executing tasks. And neither the orchestrator nor the language models decide anything themselves or do anything on their own initiative. Having received the plan from the planner, the orchestrator sends it to the critic. The critic, as expected, criticizes, and the orchestrator sends the initial request, the previous plan, and the criticism back to the planner, demanding that the remark be taken into account. The planner obediently sends a new version of the plan. That is, it is clear that such recursion can already make the initial plan better, since criticism will be taken into account. However, the plan improvement procedure can be repeated several times, which is usually done. Since there is no criterion for plan perfection, the orchestrator is simply given a setting, say, to perform three improvement cycles. The orchestrator can find out from the planner or executor what clarifying questions they have, and then conduct good old semantic search in an external library. We know that there have long been very sophisticated, effective methods of data search, including semantic ones. If a regular RAG, as we know, takes the aggregate vector of the current context and searches for several similar vectors in external embeddings, then text search algorithms like Text-to-Text Transfer Transformer (T5) will scan all available documents, select, compare, sort, and output practically everything that is in the sources related to the query. To be fair, in principle, one can build a RAG that in a recursive system like RLM will probably not be worse than T5, but RAG is fundamentally for something else. It is good where similarity is sufficient, speed is important, and exhaustive information is not required. The task for the first step, along with the exhaustive and compressed information found in the sources, is finally sent to the executor. The latter responds, and his answer, along with the request and information, goes to the critic, who again indicates what is wrong and what needs to be clarified. A repeated search in the sources is performed, and its results, the initial request, and the criticism go to the executor again. Thus, for a given number of times, until the answer is recognized as final, but not by any criteria, but simply because the number of recursions has ended. After each final answer for a step, the remaining steps of the plan are sent to the planner along with the already received answers and the request: is it necessary to adjust the plan? Accordingly, the cycle of developing a new version of the plan is repeated. Thus, all steps are passed one by one. The orchestrator stores all iterations and their results. And when the last item is completed, he receives the final answer from the executor and runs it again through the critic. Cumbersome. Yes, but it's almost exactly the same as what we humans do when analyzing. Not to mention that a person can be lazy, for example, to criticize their own plan or conclusion, but computer programs cannot be lazy. A good example of RLM application is solving the needle-in-a-haystack search problems, when you need to find all mentions of a term, fact, concept, phenomenon in the correct context, in the entire data array, and not miss a single one. And logical problems with non-obvious connections and large nesting. Here I had a long example of a query from an article, but I removed it for brevity. Honestly, that query reminded me of a riddle. Two geese were flying, one south, the other gray. How much do nails cost at the market if there is no water in the tap? But, unlike this anecdote, the query was solvable. There is also a class of analytical tasks in which LLMs seem to have no equal. This is finding relevant pairs in a large volume of data. Something like finding all mentions when x = y, provided that z > n. As you can understand, the conditions can be much more complex. The important thing is that if you need to sift through the entire, without exception, huge volume of external data, a regular RAG simply cannot cope. It finds a given number of best options, and increasing the number of these options is limited by the context window. And here there is no need to inflate the language model's context window, because all data is stored and structured by an external program. And finally, the continuous probabilistic generation process of the language model is embedded in a sequence of verifiable steps with a given number of iterations. I note, this is an understandable and repeatable process. Where do the enthusiasm and hype come from? I cannot say with certainty, but I can assume that RLM is better than LLM with any RAG because it does not try to guess, but is obliged to check, i.e., it approaches the reliability of the answer. Usually, LLM+RAG searches for similar fragments and constructs a plausible answer, not knowing that it might have missed something. LLM reads data step by step, returns to what has already been processed, and records the result. This makes it slower, but more reliable where completeness and verifiability are important, not a quick, beautiful answer. However, the answer will still be very beautiful. It, as usual, will be generated by the LLM. It must be said that the authors of the framework see many shortcomings in it. They have a whole section on what did not work out. I would call it more of a list of unfulfilled expectations. I have described the recursive model more as an example of the very agentic AI, which we will discuss later. But even so, the idea of recursion as an external mechanism for squeezing the maximum out of a language model, isn't that a discovery? For example, immersion in context according to Larian, using clarifying and leading questions step by step. An excellent application for RLM, in my opinion. Well, what do you think, how can this be used for your needs? If you are at a level where RAG and probabilistic generation will stop working for tasks, and a recursive process will be required. Such a system is not difficult to create with current programming tools. I, for example, would use it to search for implicit patterns in open sources. If you know what I mean.