Transcription
One thing that large language models are very good at is in-context learning. You give a few input and output examples to a model, and afterward, you ask it to complete a task. Surprisingly, even by not being specifically trained for it, the model is able to intelligently complete it. How are these models able to do that, and what is exactly in-context learning? In this video, we will explore the recent theory about how this behavior is possible and how the research community currently thinks in-context learning works. We first go through some definitions to set up the table. Then we'll highlight why, from a machine learning standpoint, in-context learning is a surprising behavior of LLMs. We'll then explore the first hypothesis of how in-context learning works, which takes the form of a Bayesian framework. Finally, we'll explore a second connected hypothesis which hints towards large language model training in some ways implicitly during in-context learning tasks.
Disclaimer: This area of research is still in heavy development. If you're listening to this in the future, do put this into context. So let's start with some quick definitions. A prompt that leverages in-context learning will have the following four elements: an input distribution, a distribution of output, both of which are mapped together in an input-output mapping, and finally, a format in which the in-context learning prompt is presented. Each of these elements plays a role in the in-context learning performance of a model. The term in-context learning was coined in the paper "Language Models are Few-Shot Learners." This paper was the one that introduced GPT-3 in 2020. In the paper, the authors from OpenAI realized that in various tasks, the large language model seemed able to understand what to accomplish with enough examples and to do so with no change in weights. As you can see, when the 175-billion parameter large language model is given one example in context, its performance on unseen tasks increases dramatically, even more so when the prompt is given with natural language. As the number of examples in context increases for all of the models, the performance increased in a linear way. This difference in performance is especially present in larger models. The in-context learning task can be varied and can range from learning to do addition to learning how to unscramble words to translating from English to French in the proper format.
Now, why is it so surprising that, given a bunch of examples, the models can learn to do a task right? GPT-3 isn't the first model that is able to do few-shot learning, as other meta-learning paradigms were developed years before, like in this paper which used an LSTM as a basis model for meta-learning in 2017. In this setup, there were two models: a meta-learner that is training another learner on a very few examples to complete a task. What is surprising in the context of large language models is that they are not especially conditioned for this meta-learning. Broadly speaking, GPT-3, where in-context learning was first witnessed, is trained mostly in an unsupervised fashion to predict the next token. Nothing about this setup conditions explicitly for learning with few-shot examples. By the way, this slide was stolen from Micha Laskin on Twitter; it's a great account to follow for deep learning content. So going from next-token prediction to solving complex problems in a structured way with little examples is a very big leap forward in terms of ability.
So how does in-context learning work then, if it wasn't explicitly trained for? The first hypothesis that starts to explain this phenomenon is a Bayesian framework built out in 2022. The framework is mainly built up from these two papers from Google at Stanford and Min from the University of Washington. The core idea is that when given the prompt, the language model will infer latent concepts from the prompt and retrieve the same concept in its pre-training distribution. In the example to the left, the main latent concepts are finance, news, and sentiment analysis; while on the right, the main concepts are news topics for classification. In simple terms, the framework for in-context learning defines a kind of probabilistic retrieval process that is guided by the prompt concept distribution. As the examples are given as input to a trained language model, it will search the concepts learned within its weights. Then it will retrieve the most likely concept and output a likely answer to the prompt. That knowledge of all these latent concepts comes from the pre-training data concept distribution where the model learned and stored them somewhat in its weights. Now that pre-training concept distribution is built up of many, many documents—billions in the case of GPT-3. These documents could be seen as being generated from latent core concepts. To go from the pure latent concepts to the pre-training documents, a sampling procedure has taken place from a human at some point in time. The sampled concepts were then used to generate a document and crystallize one or many concepts into the pre-training data for the LLM. A similar flow can be sketched out for the prompt where the prompt is built out by first sampling from latent concepts, and each of the examples in the prompt are generated in an independent and identically distributed manner by humans and crystallized into the input for the LLM.
If you look at an example, we have the following from an X paper: The prompt is "Alberstein was German, Mahatma Gandhi was Indian, Marie Curie was," and the output is "Polish." Now the general concept we're talking about from the prompt is sharpened. The more examples we give, the concept seems to be related to weakly informative biographical information and nationality. Once this concept is properly teased out by the model, it can then infer the most likely output given the concept and the prompt. There's an interesting phenomenon of sharpening the concept signals here. In this particular framework, you can separate the information within the prompt into two things: signals, which are the information-rich portions of the prompt; and noise, which are regions in the prompt that could potentially add confusion to the understanding of the in-context learning task. As you can see in this example, the transition within training samples has a lot of signals, while transitions between examples add in noise. The results obtained by Min were pretty well aligned within this framework. One of the most interesting results is this graph, which compares two models together in various ICL—so in-context learning tasks. In the first type of ICL task, when we do not give a prompt example for a given task, the model has some baseline level of accuracy. We can see that a larger model like GPT-3 seems to be doing generally better than the others. Now, when we give prompt examples with the right labels, we increase the accuracy by a large amount. Again, the performance here is dominated by the largest model. However, randomizing the output label in the example barely changes anything compared to having the right label; the models are still better than without examples.
Others investigated further by manipulating each of these four aspects of a prompt we saw earlier. Remember, we don't only have the input and output distribution; we also have the format and the input-output mapping. What they saw was the following: Having perfect in-context examples is usually the best, as expected. Removing the right input-output mapping—a.k.a., scrambling while keeping the right output space—is not changing anything much. Yet the in-context learning deteriorates way more when you remove the input space information; for instance, when you sample random words from a document to replace the actual proper in-context input. To summarize these early findings, the input distribution signal is really key. The output space and format signals are also important, but surprisingly, the input-output mapping signal is not that important, which means you don't need to have the right output for each example for the model to understand the in-context learning task. This is a pretty big deal. These results were concurrent by other explorations of in-context learning by Anthropic, namely, which discovered what is called induction heads in the Transformer architecture. The induction heads are defined as follows: Induction heads are named by analogy to inductive reasoning. In inductive reasoning, we might infer that if A is followed by B earlier in the context, A is more—more likely to be followed by B again later in the same context. Notice that induction heads are implementing a simple algorithm and are not memorizing a fixed table of N-R statistics. The rule "A, B...A, then B" applies regardless of what A and B are. The important part is that, after studying these induction heads, the authors realized that the models were implicitly implementing some type of algorithm, or at least using them, which is intimately connected to the second hypothesis. The hypothesis is that the large language models are, in fact, implementing learning algorithms implicitly to do in-context learning, and that actual learning is happening. The notion of Transformers being able to implement some form of learning algorithm internally is a recent and intense area of research. It seems like, in theory, the Transformer architecture can do in-context learning to something closely resembling gradient descent or other learning algorithms.
Let's see a very concrete example from Google Research that ties in the results we just found out about the input-output mapping not mattering for the in-context learning. The models in question were the whole family of GPT-3, Str, GPT-Codex, PaLM, and FL-PaLM, from small to large. There were various tasks that were studied with three main ICL techniques: regular ICL, which involves giving examples with a proper four-parameter of I-prompt; flip-label ICL, where the examples have the opposite meaning they should have; and finally, semantically unrelated targets, which mean that the elements have no connection between the input and the output. For instance, in a sentiment analysis task, "positive" might be replaced by "apple," and "negative" might be replaced by "oranges." So we change the output distribution here. In the first result, we'll explore the authors' use of the flip-label flavor. What they saw is the following: On the y-axis, we have the accuracy on the given task; on the x-axis, we have the percentage of flip labels in an in-context learning prompt. This is per family of models and per model size. As you can see, with the right labels given in the prompt, most models are running better than chance, except some variations of GPT-3. However, as the number of flipped labels reaches 100%, the largest models are making guesses that are worse than chance. As we've seen previously, this result is a bit weird because it contradicts what we've learned in the Bayesian framework, where the input-output mapping wasn't that important. It seems like their largest models are learning to stop following what their pre-training data distribution is telling them about that latent concept, and instead, they are listening to the input-output mapping signal. Now, in this instance, the larger models are more correct than the smaller ones due to the nature of the flip labels.
A second hint that some sort of on-the-fly learning is occurring comes with a semantically unrelated target ICL experiment. In that setup, we have the various models and their variants doing a task, either prompted with regular ICL or the semantically unrelated ICL. What we see is that the biggest models, except GPT-3, are almost as good in the two scenarios, effectively learning something that wasn't in their training distribution, now being able to use that during testing. Yet the smaller models have a harder time, being very close to random with a semantically unrelated target. The authors concluded with the following: These results underscore how the in-context learning behavior of language models can change depending on the scale of the language model, and that larger language models have an emergent ability to map input to many types of labels—a form of true symbolic reasoning in which input-label mapping can be learned for arbitrary symbols. So, as models scale, they gain an emergent ability to create or use functions to map input to output, like we've seen with the induction heads in Transformers. There's a mounting amount of research in that line of thought that the larger language models might be doing something close to learning on the fly, although it's still an area of intense research. This seems to be something that is not only theoretically feasible but also something the larger models are actively doing during testing.
If you want to learn more about in-context learning, I would recommend this blog post by the Stanford AI Lab, which does a good job of summarizing the subject, as well as this NeurIPS article, which includes the perspective of learning on the fly. Both will be in the description along with dozens of papers presented. I hope you enjoyed the video. Don't forget to like it if it was the case and leave a comment if you have any questions; I'm here to help. Have a great week everyone, and see you in the next video. [Music]