Transcription
Modern AI models are huge. These models power large language models, coding agents, image and video generation systems, and protein structure predictors. While pre-trained models are general, we often want to specialize them for domains like law, finance, or healthcare. But we run into a problem for fine-tuning. Updating every weight in the model is extremely expensive. For a 70 billion parameter model, the weights alone can take around 140 GB at 16 bit precision. If every task required its own full copy of the model, just storing all those copies would quickly become unmanageable. In this video, we will explore the key idea of parameter efficient adaptation. Specifically, we'll explain what Laura is, how it works, and share a few recent advances.
In the transformer, much of the work happens in the linear layers. For example, in the attention layer, we use linear layers to create queries, keys, and values. In the feed forward network, we use linear layers to expand the representation, apply a nonlinearity, and project it back down. All of these linear layers do the same basic operation, multiplying a matrix by a vector. We can summarize one of them as Y = W * X. X is the input vector. W is the weight matrix, Y is the output vector. Now, instead of directly fine-tuning all the individual weights, we freeze the original pre-trend weight matrix and add a new trainable matrix called delta W. The delta W captures the change we want to add to the original weight. We can think of delta W as a task specific correction. After fine-tuning on new data, we can merge the train delta W with the original pre-trained weight w0. Merging the weights ensures that during inference, the model runs just as fast as the original pre-trained version. This is possible because W plus delta W can be precomputed as a single ordinary weight matrix after training. But this still does not solve the core problem. We are still training a huge weight matrix. The key idea is to assume that this correction matrix delta W has a low rank structure. The task may require moving the model only in a small number of important directions.
Here is an example. We have a 6x6 weight matrix. There are a total of 36 numbers. But if we look closely, we see that the column vectors are redundant. Except for the first two column vectors C1 and C2, every other column is a linear combination of them. This redundancy allows us to decompose the full matrix into a simpler form. On the left, we have a 6x2 matrix containing the first two column vectors. On the right, we see a 2x6 matrix. Each column in this smaller matrix contains the coefficients needed to reconstruct one column of the original matrix from the first two basis columns. The number two here is called rank. Rank measures how many independent directions a matrix really contains. Let's apply this low rank idea to the matrix update. Instead of learning one big delta w directly, we learn two smaller matrices a and b. B is tall and skinny. A is short and wide. This trick is called low rank adaptation or Laura. By decomposing delta W into these two low rank matrices, we substantially reduce the number of trainable parameters.
In practice, Laura typically scales the update by a factor that controls the adapter strength relative to the frozen model. This factor is alpha divided by the rank r. Dividing by the rank keeps the size of the update roughly stable as we change the rank. So we don't have to retune the learning rate each time we pick a new rank. Given an input vector x, the output vector y consists of two components. The first part is just the original output before adaptation. The second part is the learn correction. But what does this low-dimensional correction actually mean? Let's break the matrices A and B into row and column vectors. Now we can rewrite the matrix B * A as a sum of vector outer products. This expression offers an interesting interpretation. We can treat the row vectors A1 and A2 as keys and the column vectors B1 and B2 as values. The dotproduct between a key and the input vector X measures how strongly X aligns with that key. So the key vector A behaves like a feature detector. The column vector B specifies what correction to add if that detector activates. For example, if we fine-tune a general purpose model on medical literature, key vector A1 might detect hot disease questions while value vector B1 adds cardiology expertise. The key vector A2 may detect diabetes questions and the value vector B2 may add diabetes expertise. This gives us a useful interpretation. Laura behaves like a form of local associative memory. It's considered local because if the input vector X isn't relevant to the specific domain, the similarity scores tend to remain small and the adapter introduces little or no correction. The base model keeps its general knowledge and Laura adds a compact task specific memory bank. This property also makes Laura practical for different tasks. We can keep the same frozen backbone and swap in different adapters. We can even compose multiple Laura adapters and adjust their relative contributions.
Now that we know what Laura learns, the next question is how to start training it safely. A natural choice is to initialize the weight matrices with random values sample from a Gaussian distribution. However, if both matrices B and A are initialized randomly, the output correction delta Y will generally not be zero at the start of training. This is undesirable because it means the adapter is already modifying the model's outputs before training has even begun, which can hurt stability and interfere with the pre-trained model's knowledge. Therefore, we want the adapter to introduce no change initially so that any corrections it learns are entirely due to fine-tuning rather than random initialization. Setting both matrices B and A to start at zero does the job. The initial output correction becomes zero no matter what the input vector is. But if both B and A start at zero, then during back propagation there is no learning signal; the adapter is dead on arrival. So one matrix should start random and the other should start at zero. This gives us two possible choices. We can start with a random matrix B and a zero matrix A. At the very first training step, the matrix B receives no gradients, but the matrix A starts learning. After the first step, both B and A can learn without problems. Alternatively, we can start with a zero matrix B and a random matrix A. This approach also ensures that the initial output correction is zero for any input vector. At the first step, the matrix A receives no gradients, but the matrix B starts learning. After the first step, both B and A can learn. Both options work. Laura normally uses random A and zero B. With random A and zero B, the adapter initially does nothing, but B can immediately receive a useful gradient. Once B moves away from zero, A can begin learning as well. Why is that? The matrix A acts like a feature extractor. It projects the input into the low rank space. The matrix B then maps those low rank features back into the model's output space as corrections. Initializing A randomly and B at zero gives the adapter diverse initial feature directions while ensuring the initial correction is zero. By contrast, if A started at zero, the low rank features would all begin at zero and only grow slowly, giving B a weaker, less diverse signal early on. Random matrix A provides a rich set of directions from the very first step.
Since matrix A and B play different roles, we may want to use different learning rates for them. Laura Plus assigns a smaller learning rate to matrix A and a larger one to matrix B. The reason is that A and B operate at very different scales. The matrix A maps a wide input down to a few dimensions. The matrix B maps those few dimensions back up to the full output space. With equal learning rates, B tends to be under trend. Giving B a larger learning rate helps keep both mattresses learning at a match pace. This simple change leads to faster fine-tuning and often better final performance without additional parameters.
Although Laura adapters are compact and easy to train, a major challenge remains memory usage. Let's walk through an example. Suppose we want to fine-tune a base model with 70 billion parameters. We set the rank R as 64 and adapt all linear layers. In this case, the number of trainable parameters is 587 million. This amounts to less than 1% of the parameters in the base model. The trainable parameters only occupy 1.17 GB of memory when using 16bit precision. But the base model is huge. It takes up around 140 GB in 16 bit precision. The key idea for making fine-tuning more broadly accessible is to compress the frozen base model using quantization. The base weights are stored in low precision, but during computation they can be dequantized on the fly into a higher precision format used for matrix multiplication. Only the adapter weights stay in full precision and remain trainable. If we can do this, fine-tuning the entire system can fit on a single 48 GB GPU. This is called quantize lura or Q lora.
Let's see how we quantize the weights. We start with the raw weight values in 16 bits. We identify the maximum absolute value in the group and rescale the values so they fall between -1 and one. With four bits, we only have 16 possible codes. But we want the levels to be symmetric around zero. The most negative and most positive codes mirroring each other. And we also want a code at exactly zero. A symmetric set that straddles zero this way must have an odd number of codes, yet four bits give us an even 16. To ensure symmetry and to ensure zero can be represented exactly, which is important for neural networks, we use the range between minus7 and 7, dropping the extreme code minus 8. This way there are equal numbers of positive and negative quantization levels and zero is directly encoded. The quantization process snaps each normalized value to the nearest 4 bit level. The frozen base model becomes much smaller while the lower adapter remains trainable in full precision. We get the benefit of a huge pre-trained model but we only pay training memory for the small adapter.
But there is a problem. The network weights are not usually evenly distributed. Most weights are clustered near zero following a normal distribution. If we choose 16 quantization levels with equal spacing, we waste precision. Many codes end up in regions where weights rarely appear, while the region near zero, where most weights cluster, lacks sufficient resolution. To address this, we use quantile quantization tailored for zero mean normal distributions. This approach places the levels so that each bin holds an equal share of the weights. The bins are narrow where weights are dense, near zero, and wide out in the tails. In other words, we spend more levels where the weights actually are. Unlike the uniform grid where we drop the code to stay symmetric, NF4 keeps all 16 non-uniform levels. But because 16 is even, no level lengths exactly on zero. So we build the levels asymmetrically. We create eight levels from -1 to zero and nine levels from 0 to 1. Both halves include a level at zero. So we remove the duplicate. 8 + 9 - 1 gives exactly 16 levels. This is called normal float 4 or NF4. NF4 is not just an ordinary 4-bit integer. Its levels are chosen specifically for weights that are approximately normally distributed. In other words, NF4 spends the 4bit budget where it matters most. This reduces quantization error and represents neural network weights.
Well, there is another problem. If we use one scale C for the entire model, a single outlier can dominate the scale. For example, suppose one outlier weight has a value of 4.17. So the scale C becomes 4.17. When normalizing the weights according to the scale, all the weights get squeezed into a tiny region near zero. To alleviate this, we split the weights into small blocks. Each block gets its own scale. A single outlier no longer ruins the scale of the entire matrix. This is called blockwise quantization. But now we have many scale values to store. If every scale is stored in 32 bits, then for a block size of 64, this requires 0.5 extra bits per parameter. So cqora further quantizes the scale values themselves into 8 bit values. Since we quantize both the weights and the scales, we refer to this as double quantization. So with blockwise NF4 quantization and double quantization, Qura enables fine-tuning of a 70 billion parameter model on a single 48 GB GPU. Qura enables more people to fine-tune AI models even on modest hardware.
Now let's push the idea even further. In Laura, we learn two matrices B and A for every adapted linear layer. These matrices are already much smaller than the original weight matrix, but across many layers, they can still add up. To make adaptation even more parameter efficient, Vera uses a single pair of randomly initialized matrices, freezes them, and shares them across all adapted layers. The model no longer learns all the entries inside B and A. Instead, it introduced trainable scaling vectors, enabling each layer to adapt individually. So, Vera shifts the learning problem from learning new directions to learning how strongly to use a shared set of frozen random directions. Each diagonal matrix rescales the shared frozen directions. One acts on the R low rank directions, the other on the output rows. The scaling vector B starts at zero. So the weights are unchanged at first while D starts from a small tunable constant value. Vera produces weight updates with this equation. With Vera, the number of trainable parameters becomes much smaller. For example, suppose the input dimension is 4,96 and the lower rank is 64. Suppose we add adapters across 80 layers. Standard Laura trains the matrices A and B for each layer. That is roughly 42 million trainable parameters. In contrast, Vera only trains the scaling vectors. That is roughly 333,000 trainable parameters. In this example, Vera uses about 126 times fewer trainable parameters. Instead of directly learning the low rank directions, Vera reuses a fixed set of random directions and learns how strongly to use each one.
But there is one more limitation. Laura adds a low rank correction to the weight matrix. Here is a 2D illustration. This is the frozen pre-trained weight W0. We add the correction weight delta W and form the new weight W prime. This correction can change both the direction of a weight vector and its length. But the two effects are tied together. In this case, the new weight W prime results from slightly adjusting the direction and increasing the length. However, Laura couples changes in direction and magnitude. So, it cannot control them separately. So the key idea is to learn to correct the direction and flexibly update the magnitude separately. With this intuition in mind, let's make the idea more concrete. Since each row in the weight matrix corresponds to one output neuron, we decompose each row into its direction and magnitude. For each row, we compute its norm m_sub_1, m2 and so on. This means the pre-trend weight matrix W0 can be written as the product of a diagonal matrix of magnitudes and a matrix of unit directions. The diagonal matrix contains the per row norms and W hat contains the normalized direction vectors for each row. This decomposition is powerful because it allows Laura style updates to concentrate on updating the directions while still enabling flexible adjustments to the magnitudes by learning separate magnitude parameters. The adapter gains an additional degree of flexibility. The Laura style update then handles the direction. This is called weight decomposed low rank adaptation or doro. This added flexibility lets Dora behave more like full fine-tuning while still using very few additional parameters. This matters because full fine-tuning tends to change a weight's magnitude and direction fairly independently while Laura's coupled update changes them together and decoupling them is exactly what closes the gap.
Of course, there are some trade-offs. First, while inference speed remains the same, the weight decomposition slows training. Second, unlike Laura, Dora does not support the same simple linear composition of multiple task adapters because the normalization step makes the update nonlinear. Nevertheless, Dora's strong performance has attracted significant attention from the community. It's incredible that a tiny well-designed adapter can steer a massive pre-trained model in powerful new directions. Thanks for watching and I'll see you next.