We are into the part two of fine tuning series, and in this particular video we are going to discuss
about Laura and Laura in depth.
Intuition.
Already in the playlist of fine tuning I've discussed about quantization and I hope you have seen the
video early.
Many people were requesting for Laura and Laura also.
So let's, uh, discuss, uh, and uh, we will try to discuss the complete in depth maths intuition.
Uh, the best thing will be that guys, I'll try to explain this maths in depth.
Intuition.
Uh, I've already seen the research paper, and there are a lot of complicated things that is probably
there in the research paper.
But I will try to teach you in such a way that at least you should be able to understand, you know,
what exactly is Laura?
What exactly is Laura?
And then we'll also see one example.
You know how with the help of code, you will be able to do it And trust me, these are some of the
very important things in fine tuning.
Because tomorrow you go in any interviews, you are going to get asked respect to this kind of questions
that may be coming because at the end of the day, any generative AI projects specifically with LLM
right, LLM models, if you are working in the company, they will be giving you fine tuning tasks to
talk about the research paper.
So what does Lora means?
Lora basically means low rack adaptation of large language models.
Uh, this is this amazing research paper.
And, uh, probably you'll be seeing a lot of this kind of equations as you go ahead.
They'll be different, different performance metrics.
But as usual, I will do what I'm good at.
I will try to break down all these things and probably explain you with respect to examples, with respect
to code and many more things.
So quickly, let's go ahead.
So why Laura and Laura is basically used?
Laura basically means low land, low rank adaptation, lower order rank adaptation.
It is specifically used in fine tuning of LM models.
Okay.
So let's go ahead and discuss this.
And I've created this amazing diagram over here.
Initially whenever whenever you have a pre-trained model that basically means let's say that there is
a model something like GPT four or GPT four turbo, right?
And this model has been created by OpenAI.
So we basically say this model as the base model okay, GPT four turbo and this model is trained with
huge amount of data, right.
So the data sources will be internet.
It can be books, it can be multiple sources.
Right at the end of the day, all these models, how you can measure them.
They may be saying that, hey, uh, it supports 1.5 million tokens.
You know, it has been trained with this many number of words, right?
So many tokens of words.
Now what will happen is that all these models, you know, probably to predict the next word, it will
have the context of all those tokens and then it will be able to give you the response.
So this all models are basically base model.
And we also say this as a pre-trained model okay.
Some of the examples.
Again I'll be telling you GPT four GPT four turbo right GPT three GPT 3.5 something.
So all these models are specifically pre-trained models.
Now further, we can take this model and there are various ways of fine tuning it.
Please make sure that you watch this video till the end.
If you watch this video, you will understand everything that is actually required with respect to fine
tuning and there are multiple ways of fine tuning also.
Now let's say that I take this model.
I do some amount of fine tuning okay.
And this fine tuning is done on all the weights of this specific model.
Right.
So some of the application that we may generate is like ChatGPT, you know we may generate like cloud,
uh, cloudy ChatGPT.
Right.
Cloud, GPT itself like the chatbot that we specifically use.
Some of the examples okay.
So this one way of fine tuning is there where we train all our parameters.
So here we specifically say full, full, full parameter training okay.
So here you can see full parameter fine tuning.
So here I'm going to write full parameter fine tuning.
Now this is one way of fine tuning where we train our entire parameter based on our data that we have
okay.
And after training it we can develop applications like ChatGPT or any custom GPT that you specifically
create.
As we go ahead, you can also take these models and perform domain specific fine tuning.
Okay, so one type of fine tuning is this the other tune, other fine tuning technique that you can
specifically use is called as domain specific fine tuning.
Some of the example let's say that I'm fine tuning a chatbot model, you know, which will be for finance.
It can be for sales.
It can be for different different domains itself.
Right.
So here the main important word is domain.
So this fine tuning also we can perform okay.
Again why I'm saying all these things.
Because there are various ways of fine tuning things.
Right.
One more fine tuning we can basically divide by is something called as specific task fine tuning.
Now in case of a specific task fine tuning.
These are my different different task.
Let's say this is task a, b, C, d.
This task can be something related to Q&A chat bot.
This task can be something related to document Q&A chat bot different different applications.
So that is the reason why we are specifically saying over here as specific task okay.
Specific task fine tuning.
Now perfect.
Now this is good.
You have seen all the different ways of fine tuning.
Okay, now let's talk about full parameter fine tuning.
Again I'll repeat it.
This is my base model right now.
If you use it as an example, as I said GPT four turbo, GPT 3.5, you know Gemini.
Gemini 1.4 different different models can be there.
We can take this base model.
We can fine tune and create applications like ChatGPT.
We can create other other applications like stable Diffusions, you know, not specifically to LLM,
but Liam, we can actually do it.
Then we can also further fine tune this based on domain specific fine tuning, right?
Based on different different domains like finance, sales, retail.
We can also take this model and do more specific task fine tuning like task A, task B, task D.
Let's say I want to convert this into text to SQL.
I want to have this as document Q&A so I can further fine tune it based on specific task.
Now let's talk about this full parameter fine tuning okay.
And what are the challenges with full parameter fine tuning.
And that is where see I'm building up the story.
Later on I'll be explaining you where Laura will be used now in full parameter fine tuning.
The major challenge is that we really need to update all the model weights.
Let's say that I have.
I have a model which has somewhere around 175 billion parameters.
That basically means 175 billion weights.
Now, in this particular case, whenever I fine tune this model, I need to update all the weights.
Great.
Now when I'm saying updating all the model weights and when we talk about this many number of parameters,
white can be a challenge because there will be hardware resource constraint.
Right.
So with respect to different different task, if I really want to use this particular model that much
Ram, I really require for inferencing purpose, that much GPU I really require, right.
So for downstream task it becomes very difficult.
Now what is downstream task.
Downstream task.
Some of the example is like model monitoring.
Right model monitoring.
The other task can be like, uh, model inferencing.
Right.
Model inferencing.
Similarly, the GPU constraint that we may have the Ram constraint that we may have.
So we may face multiple challenges when we have this full parameter tuning file for full parameter fine
tuning.
Now, in order to overcome this challenge, we will specifically use Laura and Laura.
Okay.
What exactly is Laura?
As I said, low order rank adaptation and Laura is something.
It is also called as Laura 2.0.
So we'll discuss about both of them with respect to mathematical intuition, and you'll get a complete
idea what I'm actually trying to say.
Now what does Laura do?
Okay, now let's read the first point.
And this is very much important because in the research paper you will find this equation okay.
This equation.
Now what exactly Laura will do.
Laura says that instead of updating all the weights in full parameter fine tuning, right.
Instead of updating all the weights, it will not update them.
Instead it will track the changes.
It will track the changes.
Now what changes?
This is basically tracking.
It will track the changes of the new weights based on fine tuning.
Okay.
This is very much important to understand.
So uh, based on the new weights, how we are going to combine these weights with the pre-trained weights.
Okay, so here you can see these are my pre-trained weights from the base model.
Like let's say that, uh, this model is llama two.
Now if you are performing fine tuning using Lora, then Lora will track the new weights over here,
which will be of the same size.
Okay, so let's say if this, uh, weight is three cross three, then the new weights, when it is probably
doing the forward and the backward propagation, those new weights will be tracked in a separate matrix.
And then these two weights will be combined wherein we will get the fine tuned weights.
Now this way what will happen is that this tracking will happen in a separate way.
But still you may be thinking Krish, here also we are updating all the weights itself, right?
So here also the resource constraint will definitely happen.
Yeah fine I'm talking about three cross three.
But what about uh weights and parameters where they are 175 billion.
Right.
175 billion parameters or 7 billion parameters.
That time I'll be having a huge matrix.
Right.
So at that scenario you need to now understand how Laura works because this weights how it is getting
tracked.
It will just not get tracked in this three cross three matrix.
Instead all these weights that is getting tracked there, a simple mathematical equation will happen.
Or I'll not say mathematical equation will happen, but there'll be a technique that will happen which
is called as matrix decomposition.
That basically means the same three cross three matrix is saved it in a two smaller matrix.
Now in this two smaller matrix you can see this is nothing but one cross three and this is nothing but
three cross one.
Right.
Sorry.
This is three cross one and this is one cross three.
Right.
So this is three cross one and this is one cross three.
When we multiply both these weights then I will be getting this weight only right.
So over here if I consider I have some around nine weights 456789.
Right.
You'll be able to see that I will be able to get all these nine weights from how many number of parameters?
Just six parameters.
Right.
Because when we multiply this then you'll be able to see that I'll get all these nine parameters or
nine weights that I have.
Right.
So in short, what Laura is doing is that it is performing this matrix decomposition where a big matrix
and this matrix can be of any size is decomposed into two smaller matrix based on a parameter, which
is called as rank.
How to calculate a rank of a matrix.
You can definitely check out any YouTube channel.
It is a simple algebraic equations based on transpose of a matrix.
How we calculate the rank.
But let's say that this matrix that I have, which is a three cross one over here, the rank of this
particular matrix is one.
Right.
And if I use this two matrix, you can obviously see that the number of parameters that I'm storing
over here is less when compared to this, right?
Yes, there will be a loss in precision, but it is making sure that when we combine both these metrics
will be able to get the entire updated weights.
Now just imagine start thinking guys, let's say that if I have 7 billion parameters now I'm trying
to perform fine tuning on those parameters.
So whenever I track those weights, this huge matrix will be decomposed into two smaller metrics.
And when we are decomposing this this matrix, this updated tracked weights matrix into two smaller
metrics obviously will be requiring less parameter to store all these values.
Right.
And this way your fine tuning becomes very much efficient.
And this really solves the resource constraint.
Right.
This is the most important thing.
Right.
And so in any research paper that you go ahead you'll be seeing this equation w zero.
This is my pre-trained weights.
Plus the tracked changed weights is nothing but my pre-trained weights.
Plus B multiplied by a.
What is b.
B is this a.
Is this.
So when we multiply this you'll be able to see that we are able to get the all the track changed weights.
Right.
And obviously this requires less parameter if you are you're decomposing our bigger matrix into two
smaller matrix less parameters is required.
Now what will happen if we keep on increasing the ranks?
If we keep on increasing the ranks, this parameter will also keep on increasing, but it will always
be less than this, right?
If I have 7 billion parameters, if I try to decompose that into two two matrices, two smaller matrices
with increasing rank, then also the parameters that will be required will be less.
How I am saying this because in the research paper also they have tried with multiple trainable parameters.
Now let's see over here there are multiple techniques of fine tuning.
Some of the techniques that are there is something called as prefix embed prefix layer adapter.
Adapter is one very famous thing that is probably used before Laura.
right?
You can see as the rank is increasing the parameters also increases, right?
Initially the trainable parameters are 175 billion.
But when I use techniques like adapter right.
So initially it will have 7.50 7.1 million parameters with rank is equal to one.
But as I keep on increasing the rank, as I keep on increasing the rank, you'll be able to see that
this parameter is also getting increased, but you can see from 175 billion parameter.
If I compare 7.1 million weights, the percentage is very less.
Now similarly, in Laura, because of that, uh, matrix decomposition, you'll be able to see that
as I keep on increasing my ranks.
So these are my ranks with respect to q k v because in transfer you have this three parameter q k v.
Then only all the matrix multiplication will happen with respect to this three parameters.
And then we as we keep on increasing the rank here you can see for here you can see eight.
Here you can see 64.
Then you'll be able to see initially we got 4.7 million parameters.
Compare from 175 billion to 4.7 million.
How this was possible because of the because of the matrix decomposition.
Because of the matrix decomposition.
Right.
And as we keep on increasing the rank, you'll be seeing that the parameters are increasing.
Right?
The parameters are obviously increasing.
But if I compare it with 175 billion parameter, this is very less 9.4 million if you just see the percentage.
Right.
So here also when rank is equal to eight 37.7 million, then rank is equal to 60 430 1.9 million.
Right.
Parameters are there.
So still it is making sure that the parameter is not that much like like not not like 175 billion or
not near to this.
If I talk with respect to percentage, it is very, very less.
I've also made another table.
Right.
Just to show you if I have different different models.
Number of trainable parameter.
Here you can see I have one lm model with 7 billion.
If I use rank is equal to one, then I will be having 167 k parameters to fine tune.
Fine tune weights based on fine tune weights then and this one 767 k parameter basically means what
this decomposed matrix that I have right two matrix that many number of parameters.
So in the first case it is just nothing but seven 167 k parameters that is the available in this decomposed
matrix.
Okay.
When we combine them then we will be able to get how many 7 billion parameters.
If we combine both this matrix then we will be getting 7 billion parameters okay then similarly you
can see in 13 billion then you have 2 to 8 k parameters.
In 70 billion you have 5 to 9 k parameters.
In 180 billion you have eight for nine k parameters.
So as you keep on increasing the weight, this parameter will keep on increasing.
But it is not increasing with that huge amount.
Right?
Even you can see when we keep the rank as five to L right.
So here you can see 86 million parameters.
Is there when compared to billions.
Right.
Uh, Microsoft, uh, you know, it came up with this Laura technique, uh, in one of the research
paper and it has used rank is equal to eight, okay.
To probably do the fine tuning.
And it has performed absolutely well.
So most of the time we select this particular value.
But at the end of the day how to select this.
Right.
It won't matter you know because the parameters are increasing by very less number over here as we go
ahead.
So usually you can select rank 1 to 8 while you're performing fine tuning.
Now there may be also scenario that when should we use very high rank?
When to use high rank?
This answer because in the interview they may ask you if the model wants to learn complex things.
Complex things.
Then you can specifically use high rank, right?
Let's say some of the model is not trained to probably, uh, interact or probably, uh, perform some
of the behavior at that point of time.
Those complex things can be handled when you are probably increasing the number of ranks.
Okay.
So this can be a very simple question that may be asked in the interview, but I hope you got a complete
idea.
At the end of the day, this is the equation that you'll be able to see in most of the research paper.
Uh, what Laura is doing is that nothing very simple.
All the track weights is decomposed into two smaller matrixes with different different ranks.
It can be different, different ranks.
When you're fine tuning, the first thing is that you really need to set that rank.
Okay?
If you set that rank, like in this particular case, if I probably see if I go ahead and calculate
with all the mathematical stuff, I will be able to get, the rank is equal to one.
Okay.
Uh, for this also rank is equal to one.
Right.
So similarly if you have rank two.
So one of the matrix can be something like this.
So decomposed metrics.
So this is based on rank two okay.
So if I probably combine this right.
So how many.
123456789 ten 1112.
Right.
If I multiply this I'll be getting a matrix uh, of much more parameters.
Right.
But at the end of the day for in this particular case it is less number of parameters.
Right.
So this is what Laura is all about.
And because of this technique the fine tuning is done less the the weights, the parameters becomes
less.
So this is how the main resource constraint is done.
And uh, with respect to all the downstream tasks, it becomes very much easy.
Now one more thing that I really want to talk about is Clara.
Okay.
Clara.
Clara basically means quantized.
Quantized.
Laura.
Okay, now you have already learned from the first video what is quantized.
Quantized basically means.
Now in our case, what will happen is that all these parameters that is probably stored in float float
16 bit.
Okay.
We will try to convert this into four bit.
That's it.
Okay.
Once we do this, you will be able to see that we reduce the precision.
And then we try to reduce this values.
Also by this you won't require much more memory.
So that is the reason we say quantized Laura technique okay.
And the best thing about is this is that, uh, Clara also has one amazing algorithm which will be able
to take care of both this part, let's say if there is a float 16 bit, I quantize it to four bit.
I can also convert this back into 16 bit.
Okay, so with respect to this explanation guys, I have already spoken about many things over here.
Laura and Clara.