This specific video, we are going to fine tune gamma model in Keras using the Lora technique.
Created a lot of videos.
How does fine tuning happen?
What is Lora, what is Lora, what is quantization and many more things.
So in this particular video we are going to fine tune the gamma model.
And again remember guys gamma if you don't know it is a completely open source model that has been provided
by Google.
And we'll try to see that how we can actually, uh, you know, fine tune our LM models with respect
to this, with, with our own custom data.
So here, um, what?
All things we are going to use.
Everything, uh, step by step will go ahead and, uh, do it.
Okay.
Now, first of all, uh, you need to complete the setup instruction at gamma setup.
So if you probably click over here you can probably watch this entire you can see this entire, uh,
you know, the, the documentation that is specifically given the first thing that you specifically
required is an API key.
So first of all, go to AI studio.google.com google.com and then click on get API key right.
And if you don't know guys you also have now access of Google Gemini 1.5 Pro which you can probably
access it.
Right.
And then over here after uh going to this particular page you need to click on create API key.
Right.
Once you specifically create API key, you will be just allowed to give them a name, select the project,
give the name, and automatically you'll be able to get an API key over here.
Copy that API key.
My API key is already I've created it, so I'll be specifically using that.
Right.
So you have to just go to AI studio.google.com okay.
So this is the website that you will specifically go to.
Okay.
Then you need to go to kaggle.com and specifically get the access right.
So you need to get the access of this um, you know the gamma setup itself, right.
Uh, so if you go ahead and write Kaggle gamma.
Right.
Uh, axis.
Right.
If you just go ahead and write it over here, here you can see that gamma will be there.
So you have to go to this particular page, log in into it.
And here you will be seeing you have consented the license to uh, license for gamma.
First of all, if you have not consented it here, they'll be asking you an option of request access.
So once you need to click the request the access.
And once you go ahead and select all the terms and condition, you will be able to get the license agreement
right.
So then you'll be able to get the license consent and then you'll be able to use it okay.
So that is the first thing.
And right now all these models are available over here with both uh and it can run on Jax TensorFlow
and PyTorch okay.
So this is the next step that you really need to do.
Right.
And then go ahead and configure it.
I'm already using a paid Google Colab Pro account, so for me, I definitely require a lot of Ram because
I really need to show you a good fine tuning technique.
Okay, so quickly let's go ahead and configure it.
As I said once, you probably go ahead and create a API key.
Go to this particular secret key.
And then you need to right over here as Kaggle key along with the API key.
Otherwise you can also go ahead and write Google API key.
Right.
So I will just enable it so that I'll be able to use it over here okay.
So this is the Google API key that I require.
And if you really want to access it from the Kaggle itself you need to probably select this tool.
Okay.
So once it is done and uh if you want hugging face token, also if you want to use it you have to create
it.
Right.
So all these things are selected.
Now what I'm actually going to do is that import OS and how to okay.
One more thing that you specifically required is Kaggle key and username.
How do you create it.
So you have to probably go to this particular settings button, right?
If you go ahead and click on settings right.
Or once you go ahead and click on settings, I think uh, somewhere the API will be there.
So you can go ahead and create a new token, right?
So once you create a new token you will be getting two important keys.
One is Kaggle key and one is Kaggle username.
So you have to set it up.
So once you probably click on your token, generating a node token will automatically expire the previous
one.
So I'll not do that because I've already done it.
So once you do that, uh, JSON file will get downloaded and there you'll be getting two keys.
One key is the Kaggle key, and one key is the Kaggle username.
So you have to make sure that you have to set all this up over here okay.
Before you start this particular project then we will go ahead and select all the uh we'll go ahead
and set up all the environment over here.
One is the Kaggle username and one is the Kaggle key.
So I will go ahead and execute this.
Once this is basically getting executed, the next step is that we will go ahead and install Keras NLP.
And I think this is the first kind of videos where I have specifically uploaded how you can fine tune
the models in Keras using Laura.
Okay, so Keras is also having this specific feature.
So first of all, I will go ahead and install Keras, NLP and Keras greater than or equal to three.
So once the installation will specifically take place because we are going to use Keras to do the entire
fine tuning.
Okay.
So this is going to take some amount of time.
Uh, then we can go ahead and select the back end.
You can use Jax Torch TensorFlow.
So it already provides all the specific features right.
So in the environment I will be selecting Keras backend as Jax.
Right.
Uh, Jax.
The best thing is that our to to uh again it's just like TensorFlow and torch.
But again it is an completely open source.
You can also use this specific thing okay.
So till the installation is taking place uh then we are going to uh avoid memory fragmentation on the
Jax back end.
So we are going to set this XLA Python client memory fraction.
Okay.
So here we will be setting it to 1.0.
So this is the initial environment that we really need to select okay.
So you are now connected to GPU runtime but not utilizing the GPU.
Don't worry, we will be utilizing because I need to probably do the entire fine tuning with the help
of GPU.
Okay, so initially Kaggle username Kaggle key, then install all the Keras libraries that is specifically
required.
And then you select the back end that is called as Jax.
Okay.
So here we are getting some error.
I think uh, it is a conflict but no worries.
I think it is working fine.
Then we'll go and execute this where we are selecting the back end and this okay.
Now let's go ahead and import Keras and NLP okay.
Keras NLP okay.
Here uh to fine tuning the gamma model.
Right.
Uh, what we need to do is that we need to set up our data set in the form of JSON all file.
Okay.
Where you will be able to see this, I will just go ahead and click this particular link.
And here you will be able to see this okay.
How the data will specifically look like.
So this data is specifically present in hugging face if you go ahead and open it okay.
So I think uh I will go ahead and open it.
Okay.
So here, uh, let's see whether I'll be able to read the JSON, all JSON all format okay.
So JSON format I will just try to load it over here.
Okay.
JSON lines, JSON all.
I think here you'll be able to find out how you can load it.
Okay.
Uh, whether it is asking me to drop it over here.
Let's see.
I can drop it or what?
Okay, so this is how the file looks like.
Here you can see there is a JSON file has two important things.
One is instruction.
See I will just zoom out a little bit.
So here you have something like instruction.
Then you have something like context.
So in this you specifically have one is instruction.
One is context.
Instruction is what is the question.
And context is basically what is the answer.
So if you probably see all the, all the all the all the inputs and outputs are specifically in this
particular structure.
Right.
Because I'm going to use this structure itself.
And for gamma also for even for OpenAI you definitely require in the form of, uh, just JSON all file.
Right?
JSON L right.
So where you have two important information instruction and then you have context.
So based on this you can also create your own file since uh, since I'm showing you a fine tuning technique.
So I will be downloading this uh Dall-E 15 k dot JSON file.
So here you have 15,000 records with respect to this.
As soon as this file will get downloaded, you will be able to see over here.
Okay.
So here you can see Dolly.
Uh, Databricks Dolly 15kg Journal.
Again, it is an open source data set just to show it to you.
You can definitely use it.
Now the next thing over here, you'll be able to see that the code that we are specifically writing
import JSON data.
And then we are opening the JSON file.
Then we are loading all the JSON file into JSON itself.
Then we are reading the context if the feature context does not exist, we will continue.
Otherwise we will create this kind of template right where my instruction will have the instruction
data and response will have the response data okay.
And then we are going to append it inside this particular list of data okay.
Once we do this this is what my first top thousand data looks like.
And here you can see that it is read it.
And here you will be able to see this is my entire data.
uh, uh, with respect to the top 1000.
Okay.
So we are going to just use the top 1000 data.
And uh, the format that we specifically want is basically written over here in this format.
That is instruction with instruction, response with response.
Okay.
Whatever content is present in that JSON file, then, uh, it's time that we will be loading the model
of gamma.
So here we can write Keras underscore NLP dot models dot gamma casual LM from preset.
And there are two types of model.
One is gamma 2 billion parameters and one is gamma 7 billion parameters.
How I'm saying it.
So if you go over here if you go down okay.
So here we have the excess of gamma 2 billion.
Yes 2 billion parameters 7 billion.
Also if you go and search for hugging face this one you'll be able to see.
And this is what is the performance matrix looks like okay.
So here uh you can go ahead and execute this.
And this will load the model from the Kaggle itself.
Uh and then it will be loading into our colab notebook.
All the models are basically getting loaded.
All the weights are specifically getting loaded.
And you can create this model.
I will also show you till the inferencing part.
Each and every thing will be shown over here.
And once it specifically gets loaded, you will be also able to see the entire, uh, how that entire
model is basically created, how many layers it has, how many parameters it has.
And obviously we can see 2 billion parameters.
But just by downloading it, once we download this entire thing in our colab notebook, here you can
see we have this tokenizer called as gamma tokenizer.
Along with this padding mask layer and token id, everything is given over here.
And here.
The total number of parameters are somewhere around 2.5 billion and it is 9.34 GB.
One thing you have to take care guys, if you really want to run this, you really need to have a paid
Google Colab Pro account.
Okay, then, uh, let's go and see this.
And this is just the model.
I've still not fine tuned it.
So without fine tuning we will run this.
Okay.
So we will create template dot format.
And the instruction will be what should I do on a trip to Europe?
Okay.
I'm just asking a generic question to a gamma model and response is completely empty.
Then we take the same Keras underscore NLP.
So here you can see Keras underscore NLP dot sampler top k sampler k is equal to five.
So I'm saying that try to provide me five results out of there.
And whatever gamma underscore lm I have actually done.
We are going to compile a compile with this particular sampler okay.
So once we sample compile it then we can use this gamma underscore LM to generate the prompt.
And it is probably going to give me some five responses.
So if you probably go ahead and see this.
So if you're following my playlist on all the fine tuning playlist, you will definitely be able to
understand how things are going over here.
Right?
So once I execute it, I will be able to see the prompt definitely over here.
But just understand what are the steps?
Initially we create the prompt, then we create a sampler okay.
This sampler will basically say that how much top five results we want.
And then we are going to compile with this sampler.
And then we are going to generate okay.
So here you can see the response is easy.
You should just need to follow the steps.
So and so what are the benefits of travel agency.
What how do I choose.
So five different records will be probably over here over here along with the response okay.
But still we have not fine tuned it with our data set.
So one more example is over here.
Explain the process of photosynthesis in a way that child could understand.
And here again we are using gamma underscore lm dot generate.
And we have already created the sampler.
So here we will be able to see the entire response with respect to the question that we have given and
understand.
Multiple response will be able to get it.
So explain the process of photosynthesis in a way that a child could understand.
So here you can see all the responses are there.
Chlorophyll is a pigment.
Explain how plant absorbs plant capture sunlight energy through the leaves and use it.
Okay, so all these things are definitely there.
But the main thing is with respect to the fine tuning.
Now, lower fine tuning.
I hope you know about the mathematical intuition.
If you have not known, you have very late because I have already uploaded a video in my playlist,
right?
How does Laura work?
Laura works and all that is the prerequisite that if definitely you need to know.
Okay, so this tutorial uses a lot of rank for uh, what is rank?
What is the importance of rank?
Everything I've actually included.
Okay, so here we are going to enable Laura with rank is equal to four.
And now if you go ahead and see the summary okay.
So enable Laura for the model and set the rank to four.
So here you can see the parameters.
Trainable parameters becomes less when compared to the all the parameters over here.
So hardly 1 million parameters are there from billion to million.
Right.
So that many number of trainable parameters only 5.20 MB.
0MB and then note that enabling Laura reduces the number of training parameters significantly from two
point billion to 1.3 million.
Okay.
Then we are going to set the input sequence length to five to L.
Okay.
Again you can change it to 1024.
Also we are going to select the optimizer called as Adam Adam w okay.
In Keras it is already there.
So keras optimizer dot Adam w learning rate is so 0.0005.
Weight decay is 0.01.
Okay.
Uh, this is how we basically set optimizers in Keras.
And then we are also going to exclude from uh, weight uh weight decay, exclude layer norm and bias
terms for decay.
So here we are going to set this up.
And then finally we are going to compile with this specific loss that is sparse categorical cross entropy.
Again since it is a multi-class classification I'm basically using form logits is equal to true.
Then you have optimizers.
then you have weighted parametrics again over the sparse categorical accuracy is given.
And then we do the fit of the entire data with epoch is equal to one and batch size is equal to one.
So this is probably going to take if you're doing it in the paid colab, it is going to take somewhere
around 10 to 15 seconds.
Uh, you can also do along with my execution okay.
It is probably going to start.
And again it is going to take around 10 to 15 minutes.
So we will wait till this entire processing will start.
But we'll wait at least till the first epochs should get started, you know.
And it is going to based on the 1000 data points, I think it is going to take 1000 epochs.
Okay.
Because batch size is only one, because we are going to send the sentence for every, every sentence.
We are going to do the front, forward and backward propagation with the help of Adam w optimizers.
So yes.
Uh, let's wait.
Uh, and I think it should start now.
It has started.
It is hardly going to take somewhere around, uh, nine, nine minutes, 17 seconds.
So we'll wait till this particular thing is getting executed.
And then once it probably takes.
Okay, it shows one hour.
Okay.
But I think it will hardly take 15 to 20 minutes.
Okay.
15 to 20 minutes.
So here you will be able to see as as you keep on going.
The loss is also getting decreased.
The sparse categorical accuracy will also keep on increasing okay.
so we'll wait till this particular happens again.
You can increase the number of epochs to get a more accurate model.
Okay, so let's wait till this particular entire training happens.
And then we are going to see the inferencing part.
Thank you.
So guys finally the fine tuning is done.
And here uh hardly.
It took around ten minutes 10 to 11 minutes okay.
So here you can probably see all the fine tuning accuracy if you increase the number of epochs.
So definitely this accuracy will keep on increasing.
But let's check whether it is working perfectly fine.
We'll also try to understand how to specifically do the inferencing.
So here uh you will be able to see uh, now I'm giving the same question.
What should I do on a trip to Europe?
Now it will be able to give the response based on the data set.
Okay.
So here you can see the previous response was something like this.
Uh, yeah.
It's easy just you just need to follow the steps first.
You must book your trip with the travel agency and all.
But now you think, like it'll be a different response altogether based on the data set we have again
over here.
Same thing sampler gamma underscore lm dot compile.
And then we are going to generate the same thing right.
So now let's go ahead and see the response.
Uh after the fine tuning how the response looks like okay.
So yes I think we should be able to get the response now in just some seconds.
Uh, and similarly uh, the same other example also will try to see explain the process of photosynthesis
in a way a child could understand.
So here you can see now the response is completely different.
The first thing is to get a passport and vision.
Then plan what to do if you're traveling to Europe I have recommended starting out in Paris.
Uh, France.
Paris, France.
France is.
Paris is a great city to start out.
Uh, at, because it's the largest city in France and has tons of things to do.
And all everything is.
You'll be able to see over here right now.
Similarly, with respect to the photosynthesis, uh, so many different kind of answers you saw over
there, but now you'll be able to see that how quickly you are able to get a quick response, and you'll
be able to get a better response, you know, after the fine tuning, the same thing you really need
to do.
Anyhow, I will be giving you the entire materials.
Just go ahead and execute it.
Just the prerequisite is that you really need to understand about the fine tuning techniques.
I will be putting the fine tuning playlist in the description of this particular video.
So here you can see explain the process of photosynthesis enough in a way that child could understand.
Photosynthesis is the process by which plants and some other photos and synthetic organisms uses light
from the sun as a source of energy.
So and so, so and so.
All the information is given, right?
So you can also increase the size of the fine tuning data set.
Train for more steps, setting up a higher lower ranks to increase the probably the uh performance of
this models modify the hyperparameters such as learning rate and weight decay, but I hope you have
understood how you can probably fine tuning fine tuning a gamma model using.
And we have what we have done using, uh, Keras.
And again the technique was used.
Laura.
So I hope you like this particular video.
I'll see you all in the next video.
Have a great day and thank you all.
Take care.
Bye bye.