So guys, we are going to continue our generative AI series on AWS cloud.
And in this video I'm going to probably show you how you can actually deploy your hugging face model.
Let it be an open source model or any LM model.
Also, specifically in AWS SageMaker that is Amazon SageMaker.
Um, Amazon SageMaker is one of the important services in AWS where you can probably complete the entire
life cycle of a data science project or an AI project completely along with MLOps till the deployment
each and every thing.
So in this particular video, I will be showing you how we can take up any hugging face model in short
and probably deploy it in the SageMaker itself.
So please make sure that you watch this video till the end.
And one more very important thing guys.
Uh uh, just check out this particular video.
They will they may some they may be some amount of charges that may incur.
So please be careful with respect to respect to this whenever you start creating.
And finally when you create that particular endpoint, make sure that you delete all the endpoints itself.
And I will be providing you the entire guidance as we go ahead.
So first of all, we are just going to go ahead and search for AWS SageMaker.
And to start with, uh, how to probably work with AWS SageMaker studio and all.
I will also be talking about that.
So first of all, you can see, uh, you can just go ahead and click on Getting Started.
Okay.
Uh, and with respect to this AWS SageMaker, the documentation is pretty much amazing, right?
So if you don't know about AWS or Amazon SageMaker, it provides machine learning capabilities that
are purpose built for data scientists and developers to prepare, build, train and deploy high quality
ML models efficiently.
So, uh, what we are going to basically do is that step by step, I'm going to actually show you.
So I'll go to the domain part and I will go ahead and create the domain.
So once I probably go ahead and create the domain.
So here uh there are two options.
One is set up for organization and the other one is set up for single user.
Okay.
So as soon as uh, and right now we are just going to test it for single user.
So what I'm going to do is that I'm going to select this here.
You will be able to get an IAM role automatically.
It will be creating it with this Amazon SageMaker full Access policy.
Uh, public internet access standard encryption.
SageMaker studio.
You'll get the access to this.
You'll be getting shareable SageMaker studio notebooks, SageMaker canvas and IAM authentication.
So each and every thing uh, by inbuilt, it will be providing you this entire environment where you
can actually do the coding.
So just go ahead and click this.
So once you probably go ahead and click this you will be able to see uh something like uh okay, the
setup is going to take place and it is going to take some amount of time.
You know, uh, again, it depends on how much time it is probably going to take over here.
It will automatically create the users and everything will be able to get it right.
So here, if I probably go ahead and just reload this, you will be able to see that my third domain
is probably over here getting set up pending is this.
And it will probably take some amount of time based on the usage.
But other than that that you can probably see that I've already created two.
So uh, since this is already getting created, I will just go ahead and show you some more examples
with respect to this.
So let me just go ahead and quickly click on this quick set up domain.
You'll be able to see that by default it will be creating a user okay.
You can add any number of users based on the organization that you're specifically working in.
Let's say you want to provide access to multiple users in the same, uh, domain itself.
You can also provide it.
Okay.
So by default, uh, as as soon as you provide the user access.
Also, there is one very important thing that, uh, you will be able to see.
Right?
And, uh, that is nothing but the user information over here along with this.
Let me just hide my face here.
You'll be able to see that you'll be getting a launch button.
Okay, so through this, you'll be able to access canvas.
I will be talking about canvas as we go ahead in the further videos.
TensorBoard.
I hope everybody knows about TensorBoard Profiler, but we are just interested in working on this particular
studio, right?
So once I probably click on the studio and uh, you will be able to see that I will be able to get this
entire Amazon SageMaker.
It will be a studio completely, which will actually provide you the entire ecosystem with respect to
any kind of development that you really want to do.
Okay, so here you can probably see Jupyter Lab is there.
If you want to quickly start deploying, fine tune and evaluate pre-trained models specifically for
LM.
You can also go ahead with this.
You know, if you want to probably, uh, go ahead and do some kind of AutoML.
You can also see over here there is an option model evaluation.
So you will find this entire ecosystem okay.
Now what I'm actually going to do is that I'm just going to click on this Jupyter lab.
And right now you will be able to see that nothing is running over here okay.
Nothing is running specifically.
And uh, what I'm actually going to do, I'm going to create a create a Jupyter lab.
Okay.
Over here, Jupyter lab space so that I will be able to work it.
Okay.
So here I'm just going to write test demo SageMaker okay.
And this I'm going to show you with hugging space okay.
Uh sorry.
Hugging face models.
Right.
So here uh, it will first of all ask you to probably select the instance, right.
Uh, multiple instances.
You can probably select uh for my use cases again.
Uh, this instances if I go ahead and check over here SageMaker instances based on different different
instances there is a different, different price.
So you really need to be, uh, taking care of this because see, if you're working in a company based
on the requirement, you can probably select the instances.
So here you can probably see on demand pricing.
So uh, let's say if I select Mlt3 three medium $0.05 here it provides you CPU with two core four GB
memory.
But understand if you are specifically working with generative AI models, right.
You definitely require a huge amount of data, right?
So a huge amount of space.
So if you probably go further, there will be good, good amount.
Amazing system accelerating computing which also provides you not only CPU cores but also GPUs.
Right.
So as you keep on using more and more, you'll be able to see the charges will be going higher and higher.
But just, uh, in this specific video, I really want to show you some demo.
Uh, but again, if you just get this idea in any companies, if you go, you will be able to work it
out.
Right.
So I'm going to take a small system and probably work with this right now, which small system I'm going
to specifically take.
So if I go up here is something called as ML dot m5 two large.
So if you go ahead and see over here uh ML offer two large.
So this is basically providing you eight core GPUs and 32 GB memory.
The charges will be 0.461, uh, price per hour for using this particular instance.
The reason why I'm doing this is that because I will just give you an example of one of the model,
how it is deployed and all.
Okay.
And uh, we'll see to that.
Okay.
So here I'm just going to keep the storage to ten GB.
Uh, and now I will go ahead and click on Run Space.
Okay.
So once the run space is clicked on you will be able to see that my entire environment will be ready
and I should be able to work it out.
Okay.
And with respect to this, uh, any kind of code that you run right in the my recent video, you have
also seen that I have actually, uh, created entire one Lambda function with the API gateway.
I've shown you how to probably create the endpoint and all.
So all those things you can probably do with this also.
But here more amazing endpoints.
You'll be, uh, more amazing ecosystem and features you will be able to get.
Okay, so this is going to take some amount of time.
So okay now it is created.
Now I can just go ahead and click on Open Jupyter Lab.
Now understand with respect to hugging face there are multiple ways to load a specific model okay.
There are multiple ways.
Uh, all those ways I'm going to talk about and this will be important for you because you really need
to know have an idea like how we can actually work in a system, right?
So once this is loaded, uh, the first step.
Okay, what you can do, I will just go ahead and change my theme, so I'll make it Jupiter dock.
Okay.
So here you have options.
Uh, you have options for notebook.
You have options for console, terminal, Python, anything you can specifically work with I will just
go ahead and open terminal.
Uh, sorry.
Notebook.
And the first thing that I will do is that just go ahead and write pip install SageMaker.
Okay.
Which is the upgraded version.
Just go ahead and install that automatically.
The installation will be taking place and all the recent updates that is basically there with respect
to the SageMaker that will get updated and understand this is my entire environment, the same Jupyter
notebook that we usually specifically work on.
And all right.
So here you'll be able to see this now, uh, to go ahead with what we really need to do initially.
So I will go ahead and import SageMaker.
Okay.
And there are some steps that you definitely need to follow as we start.
Right.
So I'm going to import Boto3.
And I'll be providing you all the code in the description of this particular video.
So first of all we'll go ahead and create our session.
The session will be nothing but SageMaker dot session okay SageMaker dot session.
So by default whatever is the session we will be able to get it.
Uh, initially, uh, we have not set up any, uh, session buckets or anything as such.
You can also create that particular session.
So I'm just going to write over here even with some comments so that it will be beneficial for you.
So here you can see session uh SageMaker session bucket used for uploading data model logs.
If you want to probably upload data, upload a custom data or download the Huggingface model in this
particular S3 bucket and probably retrieve from that, or use from that.
You can actually do that.
SageMaker will automatically create this bucket if it does not exist.
Okay, so you can probably go ahead and write this particular condition.
And here you'll be able to see that if there is nothing as such, um, if it is none, I'm just going
to probably, uh, make if the session is not there, I'm going to probably go ahead and create the
default bucket.
Okay.
So this is the code that we do initially.
Uh, and this bucket will specifically be used to upload any custom data or training data.
It can be models after the inferencing and multiple things.
Now other than this, we, uh, really need to focus on the role management over here because since
we are using AWS SageMaker to execute any kind of code, uh, specifically by using the services of
AWS SageMaker, we really need to provide the role access and all.
So for this, uh, we can just go ahead and write some try catch block.
So here I'm just going to go ahead and write try um try and let me just go ahead and do pass over here
and let me go ahead and write accept block.
Um with respect to this particular accept block, I will just go ahead and raise some value error.
And the best thing over here is that you'll get all the suggestion right.
And that is pretty much amazing, right?
Once you are specifically using it let's say my IAM user is not yet set.
So it will obviously give an exception.
So it will go to this particular value error.
And here I will first of all go ahead and set up my uh user.
So I'm IAM user in short.
So I'll go ahead and write Boto3 dot client.
And uh I'm also checking out the documentation page.
It is always good to probably check out the documentation page and perform in that particular way.
And then I'm going to basically set up my role.
Um, I'm going to use IAM dot get underscore role.
Now it is going to probably pick up those role that is configured with this particular Jupyter notebook.
Right.
So I'm just going to write role name okay is equal to SageMaker SageMaker underscore execution underscore
role okay.
And this is basically going to be set up with our roll.
And from that we are going to probably take a number.
Right.
That is how we specifically identify the unique roll that we are probably trying to give.
Okay.
So all this thing is done over here.
Let me just make some spaces that is actually required whenever we work with this okay.
So here you can probably see my roll is also set up.
Uh this is my exception.
Now let me quickly go ahead and write in a try once the, uh, x, uh, once the role is set up.
So I can basically go ahead and write SageMaker dot get execution role.
Okay.
This will be responsible in getting the execution role.
If by default nothing is set up, then you can probably call it out.
Uh, based on the role that it has.
It has.
Okay.
Again, one common thing that I've seen that we have to provide spaces over here.
So that is one of the thing over here.
So please make sure that you keep on working with respect to that whenever you're writing any code.
Uh, after that I will just go ahead and write.
Session is equal to SageMaker dot session.
I'm just going to call my session.
And here I'm just going to use my default bucket.
Uh, so here one of the parameter that you can probably see is nothing but your default bucket, which
will be assigned to my default bucket or sorry, not default bucket SageMaker session bucket.
That is my default parameter that is specifically required.
Okay, so this is the first step that you really need to do it okay.
And then finally I can go ahead and print my you can see how it is going to pick up the r n roll okay
SageMaker role Arn.
And here I'm going to basically set it up to roll okay.
So this is the roll that is going to get print up.
And next print is equal to.
And here you can probably also go ahead and write your SageMaker session.
So SageMaker session region if I want to display it okay it will be nothing but session dot dot boto
region name.
Right.
So this is also there.
So once I execute it, uh, let's see whether it will be working.
Fine.
So here you can see uh not applying SDK defaults.
This is they're not applying info okay.
Some error I can probably see over.
Okay.
Let's go ahead and do that again okay.
Now it's working fine.
You can probably see SageMaker role is basically this.
And this is the default role that uh the domain has basically created with respect to the user.
And then us East one I'm actually working on.
Okay.
So this is the first step.
Now if I have the roles and all, uh, now I can probably go ahead and call any kind of models that
I specifically want.
So uh, first, uh, we will try to call one type of model.
And, uh, I will just go ahead and show you the code.
Okay.
So here I'm calling one model which is called as Distilbert uncased distilled squared, and this is
specifically used for question answering.
So I am basically going to give the hub configuration.
See Huggingface hub.
If you don't know about Huggingface hub then uh, there you definitely have a lot of models which can
probably use for multiple use cases.
So here you can see Huggingface uh, model.
I'd have to probably give whatever is the my model ID, the kind of task that I'm looking for, I'll
be giving in the form of, uh, key value pairs in this particular variable called as hub.
Okay.
Then we go ahead and create the hugging face model.
Right.
And for that we have already deployed or imported from SageMaker.
Dot hugging face model import hugging face.
Right.
And then finally you'll be able to see that I'm using this environment hub okay role I've already used
transformer version which has provided PyTorch version and Pi version 3.9.
Okay, so this is how the parameters you really need to give with respect to this.
Okay.
So, uh, I've already executed this, uh, and I don't want to again deploy this entire model in my
another Jupyter notebook because it is going to definitely take time.
So just to give you an idea, I have already done that.
So let me just show you over here.
This is my another instance which I was actually working on.
So here, uh, is my second Huggingface model, which I have actually given.
Third, uh, how to deploy it to the SageMaker inferences.
As you know, I have selected ML dot m5 x large.
Okay, so I am here.
I'm writing predictor by using the same hugging face model dot deploy.
I'll be saying the instance count as one okay.
Because I just required one instance.
And what is the instance type.
Instance type is ml dot m5 dot xlarge okay.
Where I'm specifically deploying.
Other than that, uh, I need to make sure that how should I give my input?
So I will be creating in this particular format, because this model that is distilled, Bert based
Uncased distilled squared will be requiring the model, uh, the input data in this particular format.
That is question what is the use for inferences then?
Context.
My name is so and so I've just given some examples so that my more charges should not incur.
And then we'll go ahead and predict the data.
So here you'll be able to see if I go ahead and write predictor dot predict okay with respect to this
particular data because this data is required in that particular use case.
So here you can see score is this one start is this one end is this one answer is SageMaker right.
So what is used for inferences.
So here you can see from this particular question uh you have that SageMaker.
It is going to pick up.
Understand when I execute this code it is going to take 5 to 10 minutes to deploy into this.
And you will be able to get your endpoint Okay.
Just to give you an idea, how does an endpoint look like?
Um, I'll just go over here just a second.
Uh, how does the endpoint look like if I, if you really want to see.
So here I, if I go back to my SageMaker studio I've created multiple endpoints.
So if you go to deployments you'll be able to see endpoints over here.
Right.
These two endpoints are there.
Let's say I want to go ahead and see this particular endpoint okay I can click on this particular endpoint.
And once we deploy it that endpoint will be created.
If I want to test the inferences you'll be able to see that, uh, I'll be getting this application
JSON and whatever body I'm specifically giving like how I'm giving over here.
This is my entire body, right.
And with respect to that particular body, if I give over here also and just test it, I should be able
to get if I just click on send request, I should be able to get my output right.
And there are also multiple options like auto scaling.
You can scale to whatever things you specifically want based on the charges.
Again, uh, but here you will be able to see that I am I have deployed this model in this particular
instance.
And, uh, you'll be able to just use that particular endpoint wherever you want in your code, anywhere
as such, based on your requirement.
Right.
So this was the entire thing, right.
Let's let's try some more thing.
Okay.
I will again create some more data over here.
And, uh, um, my name is Chris, and.
I teach data science.
I'll just give this a and I teach data science.
Okay.
So.
What does Chris like?
Okay, I'm just going to execute this.
Let's see what I'm going to get.
The answer I'm predictor dot predict.
So you have a predict method here.
You can see the answer is data science right.
What does Chris like uh what does Chris teach.
So you can basically do okay.
You'll be able to get the answer data science okay.
So in short this is just a simple model, right?
I've probably deployed this particular model.
You'll be able to see that this is my entire SageMaker.
And here uh, you know it took some amount of time I think this dot dots are like every one minute,
around 6 to 7 minutes to deploy this entire model in this particular instance, which I've actually
created.
Now, just to give you an idea like how does, uh, machine learning, uh, LM model be like.
So I'll be giving you multiple examples.
So first of all, you have to probably go and update SageMaker the same thing by keeping your role and
uh, region name.
Then you are basically going to call the SageMaker image URI.
Right?
Since we are going to run it as a container.
So here you can see hugging face deep learning container in AWS SageMaker.
So the entire LM model will basically be put in a container.
Then you can actually call that particular container over here.
So in order to call the container you will be using from SageMaker or hugging face.
Get hugging face LM image URI.
And then here is your version that you are specifically using and what kind of image you are actually
coming up with.
Okay, just to show you, if I probably give you an example, I will go ahead and install all this.
Let's see.
You'll be able to see the URI okay.
And then it will probably download it and upload it.
So there will be a lot of task in that.
And a lot of money will be required because again, at the end of the day you are see over here, you
are probably got this.
Now let me just execute this again.
So here is your SageMaker role.
And then you can you can see that which you are I am actually calling right.
So if I probably go and see this is the URI that is basically required right.
Image URL.
So if you know about Docker and containers.
So once you get the URI then you should be able to load the hugging face model.
Now here also you can probably see here we are calling Falcon B for t b instruct number of GPUs how
much you want.
And just imagine that instance type that we are going to use.
Is MLG 512 into large.
I'll just show you the cost and you will be pretty much shocked to see the cost.
So here you can see the internal storage GB is one into 308,800.
Uh total GPU memory is 96.
You know, GPU 24 core bandwidth.
Internet bandwidth is 40.
And the GPU model is Nvidia A10H, right?
Memory is nothing, but 192 CPU cores is nothing but 48.
Just imagine how much the charges will be.
So that is the reason why I have actually taken a small model and probably shown it to you later on.
You can probably do with respect to anything once you probably go through the company, right?
So that is the reason we have selected all these parameters over here.
Then we initialize the hugging face model.
Then we deploy it.
That's it.
Right.
And then after that you can probably use the same payload.
You can structure the payload over here uh with the prompt.
And then you will be able to get the text okay.
So anyhow I will be giving you this entire examples in the description of this particular video.
But again, yes, I've told that I've been uploading videos on AWS SageMaker and all, but definitely
there requires a lot of charges that is involved, but it is always good to have a knowledge about it.
That is the reason that is the main aim for this particular video.
So I hope you like this particular video.
This was it for my side.
Uh, again, uh, you can refer to multiple examples.
Uh, here I will also give you this GitHub link from the documentation of, uh, hugging Face.
So there are multiple labs which has been created, which you can also go ahead and refer to.
Okay.
So this uh, some from our set copy this.
So this is an amazing lab details that is probably provided I will be providing you this entire code
itself.
Other than that, you can also easily see the documentation of hugging face SageMaker.
So yes, this was it for my side.
I'll see you in the next video.
Have a great day ahead.
Thank you all.
Take care.
Bye bye.