Hello guys!
In this video, I am going to show you some of the amazing, powerful features of, uh, Nvidia Nim.
Uh, and again, which was recently announced by Nvidia.
Uh, and if I talk about Nvidia Nim, it is a latest breakthrough in the generative AI development.
Okay.
Nvidia Nim is a set of inference microservices for deploying AI models.
And it revolutionizes the way how we deploy generative AI enterprises.
Along with this, Nvidia Nim offers multiple AI models, right?
It can be a model, a model, multi model.
Not only that, it also provides you an Nvidia AI foundation model.
Just with the help of APIs, you'll be able to integrate in your application.
You'll be able to seamlessly run it and it is quite highly scalable.
So in this video I'm going to talk about this.
I'm also going to show you multiple examples with the help of coding.
So please make sure that you watch this video till the end because this is an amazing feature.
And as I always say, there will be many, many LM models that will be coming up.
But the clear winner will be the company that provides the best inferencing thing for us.
Right.
So let us go ahead.
And this is the page over here.
You can see instantly run and deploy generative AI.
Explore the latest community built AI models or AI with API optimized and accelerated by Nvidia, and
then deploy anywhere with Nvidia name.
Uh, you'll also be able to experience leading open source models and I will be showing more about this.
You know, uh, you can also do integration, uh, just by using an API call away.
Uh, along with this, you will be able to run anyway, accelerate your AI deployment with Nvidia name,
uh, how to buy.
So here it is.
And the best thing is that just to for you to try it out, uh, once you probably create a page you'll
be getting 1000 credits.
So which will be more than sufficient to explore and probably call multiple models to you.
Right?
So let us go ahead and let us see.
So first of all go to this particular page.
Anyhow, I will be giving you this link in the description of this particular video.
Just click on try it now.
And once you probably go ahead and click on try It now.
Here you will be able to see all the models right.
So with respect to models you can see llama 370 B.
It has almost all the open source model foundation models along with that Nvidia Foundation models.
Also it has right even open source model.
Also it has.
So here you'll be able to see gamma.
You'll be able to see 85 images.
So multi-model model LM model.
So here with respect to models you can see reasoning is there for reasoning.
You can use all these models.
Visual design.
Uh all these models are actually there retrieval right.
If you want to probably implement retrieval it is also there.
Then here you have speech biology gaming.
Multiple things are specifically over here.
So let me show you one example over here.
And then we'll also try to create an end to end application that will be a Rag application using this
Nvidia name.
Uh so let's go ahead and let's start our project.
But before I go ahead, uh, let me just go ahead and show you one of the things like, uh, so before
I start, uh, any project.
Right, it is very much important that you also need to have an API key.
Now, how to probably get an API key.
Let's say that in my project, in my Rag application, I want to probably use llama 370 billion instruct,
right?
And right now, since the inferencing is happening in the Nvidia name itself.
So if you go ahead and click this particular model here, you'll be able to see something like this.
Right here you will be able to chat.
So let's say if I say hi and if I send a message you'll be able to see that I'll be able to get a response.
Uh, how are you?
Okay.
Any question that you want over here, how are you?
So here you can see this is the code.
Right.
And when I'm writing how are you.
So this is the code that is basically getting set with respect to the content.
So you can use this particular code.
And you can also call it right.
And I'll also be showing you.
So once I probably send how are you.
You'll be able to get the response over here.
Now when you go to this uh bill.nvidia.com.
First of all you need to log in.
Right.
So here you can see I have logged in over here.
If you have not uh if you don't have an account I would suggest please go ahead and create an account.
And here you can see I have 954 credits left.
Initially.
You will be getting 1000 when you are probably creating a new account.
Okay, now uh, this is the thing here you can actually see, right?
Uh, the entire, uh, code is also visible, right?
So I will show you how you can run this particular code.
And again over here you can see that there is an API key that is required in order to generate the API
key.
All you have to do is that click on get API key over here right.
So here you'll be able to see uh get API key in this green color.
And just click on this.
This key authenticates your Nvidia AI foundation endpoint for test and evaluation.
And just go ahead and generate the key.
Now I will be making sure that I copy this particular key, because I'm going to use this particular
key for my coding purpose okay.
Now let's go ahead.
And first of all let's go ahead and open my VS code.
And I will show you how we can go ahead and create this entire project, that code that you'll be able
to see over there.
Write this code also will try to run it okay.
So let's go ahead step by step.
Let's go ahead and do this.
So first of all uh let me go ahead and create my environment.
So here I'm going to use conda.
So conda create minus p v and v python double equal to 3.10 I'll just use 3.10 and let us go ahead and
create this environment.
Okay.
Now over here you definitely require multiple, uh, multiple, uh, requirements, um, in multiple
packages in the requirement dot txt.
So we will also go ahead and update that okay.
So first of all, uh, let me just go ahead and yes, one more thing that I'm going to show you, the
Rag application that we are going to create will do it with the help of Lang Lang.
And also so Lang Syne also has an integration for that.
So first of all, let me just quickly go ahead and write requirements dot txt.
Okay.
So let me go ahead and create a file requirements dot txt.
So first of all what all packages I specifically require I'll be using OpenAI because in this particular
code that I see right there was uh OpenAI over here.
So let me just first of all copy this.
Okay.
I will be requiring this key.
So I've copied it in my node back notebook.
So this is my entire code over here.
So let me just copy this entire code okay.
Copy the code and paste it over here.
So let me just go ahead and create my app.py file and let me paste it over here okay.
And we'll go step by step.
We'll understand what exactly this is okay.
So first of all in the requirement dot txt I am going to use OpenAI along with this.
Uh you'll also be seeing that uh I will be using Python dot env.
The reason why I'm importing this because in the in my env file I will be creating that API key.
So I will be requiring that okay.
Uh this is the next thing.
And along with this uh, right now I'll just keep this two packages over here just to run that code.
Now, uh, let me go ahead and write pip install minus our requirement dot txt.
Before that, let me activate my VNC environment.
So conda activate v and v okay.
So this is the activate environment that I will specifically be using now or what I'm actually going
to do.
Just go ahead and write pip install pip install minus r requirement dot txt okay.
So the installation will specifically happen over here.
Uh all the installation will happen.
And here you'll be able to see that once the installation is taken place I will go ahead and run this
particular code, because here I just require OpenAI.
Now let's understand this particular code and uh, what all things are there in this code.
So first of all I am importing from OpenAI, I import OpenAI and then we are creating a client with
respect to OpenAI.
The base URL over here will be this integrate.api.nvidia.com/v one which is given from the code.
This is my API key okay.
Please make sure that you don't have this publicly visible API key.
Instead, what you can do you can.
As you know that I have already imported Python dot env so we can directly create an environment variable
and use this.
Okay.
So I will do that.
Uh, first of all I'll create an environment variable over here and let me do that first step uh, so
that I can actually use it when I'm creating my end to end project.
So this is my dot env.
Uh, this is my environment variable that I'm going to create.
right Nvidia underscore API underscore key.
And this is my API key okay.
And I can call this API key wherever I want.
Um by using uh this Python dot env file okay.
And that I'll show you once I probably create a rack project.
Now once I probably create this particular client, then, uh, we have to use client dot chat completion
dot create.
Here we are giving the model name.
Again.
It is been given.
The entire code is given over here.
See, I did not write it any like Nvidia name is already providing this.
And just imagine just using this and directly executing it.
It's quite amazing.
Then messages role is equal to user content.
How are you?
Okay so I have written this particular message.
Or let me just go ahead and write uh provide me a paragraph, provide me, uh, provide me an essay.
Okay.
On machine learning.
Okay.
Provide me an article on machine learning.
Okay.
Something like this.
Okay.
So then we are setting up the temperature value top underscore P will be one max tokens 1024.
And stream is equal to true.
When we are keeping stream is equal to true.
That basically means the entire completion will.
We will be able to, uh, see in the form of uh, streams.
And this is basically there in OpenAI right then from chunk in completion.
We are executing this particular code.
Now, let me just go ahead and run this and let's see whether we will be able to get the output or not.
So here I'm going to just write Python app dot Pi.
And uh here uh see machine learning all the answers is coming over here.
It is streaming.
It is giving you the entire output.
This is perfect.
You are able to get the output.
So that basically means you're just able to execute it and how quick it is.
Nvidia Nim.
Trust me, the inferencing is very, very fast and that is what it is going to basically bring up a
breakthrough and an amazing revolution, right?
Uh, in the entire process of generative AI development, and at the end of the day, company really
needs to think about inferencing.
And if I talk about Nvidia, it is the king of GPUs.
Right?
So the inferencing needs to be obviously very, very good okay.
Okay.
So this was it.
Uh, now here, uh, this, uh, simple thing we have actually done.
Now, let us go ahead and do some amazing end to end project.
And this time I'm actually going to show you a project which is basically a kind of RAC project and
how you can use along with Lang Chain.
Uh, that also I'll probably show you.
So first of all, I will go ahead and uh update my requirement dot txt.
So here I'm going to use lang chain Nvidia underscore I underscore endpoints.
This will be the lang chain integration which will actually help you to call all the Nvidia um models
that it has in him.
Then you will be also importing lang chain underscore community uh five CPU.
Along with this, since I'm going to create a Streamlit app, I'm going to use this.
And one more thing that I'm going to use is pi PDF.
Okay.
So all this uh, requirements I'm actually going to use it.
Now let me just open the terminal again with respect to this, I'm going to go ahead and write pip install
minus our requirement dot txt okay.
So the installation will take place.
Uh it is going to take again some time with respect to this.
Let me close this uh folders are visible over here okay.
So once this is done, I will go ahead and start my development.
Okay.
So here, uh, let me go ahead and write final app dot pi.
Okay.
Now final app dot pi.
Uh, this is where I am going to specifically write all my code.
Uh, and, uh, we'll be seeing what all things we are going to use.
Okay.
Now, as you know, uh, whenever we are creating a Rag application, we will be using some kind of
things, right?
Let's say PDF or something as such.
So, uh, I have some of the PDFs over here.
Okay.
So this for PDFs I have over here I will copy this okay.
And let me open this particular folder.
So we are going to read all the PDFs from here from this US Kansas census.
So all the PDFs is over here.
We'll read this and we'll try to create a Rag application wherein we will be asking any questions related
to this.
Here we are also going to perform embedding okay.
Now when I say we are going to perform embedding.
So we will be using Nvidia embeddings okay.
And that is what we are I'm going to show you also so quickly uh let's go ahead and import uh Streamlit
okay.
So first of all I'm going to import Streamlit as st.
Then I'm going to import OS okay okay then from Nvidia I'm going to import uh embeddings and chat Nvidia.
So this is a library that I'm going to import over here.
So this Nvidia embedding will be again used with the help of APIs and chat Nvidia to call any models
that are available in Nvidia name okay.
And this is the library that is used in integration with long chain.
Then uh you have this web based loader.
So since uh uh I okay, I need to probably read from my peer directory.
So for that I will be using from lantern underscore community dot document loaders.
Since I'm also going to use output parser create document chain and recursive character text splitter.
So I am also going to use this.
So these are some of the libraries that I'm actually going to use for my entire course, uh project
right where we'll be using this.
So here you can see the entire uh installation has also been done.
Now let me quickly go ahead and write from dot E and EMV import load underscore dot EMV.
So then we are going to initialize load underscore dot e and v so that we will be able to call all my
environment variables.
Now uh, load uh, the API or Nvidia API key because we need to load it.
So for that I will go ahead and write OS dot environment environ okay.
And this will be nothing but an Nvidia underscore API key.
And this is what I have actually created in my env file.
Right.
So here you go.
And then just go ahead and write OS dot get env.
And here we are going to use Nvidia API key okay.
So my environment variable is ready.
Uh now the next step or what we are basically going to do is that we are going to call our LM model.
So let me just go ahead and write my LM model.
And this time I'm going to use Chat Nvidia.
Right.
So chat Nvidia.
And as you all know I'm going to use which model.
So chat Nvidia is there I guess.
So chat Nvidia okay chat Nvidia.
And I'm going to use my model name.
Uh the same model name that I've actually called.
That is nothing but meta llama 370 billion parameter okay.
So this basically becomes my LM model.
Now once this LM model is loaded, since I have to read from this particular folder all the PDFs.
So let me just go ahead and create my one function, which is called as vector embedding because I have
to create vectors for all this PDF file.
Right.
Uh, here I'm going to specifically use sessions okay.
So that I'll be able to access it here and there.
So here, uh, let me go ahead and write form if vectors.
If vectors I will create a session which is called as vectors not in s t dot session underscore state.
Okay.
And this uh I will create another line.
Now I will go ahead and write s t dot session.
Session underscore state okay dot embedding.
So first of all I'm going to create my embeddings.
And I'm going to initialize to Nvidia embedding okay.
So this is the embedding that we are going to use in order to convert the document into vectors or the
text into vectors okay.
So this is the first thing.
So here you have to also everything looks fine.
Then uh I'm going to probably also create s t dot session underscore state okay.
Dot loader okay.
And this time I'm going to use my pi pdf.
Let's see pi PDF directory loader.
And I will be giving my folder location.
That is US census okay.
So let me quickly copy this over here.
So this is my uh another library that I have to use it Okay, so let's see, uh, till here, if everything
is working fine or not, I will just go ahead and open my terminal and let's see if I'm getting any
error.
Okay.
Streamlit, run final app dot pi.
Okay, so once this is running, I think it should be running.
And I don't think so.
Any problem should be there.
Okay, so this looks fine.
Yeah it is giving me a blank page.
Uh, now it looks fine.
Everything is fine.
Let me quickly go ahead and close this.
Okay.
Now.
Perfect.
So US census I'm actually keeping the up in this.
So basically this pie PDF directory loader is going to read this entire PDF file inside this US census
then.
Now what I'm actually going to do is that uh, after reading from this particular loader write variable,
I'm going to basically write st dot session underscore state.
Session underscore state dot docs okay.
And here uh we are going to specifically use st underscore session underscore state loader dot loader.
So what loader dot loader will specifically give me all the documents over here.
And along with this I, I'm going to probably do the character text splitting.
Uh, because I need to probably do the splitting itself after this.
So once I get my documents.
So then what we are going to basically do is that we are going to write SD dot session, underscore
state dot text, underscore splitter, recursive character text splitter, chunk size.
I'm going to take it as 700 chunk overlap I'm going to take it as 50.
And here you'll be able to see final document is there uh, with split documents, SD, dot docs.
And I've taken the top 30 docs.
Okay.
So we are splitting this particular documents with respect to this.
So guys step by step you have seen this.
First of all we created our embedding.
Then we read the entire directory all the PDFs.
We had it in the loader.
This loader dot load load will basically give you the entire documents.
And then we are taking this particular documents and applying recursive character text splitter where
the entire documents will be divided into chunks.
Okay.
Uh, here the chunk size is basically taken as 700 with overlap of 50.
And finally we get our final document by using the split documents.
Uh, and I'm going to take the top 30 records.
Okay.
So once I probably get the final documents, uh, at the end of the day, I need to convert this into
vectors.
So I'm going to basically write s t underscore session state okay.
Dot vectors.
Uh and here I'm going to use fis again weight dot vectors is equal to here uh we are going to use fis
from documents SD dot session final underscore document and SD dot session state embeddings.
So here we have also created our embeddings.
And the same embeddings will be used over here.
And finally this vector will be nothing but it will be our uh vector database.
In short the vector database okay.
So this entirely this function, what it is going to do is that it is going to read all the PDFs from
that folder.
It will divide all the documents into chunks.
And then finally it will convert into a vector and store it in a vector database.
So that is what this entire function is basically going to do.
Now, uh, it's time uh, we go ahead and probably write s t dot title.
And here I'm going to basically use Nvidia name demo.
Okay.
Uh, once I probably use this uh, as you know that I'm going to probably create my chat prompt template.
So let me go ahead and define it.
Uh chat prompt template I have already imported it over here okay.
So this chat prompt template, as I said it is a kind of a Rag application.
So I'll say answer the question based on the provided context only.
Please provide the most accurate response based on the question.
So here is my context.
Here is my question okay.
Once I probably have this uh, now I'll go ahead and create my entire a single text prompt, uh, text
input.
I'll say enter your question from the documents.
Whatever question you have from the documents, you are going to probably ask it over here.
Now I will also create a button.
So I'm going to right from start button okay.
If s t dot button.
And here let's say I'm saying document embedding.
If this is clicked right.
If this is clicked I will specifically call my vector embedding function okay.
And once I have my vector embedding function, I will go ahead and update this and I'll say, hey,
my vector store DB is ready, okay.
And this time the vector store that we are using is a farce, right?
Fast vector store DB is ready.
And by using which embedding technique.
Is ready using Nvidia embeddings.
Right.
So this embeddings we are specifically used over here.
Right.
So what happens is that as soon as I probably click this button it is going to call this entire vector
embedding.
And it is going to make sure that you have this vector store DB ready.
Okay.
And then finally I will go ahead and uh so once this is done I will go ahead and write if prompt one.
If the prompt one is there, if I am probably searching for anything, and if I press enter, okay,
then the next thing that I'm actually going to do is that create my document chain.
And for this I'm going to use my create stuff document chain here I have to give my LM comma prompt.
Right.
So this we have already seen in our Lang chain playlist itself.
Right.
So here you have this.
And then finally you will be able to see retriever is equal to est dot session underscore state dot
vectors as retriever.
So this vectors db.
When we use with as retrievers this basically becomes an interface to retrieve all the data from here.
Right.
So this basically becomes my retriever.
And since this vector is basically stored in the session state, we are going to basically use this
session state itself.
Then after taking this entire retriever, what we are basically going to do, we are going to create
our retrieval chain.
Okay.
And here we are going to use uh okay.
Create retrieval chain.
And here we are going to use my retriever comma document chain okay.
So this two things I am going to basically use it and close it over here.
Since already my LLM is basically called.
Now this create retrieval chain will basically create my retrieval underscore chain which I'm going
to use it over here.
But at the end of the day this model that is basically getting called it is from name.
Right.
So Nvidia name Nvidia name inferencing.
Right.
Inferencing.
Perfect.
So here it is.
Uh and uh after you get this retrieval chain, all I have to probably do is that I will start the time.
Okay.
So import time.
Let's go ahead and import time over here because I'm going to use the time functionality so that I will
be also able to measure and see how fast this is.
Start time.
Uh, then we are invoking based on the prompt that we have, and we finally get the response and we
are writing the entire response.
Okay.
Now, um, along with this, uh, this meta, uh, open source model, like llama three, also provides
you some context information.
So below, by using again Streamlit, we will try to display the entire we'll try to display the entire,
uh, context also.
So here I'm going to write it from with x t dot x standard document similarity search.
And here we are going to basically display the context.
Now let's go ahead and run this.
And I think it should probably work.
And we are going to basically use this chart in Nvidia itself.
Now let's quickly open the terminal.
So guys now let's go ahead and run this code and see that whether everything is working fine.
So here I'm going to basically write Python final app.py.
Oh sorry I have to basically run with Streamlit.
So Streamlit run our final app Dot Pi.
Okay.
Anyhow, I'll be giving you the entire code in the description of this particular video.
Go ahead and check it out and definitely go ahead and use Nvidia Nim.
Now the first thing over here is that I'll go ahead and click on Document Embedding.
So as soon as I probably click this, what it is going to do from that entire folder, it is going to
take out all the documents, convert that into vectors, store it in the vector database.
There we have used Nvidia embeddings.
So we will just wait till the vector db is ready.
And then we will go ahead and ask any question that we want like a Rag application.
So still it is taking time.
There are many files 4 to 5 files.
So obviously there are a lot of documents.
And again it is basically happening at completely with the Nvidia name inferencing.
So let's wait for some time.
So here you can see the vector store DB is ready.
Let me quickly go over here and see whether I'll be able to see my vector DB Okay.
So now the next thing what I'm actually going to do over here is that go ahead and ask any question
that I want.
Right.
So now over here I'm going to basically say differences in let's let's go ahead and ask this question.
What is the differences in the uninsured rate by state in 2022.
So here I'm going to put this question What is the differences in the UN uninsured rate by state in
2022.
So it obviously needs to pick up all this information and probably give you us an answer.
So here according to the context the difference in the uninsured state are.
So here you can see a low of 2.4% and 16.6% compared to the national rate, uh, 8.0%.
And from the help of document similarity search, you can see what all context it is basically taken
and is giving you the results.
And this entire thing is basically happening with Nvidia Nim, right?
All the models that are probably there from Nvidia embeddings to open source model.
And here we have specifically using llama three itself.
So go ahead and check it out.
And definitely uh, just go ahead and explore more models that you want uh based on different different
use cases like reasoning.
You have something different visual design.
You have something different Retrieval speech.
If you have if you want to probably check it out, you can go ahead and check it out.
And here you also have lot of open source models, uh, foundation models.
So it is up to you go ahead and explore this.
So I feel uh, Nvidia name Nvidia has done an amazing work and definitely this solves a lot of problems.
So yes, this was it for my side.
I will see you all in the next video.
Have a great day ahead.
And all the information regarding this will be given in the description of this particular video.
So thank you.
I'll see you in the next video.
Bye bye.