So guys, we are going to continue the discussion of building the simple LM application using LCL.
Now in this video we are going to talk about Lang serve.
Right.
And what other things we have learned in the past couple of videos.
We are going to probably deploy that entire Lang application as Rest APIs.
So here when I say deploy Lang chain Runnables and chain as Rest API, that basically means how with
the help of Lang serve, we will be able to create Rest APIs, see deployment parts and all where to
deploy, how to deploy.
I will be talking about that in the upcoming videos.
But here, before doing that specific deployment, how you can convert this entire Lang chain, complete
chains and runnables as Rest API is super important.
Okay, so for this, uh, when we are using Lang serve, this is basically integrated with fast API.
So let's go ahead and do this specific coding.
But this is a another amazing module that has been brought by Liang Chen.
And it really makes your application, uh, like it will be very much easier to probably go ahead and
create your entire chain.
Okay.
So I will be discussing about that.
So first of all, what I will do, I will just go ahead and install one more library.
So let's say that I'm in my V and V environment.
So I'll say pip install.
Um and here I will say fast API Okay.
Let me just go ahead and install this.
So I definitely require fast API over here.
So here you can see the requirement is already installed because I have done that particular installation
so that I don't want to waste any time.
So once you do the fast API installation make sure that you update in the requirement dot txt.
Also along with this we will also go ahead and install something called as Uvicorn.
Okay, so this is also one thing that is basically required.
So I will go over here.
Uh, and here you can see requirement already satisfied with respect to Uvicorn also.
Okay.
So this is perfect.
Now let's go ahead and start implementing it as you all remember.
Uh, what all things we have already added so quickly.
Uh, first of all, I will go ahead and import from now I'm using this.py file.
See, this is a different file altogether in the same folder right before we were executing step by
step over here in this particular file.
Simple lm LCL right now in the same folder location that is basics of lang chain.
Here I have created one another file which is called as server dot p y.
Okay.
And here is what we are going to implement.
Everything whatever we have implemented over here.
Right.
And finally we'll be able to see that how the chain API looks like.
So I'll write from fast API import fast API okay.
And then here I will go ahead and write from long chain underscore core dot prompts import chat prompt
template okay.
And similarly from long chain underscore core dot output parser.
We had also used something called as string output parser right.
Chat prompt lit chat prompt template we had used.
We had also used this, uh, string output parser.
And along with this we had also used from lang chain underscore grok.
We have imported chat Grok.
Right along with this we'll go ahead and say import OS.
And then I will also go ahead and write from dot env import load underscore dot env.
We will go ahead and initialize load underscore dot env.
So to load all the environment variables and you know that uh how to get our uh grok API key.
So for that I will go ahead and write grok underscore API underscore key.
This is the variable that name that I'm actually going to create.
And I'll write OS dot get env.
And here will basically be my grok underscore API underscore key okay.
Now quickly I will go ahead and initialize my grok LM model.
So quickly.
Let's go over here.
And you know that.
How did we go ahead and initialize our grok LM model.
So this was it.
So I will just go ahead and copy this entire thing.
Model Chad Grok model this one.
And I'll write it over here.
Right now my model is specifically ready.
And this is what we have learned already in our previous video.
Let me go ahead and import one more library now, which is called as Lang serve.
Okay.
Now before going ahead you need to also install Lang serve right.
And without installing Lang serve it will be of no use.
So since this is a new module, we will again go ahead and right pip install.
And here I'm going to right lang serve okay lang serve.
And with respect to the lang serve I will also give all if you want all the components that is specifically
required.
Otherwise I'll just go ahead and install Lang serve okay.
Okay, so this is for the first time I'm installing because I did not install it.
I wanted to show you, uh, so here you can see I'm having this particular length of 0.222.2 version.
Right.
So once I install it right you can just go ahead and right pip install.
But please make sure that you update in the requirement dot txt okay.
So after installing it over here I've installed the lang so or any libraries that you install.
Please make sure that you update in the Lang sir.
Okay uh sorry requirement dot txt.
Now after importing lang stuff I will quickly go ahead and uh import from lang.
So I will be importing add routes.
Okay.
You need to have a basic idea with respect to fast API.
Okay.
Now this add underscore routes will actually help you to create your APIs.
right?
So now let us go ahead.
And as usual what I will do I will go ahead and create my entire prompt template.
So here the same code, whatever we had in our ipynb file I have copied and pasted it over here.
Right.
This is my system template.
Translate the following into this particular language.
Then chat prompt template user with text.
Okay, then I have already created my model.
I will go ahead and also create my string output parser.
So I will write.
Parser is equal to string output parser.
Okay, now after creating the string output parser let's create chain.
So for creating chain what I need to do I need to combine these three things right.
One is the prompt template.
Another one is the model and the other one is something called as parser.
Okay.
So we'll go ahead and combine this.
And finally we have this app definition that we have.
Okay.
Now for this app definition what I will do is that now we are going to use this add routes.
So if I really want to add any kind of route over here I will go ahead and use this add route.
And inside this add route which is coming from Lang save Lang.
So I have to give my app the first app which I've actually created and still I have not defined the
app also.
So let me just go ahead and define the app.
So I will go ahead and create my app over here app definition.
And I'll say, hey, this will be my first API app.
It is almost like flask, but there is some more additional functionalities that you get in fast API.
But if you have seen this flask tutorials, I think you should be able to understand it.
So I will give my title saying that this is my Lang chain server.
Okay.
And the second parameter that I'm actually going to give is nothing but version is equal to 1.0.
And finally I will also go ahead and give my description.
Description will be nothing, but I'll go ahead and write a simple API server using long chain runnable
interfaces.
Okay, so this, uh, I've actually given it.
So this will basically be my first parameter in this route.
Which app?
I'm actually going to give that second thing I want to give my chain the chain uh, which has the combination
of prompt template, LM model, string output parser, and the third one I have to actually specify
my path.
So here I'm going to specify slash chain as my path okay.
You'll be understanding what exactly this route will basically do.
So here, uh, this is basically adding chain routes.
Okay.
Perfect.
Now it's time that, uh, we can go ahead and execute this.
Uh, so I will write from underscore underscore name underscore underscore is double equal to underscore
underscore main underscore underscore.
This is how we start the entry of the program I will go ahead and import Uvicorn.
Okay.
Uvicorn.
And then I will say Uvicorn dot run.
This is how we run the fast API.
We'll tell the app name I will give the host.
Host will be my localhost okay local host.
And finally I'll also give my port if I want.
So port will be 8000 okay so these are all the information.
Now see what we have exactly done.
We are loading our chat graph model.
Uh here we have created our prompt template.
Then we have created our chain and output parser.
We have created a fast API app.
We have added the route write slash chain.
And finally we are executing this.
So now let's quickly go ahead and execute it okay.
Don't forget to save the file over here.
Now if you remember this particular uh file is present in the location 1.1 Basics of link chain.
So let me just go ahead and write CD 1.1 basics of long chain.
And here I'm just going to write Python serve.py.
This is how you go ahead and exit your execute a fast API app.
Once I execute it, I think it should get executed.
But here I'm getting an error saying that hey pip install se starlet.
Okay, I need this particular dependency so quickly I will go ahead and write it over here.
Okay So quickly I will go ahead and execute this.
Let me do one thing.
Uh, instead of writing it over here, I'll open a new command prompt.
Okay.
And let me write it over here.
Pip install.
SSH.
Starlet.
This is one of the dependencies that we have.
So once I have, uh, written or created the routes, now, what we are going to specifically do over
here is that I will give the entry point and I'll start the execution.
Right.
So first of all, what I will do, I will write if uh underscore underscore name underscore underscore
is double equal to underscore underscore main underscore underscore.
So this is the entry point I will go ahead and import uvicorn okay.
And along with this particular uvicorn uh I will go ahead and write Uvicorn.
Uh, unicorn Unicorn.
Unicorn.
Dot.
Run and app.
Comma.
Host is equal to.
I will give my host as 127 .0.1.
This will be my local host and here I will give my port as 8000.
Okay.
You can also replace this particular values to local host okay.
Now let's go ahead and execute it.
Remember I'm inside this particular folder 1.1 basics of long chain here.
Uh I will just go ahead and write Python serve.py.
Okay.
And once we execute this here you'll be able to see that it is running over here.
Let's click on this particular link okay.
So here is the page that is visible.
Uh right now in the home page nothing is basically visible.
So what I will do I'll just go ahead and write slash docs Once I write slash docs.
So this is what, uh, you know, uh, fast API uses pedantic rules.
Okay.
So that actually gives you all the APIs that is over there.
Okay.
So here, uh, it gives you this entire thing.
You just need to write this particular URL slash docs.
You'll be able to see that I have this chain invoke.
Okay.
Um, here I can pass any values.
Then let's, let's discuss about this, uh, single by single.
Right.
So this basically has created this entire chain configuration, uh, with all the information that I
have over here.
Right.
So, uh, if I just write slash chain, right.
Uh, from that lang serve, you'll be able to see that here.
You have this input schema, right?
Uh, it says that you have to provide the input as string.
Here is your output.
You will be getting in the form of JSON.
Okay.
then you have this invoke function.
Invoke basically means you will provide some value okay.
And based on that value what all values will be providing like language and text okay.
And uh, this is what is the kind of response that you are going to get.
And this is all are there.
Right.
So all this information like let's say I want to go ahead and try out this invoke.
Okay.
So here uh, let's say language is called string.
Uh string and text is equal to string.
This is the request body.
Okay, let me, uh, just go ahead and pass this values over here.
Okay.
So I'll say, hey, language is French and, uh, text is basically hi.
And if I go ahead and execute it, you'll be able to see that it has got executed.
I'm getting the output as salute okay.
So guys, now let's explore more example from this particular documentation.
See I have just created one chain that is slash chain slash invoke.
You can create any number of chains over there.
Let's say I will go ahead and write.
Hello.
My name is Krish okay.
And this is what is the post request that needs to go.
And if I just go ahead and click on execute it here you'll be able to see Bon Jour de la creche.
Right.
This is the output that I'm actually getting.
So this is how you can actually go ahead and create chain.
And along with this documentation, you'll also be seeing that there are some other information like
slash chain slash input schema.
See if I just go ahead and execute this since this is a Get request.
So I'll say slash chain uh slash chain slash input Input schema.
Input underscore schema.
So if I execute it.
So what is the information that I am able to get right.
This is the prompt Im told this is the language.
The text should be a string type and all the information.
Along with this I can also go ahead and see some more information.
So this is with respect to the input schema.
Now how the output schema should look like I can go ahead and hit this URL right.
And that is what is beautiful about, uh, fast API, you know, so I can go ahead and write like this.
I will copy this same thing.
I will go ahead and write it over here.
And let me just go ahead and see my output schema.
Okay.
So this is the entire documentation, right.
Uh, like the swagger documentation we see in fast API.
So here it says that, hey, your string, uh, the output is a string output parse and the type is
string.
That entire information will be seen.
So this is one Post request.
You also have something like batch okay.
You can give the let's let's try this out okay.
So let's try it out over here.
Let's say here I'm going to give language as Hindi I want to convert text into Hindi English into Hindi.
So I'll say hello.
Hello how are you.
Okay.
So I'll go ahead and execute it.
Let's see what kind of output I will get.
So here you can see uh output like Namaste.
Right.
How are you in Hindi.
All the information over here with some more metadata information you are getting so that there is one
more, uh, by default, one more Post request.
When you created slash chain, right.
So many things have got created like slash and slash invoke slash chain slash batch.
And you can also use postman and you can probably use all these things.
Right.
Um, uh, so this is what, uh, you specifically get from this?
Uh, I hope you're able to understand this particular video.
Let me just go ahead and show you with the help of postman also.
Right.
So if you don't know about postman, postman actually helps you to interact with any kind of APIs.
Right.
So let's open this and I will create one get uh, post request, let's say.
So the post request uh, that I really want to get is slash chain slash batch.
Okay.
So here I will say okay, I'll dismiss this.
I'll make it as a Post request.
It is 127 point this the 8000, uh, port address is there.
And here, let me just go ahead and write it over here.
What it is slash n slash batch.
Okay.
Slash chain slash batch.
Now, I know, uh, in my body I have to make this as my raw.
Select my JSON.
Okay?
And here is what I'm actually going to give my entire body right now how my body should look like.
Same this way.
Uh, where is it?
Okay.
This is the Get request okay.
So I just now tried it out, right.
So I will keep this entire is my body in the form of JSON.
I'll paste it over here.
Let's, uh, execute this.
So let me just keep this in string because I need to send it in the form of body.
And once I click on send here you can see invalid JSON body.
Okay.
Let me just remove this I think this is invalid.
Now see once I send it here is what is my output along with my metadata.
So this way also you can go ahead and test each and every request.
And from the postman also you can do this.
You can just go ahead and download postman.
Go ahead and set the post request.
What is the URL and what should be your body right and you'll be able to get it.
So I hope you like this particular video.
You know.
So in this video what all things we basically did, we developed a simple LM application.
First of all I'll try to close this up okay I don't require it more.
First of all, we saw how to develop an LM application with the help of LCL.
Then we also saw how we used Lang server and probably created this entire application in the form of
chains and APIs, right?
That is the most amazing thing because after this, we can take this entire application and we can deploy
it wherever you want.
Okay.
And, uh, in the upcoming videos, you'll also be seeing, you know, in the upcoming modules, you'll
also be seeing that I'll be showing you various deployment mechanisms and techniques.
Right.
So yes, this was it from my side.
I hope you liked this particular video.
I will see you all in the next video.
Thank you.
Take care.