Hello guys.
So we are going to continue the discussion with respect to our Lange Chain series.
Now in this video, we are going to create an end to end Q&A chatbot using open source models.
And specifically we will be using llama now already in our previous video, we have seen this end to
end chatbot with the help of, uh, you know, OpenAI.
Uh, and here, uh, the best thing was that we made it so loosely coupled that we could use any kind
of models that we want.
And.
All right, so similarly, uh, what we are basically going to do is that we are going to go ahead and
do it with the llama itself.
Now, in the case of llama, you definitely require different set of libraries.
Again, I'll be talking about that.
And instead of using chat OpenAI now you will be having an another library which is called as llama
itself.
Right.
So let us go ahead and let us start this.
So first of all I will go ahead and create my app Dot Pi.
Now quickly I will go ahead and import from lang chain underscore core dot prompts.
I'm going to import chart prompt template okay.
So let me quickly go ahead and write chart prompt template.
Now after this I'm going to go ahead and write from lang chain underscore code dot outputs.
Okay.
Output parser.
We are just going to go ahead and import our string output parser.
Right.
This is also required over here.
Then since we really want to go ahead and use Olama.
Uh I'm just going to use this long chain underscore community dot Llms import Olama.
Okay.
Along with this, as you all know, uh, since I will be specifically using.
So it should be olama over here.
As you all know, I'll be using Streamlit, so let me just go ahead and write import Streamlit as st.
And here, uh, along with this, I'll also go ahead and import os.
Okay.
Now quickly, uh, what we are basically going to do is that again, I will just go ahead and open main
dot pi, the initial setup that we really want, right.
This is all specific information so that we'll be able to trace it in our Lang Smith here.
I'll just say with open I instead of writing open I will go ahead and write with Alama.
Okay.
So these are the initial things that we will be specifically requiring.
Okay.
Now quickly we are going to go ahead and create our prompt template.
Now in order to create a prompt template I have already imported that prompt template.
So let's go ahead and define our prompt template to create the similar way like how we have created
the specific chat bot.
Right.
So here I'm just going to go ahead and write prompt is equal to chat prompt template dot from underscore
messages.
And here we are going to use again two things.
As I said uh one is with respect to system and one is with respect to user.
Okay.
I'm writing, hey, you are a helpful assistant.
Please respond to the user queries.
Then you have this question, whatever question we are specifically giving.
Okay.
Now, uh, other than this, you will be able to see that, uh, we have also defined this generate
response.
Right.
So let me copy this and let me paste it over here and I'll show you what all changes will specifically
happen.
Instead of writing open I you know what exactly things we will be basically requiring?
I don't require any API key because this is completely open source.
Instead of writing chat OpenAI here.
Now I'm going to define my llama model.
Right.
And with respect to this llama model, you will be able to just see like which all models I have.
Right.
So in order to check it out I will just go ahead and import command prompt.
Okay.
Now with respect to this command prompt I will just say, hey, go ahead and write llama run.
Um, let's say gamma two.
So this is one of the models that I have.
Uh, I think it is also getting installed.
Okay.
So this is getting installed.
It's okay.
Till then I will just open my command prompt again.
Okay.
And I will see.
Oh, llama uh, let's say gamma model.
So gamma model is also not there.
Let's see uh what all models I specifically have, you know, and then we'll try to run it.
So for that I will go to Alarm.com.
Okay.
What all models alarm actually provide.
We also need to have a look onto that.
And uh we will be seeing that what all models is basically provided by alarm itself.
So here I will just go ahead and click on models.
Quickly.
Go ahead and see the library.
So many different libraries are there like gamma two and all.
Gamma two is the recent open source library that has been brought by, uh, Google itself.
Right.
So you can also go ahead and use that.
Right.
Uh, along with that you have this llama three, you have this quant two deep secret code Phi three.
So many different models are there.
Okay.
If I probably go ahead and check with respect to the prompt two.
Right.
So this is over here I will just go ahead and hide this okay.
What I'm actually going to do is that I'll just go ahead and write Allama five three.
I think this should be there.
Uh, sorry.
Allama, run.
Five.
Three.
Okay.
It should be.
The command should be run five.
Three.
Again.
This is also getting installed.
Okay, so here, whichever libraries you really want to go ahead and probably, you know, use it with
respect to Allama.
First of all you need to download all these libraries.
Right.
And that is the reason here.
You'll be able to see that as soon as I go ahead and write Phi three.
It is getting downloaded.
If I go ahead and write gamma two, it is getting downloaded.
Okay.
So, uh, this is the first important thing that you really need to see with respect to the libraries.
Okay.
So let me just reload it quickly.
So here you have this.
Let's say I want to go ahead and run anything like, uh, Allama Gamma two is here then.
Oh, llama run!
Lava is here.
Llama run!
Solar is there.
Let me just go ahead and take some Mistral.
Also, I don't know whether it is installed in my local or not, but I will just go ahead and check
it out.
So yes, Mistral has been installed.
Uh, so let me quickly we'll go ahead and try to access this model and we'll try to see whether it is
working fine or not.
So if I go ahead and write high here, you can see I'm getting all the messages over here.
Perfect.
This is great.
So let's go ahead and use one of the model which is called as llama over here.
And along with that you also have gamma two model.
This is the recent model that has been there.
Uh the previous version of the model will get automatically removed.
Like you also don't have llama to lie right now.
Llama three is also there, right?
These all are open source models, which is super amazing.
And you can also use this.
Okay.
Now you know that I have this Mistral.
So let me quickly.
And I've been installing any libraries that you want to probably go ahead and install.
Just go ahead.
And any little model you want to install in your local machine, you just need to go ahead and write
this specific command, okay.
And always make sure that you keep on looking on this because this will get keep on getting updated
okay.
So here I'm here I'm writing model is equal to whatever model name I'm specifically giving with respect
to this engine.
It can be llama two.
Llama three.
It is up to you.
Okay then we are creating this entire chain prompt output parser.
We are invoking it based on the input text.
Here we are going to get this.
So this is only the change that we have actually done over here.
And remember one more thing that guys uh that you really need to remember is that whenever we are using
llama, we are specifically only using open source models.
Okay.
Now once this is done, it is time that we can go ahead and use the same functionalities, right?
Whatever functionalities we specifically require over here.
But as you all know, uh, in llama three, in all the open source models, we don't have to even,
uh, you know, play with all the parameters out there.
Right?
So what I can actually do is that I will just go ahead and copy this entire thing, okay?
I don't require this settings.
Also write anything as such.
I'll copy this over here.
I'll paste it over here.
Okay.
Uh, I'll say hey, if user input is given okay.
Just see.
Okay.
What all things I will be doing here, I just need to pass my user input and I need to pass my engine.
Right.
Engine which model we are selecting.
And over here also we'll remove this API key because I don't require any API key.
Right.
So temperature is there, max token is there.
Everything is going on with respect to the OpenAI model.
Let's say that right now I have Mistral.
So I will go ahead and write Mistral over here.
Other models are basically getting downloaded.
So I'll remove all these things later on.
You can put any number of models that you want okay.
This is the basic change.
You know, that is what is specifically done, right?
Uh, let's see whether this open source model supports this temperature max tokens also or not.
Okay.
I'll be giving it over here, but I don't know whether it will be there or not.
Okay.
So, uh, this else if condition can be removed because I don't require it.
That's it.
See how simple it was just replicating with respect to OpenAI, with Olamide itself, right.
So let's quickly go ahead and run this.
So here I will just go ahead and write Streamlit run App.py.
Okay.
So here you can see, uh, let me just go ahead and change this particular message.
Okay.
I had to run App.py.
Let's see.
So I will say CD dot dot.
I have to go through my third folder Olama chat bot.
Now if I'll go ahead and write Streamlit run app dot pi.
So this has got executed now.
Perfect.
Mistral is there.
Let me just go ahead and select.
Hi I think this is an inputs.
Also.
It will be taking temperature and all.
How can you assist you today.
See so fast it is.
And then I will say hello.
Please talk about generative AI, right?
Once I execute it, you'll be able to see that it will also give you the response.
And similarly, uh, if you go towards the Lange Smith okay.
So let me just see here is my output.
And now if I go to my Lange Smith okay.
So if I go to just my Lange Smith Let's say over here, I'm going to do the sign up.
Or you can, once you do the sign in, I think every time it will show you the sign up.
But at the end of the day, you'll be also able to go inside this.
Okay.
Now here you'll be able to see that this entire platform will get loaded.
Okay?
So whatever conversation this time you are having, you're having it with respect to your open source
LM models, which is available in your local machine.
Right.
So if I just go ahead and click on projects.
So here you can see simple Q and a chatbot with llama.
Uh 2.64 seconds.
Hi.
You're basically getting this input.
What was the token size first token this much time it took?
How many number of tokens cost is zero.
You can go ahead and just keep on exploring this, right.
But altogether we specifically design a chat prompt template.
Then we use LM model.
And finally we try to uh display the string output parts over here.
Right.
So everything is probably visible with respect to this.
Right.
So yeah, uh, I guess you are able to understand this.
See, the entire output is also basically getting displayed, which is good enough right now.
Uh, let's see like how much time it is probably taking this chart probably now see once gamma two and
all will get added.
What I will do is that I will also give an option over here so that you include gamma 2 or 3, any number
of models over here.
You can select that models and you can start writing your code or start using this LM models for any
kind of response.
Okay.
So I hope are you able to understand this, how to create an end to end Q&A project with the help of
Allama.
So yes, this was it from my side.
I will see you all in the next video.
Thank you.