So guys, now let's continue the discussion with respect to this particular project.
Now what we are going to implement over here.
So here you will be able to see that I have a research paper folder okay.
And inside this folder I have this two amazing research paper.
That is attention is all you need and overview of large language models.
So I have kept all both these PDFs inside this folder.
And we will consider this as an external data source.
And we will try to ask any question.
And probably we should be able to get the response from this particular research paper through this
document.
Q&A okay.
So how we will be executing this again over here we are going to be specifically use this chat grok
okay.
And uh, along with this you'll also be seeing that, uh, we will go ahead and uh, you know, uh,
implement open source LM model instead of just using open AI.
Okay.
Uh, so that you will also everybody will be able to execute this, Okay.
So quickly, let's go ahead and do this now.
First thing first I will go ahead and write from dot env import load underscore dot env okay.
And then uh we will now just go ahead and execute this load underscore dot env.
Now this is specifically used to load all the environment variables.
Right now I'm going to load the grok grok API.
And this will be very much important.
And for some number of hits grok completely provides you free API itself.
So anybody can probably go ahead and create your account okay.
You have to go ahead and create your account.
And based on that you can actually go ahead and create your keys.
Now here I will go ahead and write OS dot environment dot environ.
And here I'm going to use my grok API key.
So I'll go ahead and copy this I'll go ahead and paste it over here.
Okay.
Along with this what I will specifically do is that I will just go ahead and write is equal to OS dot
get env.
Okay.
Once I do this I will quickly go ahead and write my open uh sorry grok grok API key itself.
Okay, so once I get this grok API key then I will be able to interact with any model that I want okay.
Now uh, once we are setting this up, I can also make sure that I can get the direct key from here.
Okay.
Um, so let's go to the next step.
And here, uh, let me just go ahead and write.
This is my grok API underscore key, which is nothing.
But if I just go ahead and write os dot get env, it is just going to get it from the environment variable.
Okay.
So this way also you can actually get your API key.
Now is the most important step is creating your LM model.
For this we will be using this Chad Grok.
And here we have to first of all provide our grok API key which will be equal to nothing, but it will
be equal to this grok API key.
Right.
So this is the first parameter that is required over here.
And the second parameter is nothing but my model name.
So in order to provide the model name I will go back to my website of grok.
And here you'll be able to see if I go to the first home page.
Here are different different models that you can specifically use right.
So you can provide this llama three 8,000,000,008 192.
You can provide llama three 7,000,000,008 192 gamma seven.
Be it.
So let's go ahead and select this I will go ahead and write like this on itself.
Gamma dash seven B dash it okay.
So I will just go ahead and write in the same way.
So here you have this gamma.
7BIT okay.
So we are basically going to use this LLM model like this.
You can also go ahead and specify your llama three model if you want.
So let's say I want to go ahead and use llama three model.
I can go ahead and specify like this.
So let's continue with llama three so that uh it's okay.
You can use any of the open source models.
It is up to you.
Now I'll go ahead and define my chat prompt template.
Okay.
Now inside this chat prompt template I'm just going to use dot from underscore template.
Now is the most important thing.
Like see the previous ways how we used to create chat prompt template.
Now this will be a little bit different because here we need to provide some context information also.
So here I'm just going to say hey um I'll go ahead and write it over here.
I'll say, hey, answer the questions based on the provided context only Okay.
And then I will say next sentence.
Please provide the most accurate response.
Response based on the question.
Right.
And here I will just go ahead and define my context.
This will basically be my context.
Here I will give a placeholder.
Now now why I am just using this keyword.
We will try to understand okay, because this is important.
The libraries that we have used this create stuff document chain and create retrieval chain.
This has a separate meaning for this kind of context message that is seen in the prompt template.
Okay we'll discuss more about it.
And then here I will just go ahead and write my question.
This question will basically have my input.
Okay.
Now here is my prompt.
So here in this prompt it is just like a document Q&A.
In case of document Q&A I need to provide a context over here.
What this context will be replaced with.
I will just discuss about it once we proceed.
Okay.
So this is my prompt, a simple prompt that I have actually written.
Now the first thing, as you know, I have to create my vectors, right?
Create my vectors.
And this time I'll be creating my vectors.
I'll say vector embeddings.
Okay.
Now inside this function, my main aim will be that I should be able to read all the documents from
here.
After reading the documents, I should be able to read the content inside this.
I mean, read this research paper, read the content inside this and after reading it make sure apply
recursive character text splitter along with that, split all this particular documents and just by
using files, we can proceed further and we can go ahead and probably store them in some kind of vector
database.
Okay.
So here what I will do, I will quickly go ahead and use uh, define this function which is called as
create vector embedding.
But here since my vector store will be used in some other, uh, code.
Also, what I will do is that I will try to use session state over here.
Session states actually helps you to remember uh, your vector store DB specifically this is for vector
store DB.
That is the reason I'm trying to create some session state.
This session state variables will be able to access in other functions also whenever it is required.
Okay, this is just to make sure that your entire application has some kind of memory with respect to
this particular chatbot.
Okay, so here I will say if my vector's not in s t dot session state, I'm just going to try to create
a vectors variable, which will be my session state variable.
So I'll say TX dot session underscore state.
Okay.
Then I will just go ahead and write it over here, and I will just go ahead and say TT dot session underscore
state.
First of all, I will go ahead and create my embeddings.
Now in this case my embeddings that I'm actually going to use is nothing but my embeddings.
Okay.
Now whatever embedding I'm basically using, I will also keep this in the session state variable right.
Specifically in this particular session state.
Then we will quickly go ahead and write SD dot session state.
Another session state we will go ahead and create.
And inside this I will go ahead and write a loader.
Right.
This loader will be my data ingestion step right now.
In this case I'm going to use this pi pdf directory loader okay pi pdf directory loader.
And inside this I'm going to give my folder name.
What is the folder name that you have?
It is nothing but research underscore paper right.
So this is the folder name that we will be giving over here okay.
So let's go ahead and give this.
So here you can see this is my research research papers papers I should right okay.
Now this is nothing but this is my data ingestion step.
Perfect.
We are loading something, loading some information.
This can also come from Streamlit directly.
It is up to you right?
How?
You basically want to go ahead and upload this.
Now.
After this I will just go ahead and say st dot session, underscore state dot documents.
And uh from this loader I'm going to load all the documents.
So for that again I have to call this st dot session state dot loader dot load.
I will load all the documents over here.
And uh this will basically be my complete document loading.
I'll just write this comment so that you'll be able to understand it okay.
Then uh, what we are going to do is that I'll go ahead and write s t dot session underscore state dot
text splitter.
I have to use some kind of text splitter over here.
And this text splitter will be nothing, but it will be assigned to a recursive character text splitter,
which we have already discussed.
Inside this there will be a chunk size of 1000, let's say.
And then I'm going to use a chunk.
Overlap of 200 okay.
I'm just going to use this chunk uh, with respect to this okay.
Okay.
Perfect.
Uh, so this is my recursive character text splitter, which I'm actually doing it.
And then finally it is just dot session underscore state dot final underscore documents.
This will be my final documents because I have to apply this text splitting technique again I will go
ahead and save everything in the session.
That is what I'm actually doing, so that my entire application has some kind of memory whenever it
has to refer to.
Okay, now I will be using this session state dot text splitter.
And I will use this split text splitter.
Dot split documents.
It is called as split documents.
This way we are actually splitting all our documents right.
So for this I will again where my documents is basically present.
It is present over here.
Okay.
Just to make sure that this will not take much time, I will just take, uh, this documents and I'll
say, hey, starting 50 documents, just go ahead and do it right.
You can also do the remaining one so that it will not take much time for doing this entire processing.
Okay.
That is the reason I have just given top 50.
Uh, now I will go ahead and write s t dot session underscore state dot vectors.
And here we are going to basically use for this.
See this last step is all about converting your final documents.
First of all converting that into vectors.
And after you convert everything into vectors, all the text or documents into vector, then you try
to store it in some kind of vector database.
Okay.
And this vectors that you are seeing SD dot session dot vectors sessions underscore state dot vectors.
This is your vector database.
So I will go ahead and write files dot from underscore documents.
And here you have SD dot session state dot final underscore documents.
It is final underscore documents.
And here I'm also going to the second parameter we need to give is our embedding technique.
Right.
So if I go ahead and write SD dot session underscore state dot embedding whatever embedding techniques
I have actually used.
Right.
That's it.
You know.
So finally after executing all these things, you'll be seeing that you're converting this into vectors.
You don't even have to return this vectors because already we are saving this in the session state.
Perfect.
Now quickly, let's go ahead and do this.
I will go ahead and create my prompt.
And here I will write SD dot text underscore input okay.
And here we are going to basically use enter your enter your query okay.
From the documents or from the research paper I'll say okay.
And then finally, uh, I will go ahead and do this.
I'll go ahead and just write if s t dot button, okay?
And I'll say, hey, document embedding and I'll just create a button, okay.
Over there.
This button will be responsible in running this particular function and creating our vector store.
Okay.
So as soon as we run this this is basically going to read all the research papers.
It is going to probably convert that into documents.
And then it is finally going to convert into a vectors.
Okay.
So here, uh, what I will do is that I'll just go ahead and call this and I'll say hey go ahead and
call this create vector embedding.
And after doing that, if once it is successful I'll go ahead and say read dot write.
Your vector database is ready okay.
So that you will be able to query from it okay.
Now it's time to check the Power of Grok API.
Okay, so once your vector database is ready, uh, the next step will be that, uh, we will go ahead
and check our, uh, you know, the specifically we'll check, uh, that how powerful is the grok API?
So that is the reason I'm importing time.
So here, uh, I will just go ahead and rename this to user underscore prompt.
Okay.
So finally if I give this user underscore prompt if user is giving any prompt okay.
Now is the most important step.
What should basically happen.
You know that my vectors this is t dot session state dot vectors has my vector db vector store db right
now when as soon as my user prompt.
Basically user is writing some input and pressing some enter or pressing enter right?
It should first of all create this stuff document chain.
And now let's discuss about this stuff.
Document change what it is.
Here we are going to give two important parameters.
One is LM model and the second one is nothing but our prompt okay.
So this should be the prompt.
Now let me just go ahead and define this variable document underscore chain.
Now see what this exactly does.
Create stuff underscore document chain.
It creates a chain for passing a list of documents to the model.
Okay.
What it is basically doing, it creates a chain for passing a list of documents to the model.
And here the two important parameters are LM and prompt.
As soon as I give this here, you'll be able to see that in the prompt you have this variable called
as context.
Right inside this context, all that list of documents will specifically go and whatever specific input
you are giving it will go over here, right how it will go.
I will just combine it further with one more important retrieval chain, which will discuss about okay,
but in short, this is providing a list of documents in specifically in this particular context.
Like it will get appended over here.
And this entire chat prompt template will be created from here itself.
Okay.
Now further, we will not keep quiet with this document chain here.
What I will do is that I will go ahead and write st dot session underscore state okay dot vectors okay.
You know this it is a vectors is basically there.
I will go ahead and write this as as retriever.
Now you know that we need to convert.
Uh, if I want to retrieve any information from my vector db, I need to first of all convert that into
a retriever.
Right.
So if I go ahead and write as retriever, let me just check the spelling.
I hope it is fine.
Okay so just a second as underscore retriever.
Okay.
So I'm just going to convert this into retriever and finally save this particular value in the form
of a retriever variable.
And what this retriever acts as, it acts as an interface to pass to query your vector database.
In short.
Right.
So this is my vectors over here.
I've used this t dot embeddings techniques that I've used over here.
This is perfect.
Now once I get this as a retriever you know what we do is that obviously this retriever is like querying
a specific vector DB.
This document chain has the context of a list of entire documents.
So once we combine both of them, in short we are going to create a retrieval chain that is create retriever
chain.
And here two parameters are basically given.
One is retriever and the second one is document chain.
Okay.
Once we do this we will be able to invoke this entire thing right.
So here I will go ahead and write retrieval underscore chain.
Right.
So this actually is my chain which has document chain which has retrieval combined together.
Okay.
Now let's go ahead and see the power of this grok API.
You'll just understand.
So first of all I will just go ahead and write a start timer over here I'll say time dot process time
process underscore time.
And once we process this you will be able to see my response over here.
And I'll go ahead and write my retrieval chain.
Okay.
Dot invoke okay.
And you know what kind of input I'm actually giving.
It is nothing but input over here.
Right.
This is the same input that we are giving over here.
Right.
The same variable, the same input we are giving over here.
And here we are going to just replace this with user prompt.
Now we will be able to get the answer after this.
But before that we will go ahead and calculate our response time.
Okay.
So let's go ahead and print.
I'll say, hey, uh, I'll use a f string.
I'll say response time is nothing, but it is equal to.
It is equal to.
I will go ahead and write over here time dot process underscore time minus start time.
Okay, once I do this I will be able to get my, uh, the time how much it time how much time it has
basically taken for this retrieval chain to get executed.
And then finally we go ahead and write our response.
So I'll go ahead and write TX dot write let's say response off answer okay.
Done.
Finally we will go ahead and write uh, with uh c you also need to know that what kind of response,
what kind of other context we usually get.
Right.
So for that also you can actually go ahead and display it.
So I'll go ahead and write Streamlit with a Streamlit uh expander.
And here I'm just going to write with St Dot expander.
the.
Document similarity search.
Okay.
So I will go ahead and write for I comma doc in enumerate okay.
And we basically take C along with this response of answer no.
There will be another one variable which is called as response of context.
That will actually give you all the relevant context information from that particular documents.
Okay.
We'll go ahead and write this also.
So I'll go ahead and write SD dot write doc dot page underscore content okay.
Now quickly let me just go ahead and uh make sure that right all the other information I'm just displaying
what all other context is basically coming over here.
I'll just give this.
So one final change that I really wanted to do.
Okay.
So after writing this code, see we are using all of my embeddings.
Uh, it is of uh, no worries, you can definitely use it.
But sometimes, you know, since this is running in our local machine, you know, it may definitely
take a lot of time to probably, you know, convert that into, you know, embeddings specifically convert
all the text into vectors.
So that is the reason what I will do is that over here, instead of using my embedding, I will just
go ahead and use OpenAI embedding because OpenAI embedding will be in the form of APIs.
Uh, the reason why I'm doing this is that because see, if I don't do this automatically, it is going
to take a lot of time just to probably, um, uh, convert all our text into vectors.
Along with this, we will go ahead and load our opening AI API key.
Okay.
So please make sure that you do this.
You can go ahead and try with all of my embedding also.
But as soon as you will probably run it, you will be seeing that it will take a lot of time, you know,
and I have already done that.
So that is the reason I'm saying you, uh, but I think with the help of OpenAI embedding, it should
happen very much quickly.
Right.
So let's go ahead and execute now.
So I will just go ahead and clear my screen.
I'll say, hey Streamlit, run App.py.
Okay, so now my entire website is over here.
I will click on Document Embedding.
Let's wait for some time.
Now as soon as I click on Document embedding, see what is happening inside the hood.
Right.
You should really understand this.
So here this function has got created.
Now this session state variable will get created.
We are going to create embeddings.
Then it will become loader.
It will read all the research papers inside this.
Then you have this documents.
Then you have this text splitter.
Similarly.
Finally you will be able to create your vector database.
Right.
And all these things are there.
Perfect.
It looks good enough.
Let's see whether it has got executed or not.
So yes here you can see the vector database is ready.
But if you try to just use uh uh, you know, embedding it is going to take forever because that entire
embedding is going to just happen in a local machine.
So let me just go ahead and ask, what is Transformers?
I'll ask a question, what is a Transformers?
And I will go ahead and press enter.
So here you can see based on the provided context, transformer is a model architecture that relies
entirely on attention mechanism to draw global dependency between input and output.
Right.
And here is your document similarity search.
Right.
So from all this context, it was able to retrieve from the uh research paper, which is amazing right.
This looks quite good.
Right.
Uh, let's say uh what I'll go ahead and write what is a large language model.
Okay.
And I'll go ahead and press enter.
Or based on the context.
So much information is there.
And here you can also see your entire context.
Information is also visible.
Right.
Pretty much nice.
So you're able to see this entire information.
See uh if you want to get more larger context information, uh, more larger information with respect
to this research paper, what you can do is that you can go to the code and you can change, uh, this
things the chunk size, you can keep the chunk size to 10,000, and you can probably keep the overlap
to 500 and keep on doing this.
Right.
But here, uh, in short, we have actually got this.
Uh, again, I'll tell you why I have used OpenAI document embedding.
The open source LM model is specifically used, but other than that, I think everything looks fine.
Okay.
Um, uh, let me just go ahead and write.
What is attention is all you need, okay.
If I want to ask this question, I will just go ahead and press enter.
And this is your entire information, right.
And all the related context information is over here.
See?
Everything is visible.
This is good enough over here.
I think you are able to understand this specific project.
Um, this was it for my side.
I'll see you all in the next video.
Thank you.