Hello guys.
So we are going to continue the discussion with respect to Lang Chain.
And now in this particular video we are going to talk about conversational Q&A chatbot.
So we will be building some amazing conversation Q&A chatbot.
You will be understanding how you can go ahead and build this.
Now see this in many Q&A application.
We want to allow the users to have a back and forth conversation, meaning the application needs some
sort of memory of past question and answers, and some logic for incorporating those into its current
thinking.
In this guide, we focus on adding logic for incorporating historical messages.
Further details on the chat history management is covered in the previous videos, right?
We have already discussed about some of the chat history management terms and now we will be covering
two approaches.
One is in chains in which we will always execute a retrieval step, and the second one is agents, in
which we give an LLM discretion over whether or how to execute a retrieval step for multiple steps.
So already we have discussed so much things about chat history.
Right.
But the main idea of using this chat history specifically for a Q and a chat bot.
So in this video I will go ahead and explain you that okay.
Now first of all, uh, over here also, I'll be making sure that I use open source models.
So for this I will be again importing import OS.
And then from here I will go ahead and uh write import or from dot env import load underscore dot env.
Right.
So we are also going to go ahead and import this.
Let's see whether it is executing fine.
So we will go ahead and initialize this load underscore dot env.
Because we will be having all our environment variables now with respect to the environment variables.
The first key that I will be requiring.
And here I will go ahead and write from long chain underscore grok import chart grok okay.
And I think we have repeated this code many multiple times.
Right.
So I'm just showing you again and again so that you don't forget this okay.
Now when I'm importing this chart grok along with this you'll be able to see that I will also go ahead
and take my API key from my environment variable.
So for this I will go ahead and write OS dot get env.
And inside this we will go ahead and write grok underscore api underscore key okay.
So once we get this uh the next step will be that we will go ahead and call our uh LM model specifically
over here.
So for this we will go ahead and use llama three in this particular use case okay.
So this is where we specifically get our LM model.
So this is the fundamental thing that we should definitely do in each and every um, basically whenever
you are developing most of the application, this is common.
If you are really interested to just work with open source model.
Grok is the grok.
Infrastructure is the thing that you should really go ahead with.
Okay, now let's work on some of the important things.
So first of all, what I will do is that I will go ahead and install pip install bs4.
Okay.
So this is a library which is called as Beautifulsoup for this is used to extract a HTML page okay.
Extract the content of the HTML page.
So here you can see that the requirement is already satisfied.
I have selected this v and v environment and make sure that whenever you do this installation, you
update that in the requirement dot txt in this specific project.
Okay, now the first thing what I am actually going to do over here is that I will quickly go ahead
and upload or import some of the libraries.
Okay, so I'm going to import something called as chroma.
So here in this code I am going to go ahead and import chroma from lang underscore chroma.
Along with this I am going to use a document loaders and will be importing from a web based loader.
Because I want to read the content of a web page and we'll try to create a conversational queue and
a chat bot.
So that basically means I'll be having a website which will be my external data source.
And from that I will try to chat through my chat bot.
Okay.
And whatever conversation we basically add, we will also add our message history.
Right then.
Along with this we are going to use this chat prompt template and you know where it is available inside
prompts.
Now instead of using OpenAI embeddings, I will remove this okay, because I will not be using OpenAI
embeddings because it will charge you some amount of money.
Instead, what I will do, I will go ahead and use this particular code wherein I'll go ahead and import
my TF underscore token.
If you remember, how did I import h underscore token along with this in my environment variable, this
is also present okay.
So from Langston underscore Huggingface import Huggingface embeddings.
And this h f underscore token from where do you get it is you get it from your huggingface, uh, account.
Right.
So this is the embedding that I'm actually going to use, right?
Huggingface embedding.
So let's go ahead and execute this.
Perfect.
Till here I think everybody everything looks fine.
And then after this we are going to also import recursive character text splitter so that we will be
able to split our document and chunks okay.
So once we upload or once we load all these things now we are also going to use two important functions
also.
Right.
And this functions will discuss about it.
So one is from long chain dot chains right.
So inside this whenever we try to create a chain by using this library or by using this function we
will be able to create chains okay.
So create retrieval chain like let's say if I if you know about retrieval, retrieval is nothing but
it is an interface to a vector store database.
Right.
And if I want to create a chain with this particular retriever, I have to use this particular function
which is called as create retrieval chain.
Okay.
We'll discuss more about this as we go ahead.
The next thing that we are going to use is something called as combined documents create stuff document
chain okay.
So as you go ahead, uh, in the future, right.
In an upcoming, uh, sessions and modules, I will be talking about text summarization.
Now when we discuss about text summarization, there we will be discussing about three types of text
summarization.
One is stuff, one stuff document chain, one is map reduce and one is refine in stuff document chain.
What it does is that it combines all the documents and then it sends it to the prompt template.
And that is why we will be using this specific uh uh, function.
Okay.
So I will show you everything with an example.
Okay.
Now let us go ahead and execute this okay.
So here we have got executed it.
Now I'm going to use a web based loader.
And let's say I'm going to use this particular website okay.
So let me just go ahead and quickly import BS4 also.
So I will go ahead and write import BS4 because I'm going to use BS Beautifulsoup for now here inside
this web loader I'm giving my web path.
This is the URL that I'm trying to read the content from okay.
And then here you'll be able to see I'm using a class which is called as post content from that page
and post title.
So if I go ahead and hit this page, let me just show you by opening this page over here, okay.
Let me open my browser and let me just hit this page.
Okay.
So here you can see that this is the entire page content.
And I will be using this particular page content as my external data source.
Okay.
Now from this what all fields I'm really interested in.
So I am interested in getting the post content field, post title field and post header field.
So if I go back if I just go ahead and do the inspect.
So here you'll be able to see I'm getting the post title field, post content field and post header.
Post header will also be somewhere here only.
Yeah here.
So all this information will try to extract from this.
Now, uh, let me close this console.
Let me go back to my code.
Okay.
Now, once I probably, uh, use this loader that is a web based loader.
This is for the data ingestion from a website.
I will go ahead and write loader dot load and I will load all the documents.
Okay.
So here I will be able to get my documents okay.
So if I go ahead and execute this and see my documents you'll be able to see that one single document
I'm able to get over here.
And all the information is basically shown over here inside this particular page.
Content.
Perfect.
Now usually what we do is that whenever once we get all this content, it is a good practice that we
try to divide this entire document into chunks of document.
The reason is very simple, because every LM models specifically have some kind of context size, right.
So it is better that we try to break all this particular documents into chunks.
So for this we will be creating a text splitter variable.
And here we will be using some recursive character text splitter.
And let me go ahead and write.
Our chunk size is equal to 1000.
And here we are going to basically use chunk underscore overlap with 200 okay.
So once we actually do this uh then we are basically going to create our text splitter over here.
The next step, uh, what we are going to do is that I'm just going to write splits is equal to splits
is equal to text underscore splitter, dot split underscore documents.
And whatever docs I have I will be giving over here.
So it will basically go to do the splitting.
And after doing the splitting I will go ahead and store this entirely in our vector store database.
Right?
So for that I will write from chroma dot from documents and I will give this documents is equal to whatever
splits I'm actually getting over here okay.
And here I'm going to apply my embedding which will be equal to this particular embeddings that I have
actually used.
That is nothing but huggingface embedding.
Right.
So here you can see that I have initialized this Huggingface embedding.
Perfect.
So till here everything looks good.
Uh, this will finally give me my vector store okay.
Vector store.
And this we have already reported in our previous video.
Right?
And if I really want to convert this into a retriever, all I have to write is that I have to write
something like vector store dot as retriever.
Right.
And here is your entire retriever right now.
So this is the retriever, uh, over here.
Okay, perfect.
Let's see.
So here is your entire retriever that we are able to get it.
Okay.
Now, this is fine.
Now it's time that, uh, once we get this entire retriever.
And this is for the vector store retriever basically means it is an interface with respect to vector
store.
So that whenever I ask any query, I should be able to get it from the vector store itself.
Now this is important.
And this you have actually seen it.
Okay, now it's time.
We will go ahead and define our prompt template okay.
See the same thing what we are doing.
Because after this I'm going to execute more things over here okay.
So let me do one thing.
Let me go ahead and define my system prompt.
So this is my system prompt.
I'm saying hey you are an assistant for question answer task.
Use the following piece of retrieve context to answer the question.
If you don't know the answer, say that you don't know.
Use three sentence maximum.
Keep the answer concise.
Okay.
And here a new new line.
And here we will be passing the context.
Now what this context will be getting replaced with I will talk about it okay.
Then I'm going to take the system prompt.
I'm going to use it as a chat prompt template.
I'll again dot from underscore message I'll use I'll give a system prompt over here.
And along with that human whatever input I'm giving with respect to the human I will pass it over here.
So here we can actually go ahead and pass this particular input.
And this is my system prompt, which is basically saying how the system should behave or how the LM
model should act like, okay, now this is absolutely fine.
Now my time, like now is the time that we need to attach this retriever along with the system prompt
and along with the prompt.
Right.
So here we really need to create this particular chain because this retriever is pointing to the vector
store database.
This is my system prompt based on which I should be able to get my information.
Um, which is this is specifically my prompt.
So I need to probably go ahead and create a chain.
Now the one thing is that how do I get this context information?
Now to get this context information I will first of all go ahead and create my question answer chain
okay.
And here we will go ahead and write create stuff document chain.
And here we are going to pass LM comma prompt.
Now the reason I'm passing my LM model and prompt okay what this exactly is going to do okay.
And that is the reason we have imported this create stuff document chain I've already told that this
create stuff document chain is going to make sure that it will combine all the document and it will
pass in the form of the specific context in that specific chain.
When we create the chain with respect to system, prompt with respect.
Sorry.
With respect to this particular prompt, LM model.
Right.
You'll be seeing that when we use Create stuff document chain.
Here I'm passing two parameters LM and prompt.
Now this prompt right write is requiring some context information.
Right.
And this context information needs to be provided with respect to all the list of documents.
Like what list of documents you have over here.
Right.
All this list of documents.
Right.
We did this particular split and all right.
So that it will combine and it will send it to this particular prompt itself in the form of context.
Right now, if we are doing this, the next step will be that we need to go ahead and create our entire
rack chain.
For creating the rack chain, we will be using a create retrieval chain.
Now inside this create retrieval chain I'm going to make sure that I pass my retrieval.
Along with this I also pass my question answer chain right.
So this create stop document chain will also get passed over here.
Now I have retriever I have LM model I have prompt.
So these all are connected in a case of a retrieval chain itself where I will be able to get the response
okay.
Now once I execute this, now it's time that we will go ahead and execute it.
So in order to execute it I'll just write RAC underscore chain dot invoke.
And here we are going to give my input.
So I know my key is basically a input over here I will write hey what is uh.
Let me see.
What was the page over here that we use?
What is?
I'll ask any question.
What is, uh, task?
What is self reflection?
Okay, I'll ask this question.
Okay.
From here.
So here, if I go ahead and ask what is this?
I should be able to get my response.
And my response is nothing, but it will be over here.
And with respect to this, I will go ahead and write a response off and I'll display the response.
So here you can see what is self reflection.
And I am getting this entire context.
See this context is basically having this particular documents, how it is combining all the documents
and giving it to the prompt.
It is because of this create stuff document chain.
And this stuff document chain is basically used in text summarization, where it combines all the document
and send it to the prompt.
Write further from the prompt it goes to the LM model.
Right.
So here you can see this is my entire context.
And from that I'm able to get the answer.
So if I just want the answer over here I will go ahead and write a response of answer.
Perfect.
And here you can see self reflection is a mechanism that allows autonomous agents to improve iteratively
by refining action and more information.
Okay, now here you can see we have used built in chain constructors, which is called as create stuff
document chain and create a retrieval chain.
Right.
But at the end of the day, the basic ingredients that you will be able to find is something like a
retrieval prompt LMS.
Right.
Just a way to how to replace data with respect to this particular context and how to replace, um,
how to probably get the answer with the help of this particular retrieval.
Right.
But now it's time that we start working towards adding chat history.
Okay.
So here we are going to basically go ahead and add our chat history.
Now how to add this chat history right.
Let's say I've asked a question over here right.
What is self reflection.
Now I will again go to go to this and ask another question over here.
Let's say I'll say I'll just continue and ask.
This is my invoke.
And here I'm just going to take the input.
Now instead of writing what is self reflection I'll say how how do we achieve it?
Okay.
So this is what I'm going to ask.
Now here you can see achieving task decomposition.
It is basically same okay I have I was trying to ask with respect to what is self reflection.
But here it is probably giving me some other thing like how do we achieve it.
Achieving task decomposition is is basically selecting.
It is a tree of thoughts involves so and so.
But I wanted the information with respect to self self-reflection.
Now, the problem with this rank chain is that it's not able to understand the context, right?
It is because it obviously does not have any kind of chat history.
So what I will do now is that I will go ahead and show you how we can go ahead and add chat history,
along with the prompt template, and how we can do the same task and see how things work with chat history
over here.
Okay.
So let's go ahead.
And uh, first of all, what I'm actually going to do is that for adding chat history, I will be importing
from Lang Chain.
Okay.
And this chat history specifically for conversational Q and a chat bot I will go ahead and write lang
chain dot chains import create history aware retriever.
Now the kind of retriever that I am going to create or the chain of retriever that I'm actually going
to create.
See here also we created a chain of retriever only.
Right.
So this is nothing but create retrieval chain.
Right.
So this is nothing but a retrieval chain.
But when we use this create history aware retriever chain, that basically means the retriever will
also know about the history of information or history of conversation that has been basically made with
the LM model.
Okay.
So we are going to probably create our chain with respect to with the help of this particular function.
Okay.
The next thing is that I will go ahead and use from lang chain.
Underscore core dot prompts okay prompts I'm going to go ahead and import.
Messages placeholder okay.
And why I have to use this because I will be able to create a chat history.
Uh.
Uh, I'll just say that, okay.
This can be a key value pairs, uh, where I will be storing all the information.
Right.
And that is where I will be using this message placeholder.
Just wait for a second.
I will just make sure that in the coding will be understanding where we use this message placeholder.
Right.
If we define a message placeholder with any variable in that particular variable, only all the chat
history will be saved.
Okay, this is what I really want to talk about Okay, now I will go ahead and create a prompt okay.
So this is basically called as continuous uh contextualized queue system prompt okay.
So I'll say given a chat history and the latest user question which might reference the question in
the chat history, formulate the standalone question which can be understood without the chat history.
Do not answer the question, just reformulate it if needed.
Otherwise return it as it is.
Okay, so this is the kind of, uh, prompt that I have actually created over here.
I'm saying, hey, please, you have the chat history.
You have the later user question which might reference the context in the chat history, formulate a
standalone question which can be understood without the chat.
Do not answer the question, just reformulated if needed.
Otherwise return.
That is okay.
So simple uh, prompt that we are going to use over here.
Now for this we will be using uh I will be creating a prompt template.
Now see I have given the system message with respect to this.
And this is where I'm going to use message placeholder.
Now in this message placeholder I'm creating a uh variable like chat underscore history where all my
history of conversation will be stored.
Okay.
And here is my human input okay.
So guys now here you can see that I have created my chat prompt template.
Now instead of using a retriever I will go ahead and create this, uh, history aware retriever.
Right now I will be getting a new type of retriever instead of the previous retriever.
And here I will write history aware retriever is nothing but create history aware uh, retriever.
And here I'm going to provide the parameters LM, uh retriever.
So let me just go ahead and write this retriever.
Along with this, I will also go ahead and provide my contextual cue prompt.
Okay.
Q prompt.
In short, what we are doing over here is that we are upgrading this retriever to history aware retriever,
which will also be able to retrieve the results, uh, or retrieve the uh, get the results from the
vector store DB.
Considering considering the considering the chat history.
Okay, that is what we are basically going to do.
And now if I just go ahead and display this, you'll be able to see this entire thing over here right
now.
This is perfectly fine till here.
Everything looks good.
Okay.
And, uh, we are going on very on, very on, like on point.
Okay.
Now it's time to go ahead and create the chain.
Okay.
Now for to create the chain.
What I will actually do is that again, we will go ahead and write the same question answer chain.
Here we are going to use the create stuff document chain okay.
And here we are basically going to use my LM comma QA, uh QA or what is the prompt that I had actually
used over here, which is nothing, but it is a contextualized Q prompt.
And instead of this I can go ahead and create the new prompt over here okay.
And I will name it as Q a prompt.
Okay.
The same prompt, only nothing different.
So I'll use this Q a prompt okay.
It is same thing right.
So here I'm going to use this particular prompt.
And along with this prompt what I will do in the next step I will go ahead and create my action.
But this time my rack chain will be having this create retrieval uh, chain.
I will go ahead and create this retrieval chain inside this instead of just giving the retrieval.
Uh, I will be giving my history aware retrieval.
Comma.
The question answer chain.
See, uh, sorry, the question answer chain.
If you remember previously when we created this create retrieval chain what all inputs we specifically
give.
See over here we gave the retrieval we gave question answer chain.
But this time we are giving history of a retriever and question answer chain.
Now it's time that we can go ahead and use this rack chain.
Okay, now let's go ahead and add it and see what exactly action will basically do and what all things
we can specifically do it okay.
So first of all what I will do I will go ahead and import from long chain underscore core dot messages
import I message comma human message.
These are the list of messages that we usually append here.
I'm just going to go ahead and create my chart history.
And remember the variable name should be same like how we have defined over there right in the message
placeholder.
Then my first question will be something like question is equal to I will go ahead and write what is
okay.
What was the question that I had asked earlier?
Uh, what is self reflection?
Okay.
So from that particular page I'm just going to ask this particular question what is self reflection?
Okay.
So this basically becomes my question okay.
Now if I want to execute it.
So let's say this will be my response one.
And inside this response one I will go ahead and use the same chain.
I will write dot invoke okay.
And let me just go ahead and give my input.
The first input will be nothing, but it will be go question.
Okay.
And the second input will be nothing but chat history.
Okay.
So here we are going to basically go ahead and use chat underscore history okay.
Which will be assigned to my chat history.
Right.
Whatever chat history we are creating over here.
And this is the message placeholder that we are giving.
Right.
And here only we are going to save all our chat history.
Now along with this, what I will do since this is a list right?
I'm going to go ahead and append this with every conversation that I have.
So here I'm going to write extend and let's say here I create a list.
Here I'm just going to add my human message.
So the content will be assigned to this particular question, whatever question I'm actually giving.
And along with this I will go ahead and write I message I message because we need to append it right.
And content is equal to nothing but response.
Whatever response one we are getting.
And inside that there will be a variable which is called as answer because see, whenever we get, uh,
the output from the retriever, you'll be seeing that there is a answer key.
Okay.
So we need to append this in the chat history over here okay.
So once it gets appended or extended, expanded basically means it will just get added at the last.
Okay.
Now similarly once I extend execute this I will go ahead and write my question to the question.
Two will be that how or what are the common ways of achieving it.
Okay, I'll just go ahead and write it.
Or let's say I'll just go ahead and ask a question.
Tell me more about it.
Okay.
Something like this.
So this will be my second question that I'm going to go ahead and write.
Now I will go ahead and write in this question dot invoke.
And here the same thing I'm going to pass right with respect to this history, I'll get my response
to.
So this will basically be my response to.
And now I will go ahead and print my response to.
Response to off my answer.
Okay.
Now I think it should be able to understand the context and it should be able to give the answer.
Let's see.
So self mechanism.
Self reflection is a mechanism that allows autonomous agents to improve iteratively by refining past
action decisions and correcting previous mistakes.
He initially asked what is self reflection?
Then I told tell me more about this and this is what we are basically printing.
Okay, if I go ahead and see with respect to the chat history, you'll also be able to see that first
one is this particular message that we got right.
Errors are inevitable.
Okay.
And second time when I asked tell me more about it, it was able to understand the context and it was
able to give us the answer.
So, uh, I hope, uh, you're able to understand this amazing thing.
Okay.
Now, uh, there are some more things that you can actually do with, uh, whenever you are working
with LCL.
Right.
Uh, that is adding, uh, chat history with respect to session IDs and all that we have already discussed.
So let me quickly copy and paste some of the code over here so that you will be able to understand.
So if I go ahead and write this, if you remember this base chat history in our runnable with message
history and chat history.
Right.
We used it.
So here you can create this particular session.
And based on the session ID you will be able to retrieve all the chat history.
The output key will be the answer.
The input key is nothing but input, right?
So here if I go ahead and execute this, here is nothing but it is your entire conversational chain.
In short, right.
Now I can go ahead and invoke anything based on my session ID.
So here I will go ahead and write conversation rack uh chain dot invoke.
What is this composition is another one question.
I can write my config.
This config is going to make sure that it will store all the information with respect to the session
ID, and then we can also get the answer right.
So here you can see our task decomposition is the same thing.
And here you can probably see the same answer okay.
Now similarly if I go ahead and use the same session like I'll go ahead and ask what are the common
ways of doing it.
So I'm actually talking about task decomposition.
The reason is why I have actually used the same session ID.
You can go ahead and see the answer.
Now it will be able to understand that yes, we are fine talking about this, uh, task decomposition.
So here you can say that according to context, most common ways of doing task decomposition includes
NN1 something new line characters.
Using a large language model, everything is basically displayed.
Let me see.
One more thing.
Uh, did I print okay?
Yes, I printed the right response.
So yes, this was all about the conversational Q&A chat bot with memory, right?
With chat history.
And I hope you are able to understand this particular question.
This is how we basically create a history aware retriever.
And you can actually use all these things.
Right?
So yeah, uh, this was it for my side.
I hope you like this particular video and uh, yeah, I will see you all in the next video.
Have a great day.
Thank you all.
Take care.
Bye bye.