Hello guys.
So we are going to continue the discussion with respect to creating the chatbot.
And uh, already in our previous video we have discussed about Prompt template with, uh, you know,
the conversation history.
Now we are going to focus on understanding how do we manage the conversation history.
See, one important concept to understand when building chatbot is how to manage conversation history.
If left unmanaged, the list of messages will grow unbounded and potentially overflow the context window
of the LM.
Therefore, it is important to add a step that limits the size of the messages that you are passing
it.
Okay, so let me just go ahead and show you how it can basically be done.
So here, uh, I will probably go ahead and write or import one important library.
Let me say from lang chain underscore core, I'm going to write dot messages.
And here we are going to import system message.
So system message.
Along with that we are going to also use one more method which is called as Trim messages.
Now what exactly is this trim message.
Okay so here I will just go ahead and write over here.
Okay.
So trim message trim underscore messages.
This will help.
This is a helper which will help reduce how many messages we are, uh, sending it to the model.
Okay, so let me write some brief information over here.
So this helper read, uh, helper to reduce how many messages we are sending to the model.
The trimmer allows us to specify how many tokens we want to keep along with the other parameters.
Like if you want to always keep on the system, keep the system message and whether to allow partial
message or not.
Okay, so this is what specifically we do with the help of trim underscore message.
And this trim underscore messages we have probably imported over here okay.
Now quickly I will go ahead and initialize this trimmer.
And I'll go ahead and write trim underscore message.
Inside this there are some tokens.
We will some parameters that will be uh will be playing with you know, so the first token over here
is nothing but max token.
Let's say I want to keep the max tokens to 70.
I want to limit it to 70.
Okay.
Then I will ask for a strategy.
We will go ahead and say, hey, what strategy do you need to follow for trimming it?
Right.
And there are multiple strategies.
One of the strategy over here that we are going to use.
Last, if I say last, it is just going to focus on the last conversation that it had.
You know, from there it is going to count the entire tokens.
Okay.
Then uh, the token there is also one more parameter which is called as token counter.
See see max token is max token count of trim message.
Token counter basically means function or lm for counting tokens.
Right.
So here, uh, if I go ahead and write token counter, I'm going to use my model which will be used
for uh, counting the tokens.
Right.
Then I will say include system.
So there are more parameters over here.
If you see over here there is also maybe one more parameter not able to see it over here.
And this is one example.
It is given.
Right.
I will discuss about it okay I will I'll give you one very good example over here.
So here what we'll do here also we will go ahead and set our include system is equal to true.
I'm saying hey include the system message because system message is important.
We cannot remember.
We cannot because that is where we say what exactly the LM model should do.
Okay, then I'll say allow partial is equal to false.
So I don't want partial information, I just want to trim.
That's it.
And here I will go ahead and say start on from where it needs to start.
It should not start from the human conversation okay.
Perfect.
So this is the trimmer that I'm actually going to use okay.
Now let me do let me do one thing.
Let me keep a set of messages Okay, so here are my set of messages.
Um, so here you can see system messages, content.
Your good assistant.
Then human message says, hey, I am Bob.
Then a message says hi.
Human message says I like vanilla ice cream.
I message says nice.
This is the response from the LM model.
Then human message is like what is two, two plus two I message says four.
Human message says thanks.
So this is a set of conversation that has happened between human and AI.
Okay.
So now if I just use this set of, uh, image, uh, conversation or messages, and if I use streamer
dot invoke and I pass it over here, now you see the magic?
What it will happen if I execute it.
Okay.
Uh, trim underscore message allow partials.
Okay.
Allow partial should not be there or what?
So here, uh, you will be able to see instead of allow partials, it should be partial.
Okay.
Now, if I go ahead and execute it here, you'll be able to see that it will apply all the trimming
on this with the help of that same model.
Now you can see system message has been considered and here you can see human message.
Hi I am Bob so hi I am Bob.
Over here you can see hi I like vanilla ice cream.
So everything is basically taken.
The reason is I've taken all the 65 tokens.
Okay let me sorry this if I probably go ahead and count it, the total number of tokens may be 65 or
70 over there.
So for this I will just go ahead and use 45 tokens.
Now let's see whether it will change.
So here you can see you are a good assistant.
And it has started now from I like vanilla ice cream.
See I like vanilla ice cream.
Right.
So this information you can see from here the top two has been trimmed off.
Right.
Not top two.
The top two human and I message conversation has been trimmed off.
Okay.
And remaining all you can see that it is you are able to see it.
And this is how you specifically manage the entire information.
We be we don't need to every time pass the entire, uh, context.
The reason is very simple because there is a limitation with respect to every context window in our
LM models.
Okay.
Now let's do one thing.
Uh, we can also go ahead and use chains.
Right.
And, uh, see, uh, right now we are just passing all the list of messages and at all the time we
need to probably go ahead and pass this, uh, trimmer.
Right.
So in a chain, how do we pass it?
Okay, so here I will go ahead and, uh, like, if I'm creating a chain, how do I pass this particular
trimmer function that we are going to see?
So here I will go ahead and write from operator import item getter okay.
And I'll tell you what exactly we do with this item later.
First of all, I will go ahead and create my I'll import my runnable library.
So here I will go ahead and say from long chain underscore core dot runnables okay.
And from this runnables I'm going to import the runnable pass through okay.
So it will be runnable pass through okay.
Now once I have this runnable pass through I think it should be imported.
Lang chain.
Okay.
Now see this?
Whenever I create my chain.
Okay, I know that first when I'm creating this chain.
So now if I really want to apply the trimmer function, first of all, the trimmer function in my list
of messages, I can actually use this runnable pass through.
And I'll say and this I will be providing in my chain itself.
I'll say dot assign.
And inside this I will go ahead and write my messages is equal to item getter okay.
And here the item getter that we are going to basically say is messages.
By using this item getter of messages, we will be able to retrieve the entire messages that is available
in the prompt template.
And along with this we will go ahead and apply primer.
Okay.
After applying this, uh, one step by step, I have applied over here, right?
The next important thing will be that along with this, I will go ahead and concatenate the chain with
prompt.
Okay.
Then I will go ahead and concatenate.
And this is basically using LCL right.
I will go ahead with model okay.
So this basically becomes my chain.
Now if I go ahead and say chain dot response.
And here uh sorry chain dot invoke.
We have to do invoke okay.
And here we will be passing the parameters.
The first parameter that we are going to pass is nothing but messages.
And this will basically go with all my messages that I have.
Okay.
So all the messages over here you can see right.
This is the messages that I have actually created okay.
And along with this messages I will pass it over here.
I will say hey let's go ahead and add my human conversation with this.
So it will be my human message.
I'll say, hey, content, what is my or what ice cream do I like?
Okay, I'll just go ahead and ask what ice cream do I like?
So this will basically be my content.
I should not give this in this one.
So this will basically be my content okay.
Content over here and this will be my first information.
And again we need to give this entirely, uh, this human message.
I need to give it in the form of list.
So let me just go ahead and close this.
Now, the second parameter here, what I will be doing is that I will just go ahead and write language
is equal to English because in the prompt template we have used two parameters.
Right.
So finally I will store this in my response.
And here you will be able to see that I will go ahead and display response dot text or sorry response
dot content.
So once I execute it, uh, I'm getting an error saying that messages over here.
Okay.
This should be in the form of key value pairs.
Okay.
So so guys now let's go ahead and execute this.
So once I go ahead and execute you can see as an I don't have access to a personal preference.
The reason is very simple why we are not able to find out what ice cream do I like.
right?
The reason is very simple because the trimmer, which is specifically applying this 45 tokens, right
Max tokens when it is applying the trimming on this particular messages, uh, that context is not there.
Right?
So that is the reason you can see over here that I'm not getting that particular information.
So it is saying I don't like your favorite ice cream, uh, over here.
And I think, uh, uh, I think this one, I like vanilla ice cream has also gone okay.
Uh, since we have changed the context.
Maximum number of tokens to 45.
Okay.
But let's see for this whether this will be able to remember this.
What is two plus two.
So here what I am actually going to quickly do is that I will go ahead and invoke one more chain.
And this time I will just go ahead and ask what math problem did I ask for.
So if I go ahead and execute it here, you can see you asked for what is two plus two?
Okay.
So this way you are able to get all the informations and you are able to see it.
Okay.
Um, again uh we can go ahead and again still work with.
Now let's wrap this thing this in the message history okay.
We will wrap it message history.
Now in order to wrap it again, what I really need to do is that I will go ahead and create my config
with messages tree.
So I will go ahead and write with messages tree runnable, uh, message tree chain get sessions, uh,
history.
And here you'll be able to see all this information.
Okay.
Um, now I will just go ahead and set up my config.
So my config will be nothing, but it will be configurable.
This is one of the key that is used.
And I will go ahead and write my session underscore ID, which will be nothing but chat five okay,
so let's say this will be my chat file.
So here I'll just go ahead and execute it okay.
Now based on this particular context, uh, let me go ahead and have some of the conversation okay.
And here I will go ahead and execute what's my name?
Uh, obviously I'm not going to get anything right.
So human message is this.
And I'm calling this particular configuration.
And if I even go ahead and ask the math problem right.
What math problem did I ask again it'll be showing me that as a large model, I have no memory of the
past conversation.
So, uh, this was it.
You know, I hope you were able to understand this entire thing.
And this is important.
Whenever you really want to work with the chat conversation history.
Right.
So what are the things we did in all this, uh, three, four videos, right.
We played with messages.
We played with tremor, we played with managing the conversation history with prompt template how you
can actually work with the entire, uh, conversation history.
We have also discussed that and all these things has been discussed.
And these are some of the very important components that will be required whenever you're building your
chat bot.
So yes, this was it from my side.
I hope you like this particular video.
I'll see you all in the next video.
Thank you.