Hello guys!
So in our previous video we have already discussed about the architecture of LSTM, RNN and along with
this we saw three important gates.
One is for gate gate.
Then we saw about input gate.
And this was with respect to with along with the candidate memory.
And finally we saw about output gate and we understood what forget gate was doing, input gate was doing
and output gate was doing.
Right right.
Now let's take one specific example and let's see that how this entire data is basically trained in
an LSTM RNN.
Okay.
So here uh, obviously I have one example.
So this is my text paragraph and this is my output okay.
Let's say that based on this information I need to find out whether this particular text based on this
particular text, the food is in the restaurant is good or bad.
Okay.
So I need to predict whether the food is good or bad in the restaurant.
Okay.
And these are some of the reviews that have come with for this particular restaurant.
So here let's read this text paragraph.
It says, hey, I went to the restaurant and ordered burger.
Okay, so in this paragraph, this is my first sentence.
Then here you can see the burger looked tasty and crispy.
Okay.
But burger is not good for health.
It has a lot of fats cholesterol.
But this burger was made with whey protein.
and only vegetables were used, so it was good.
Now see all this information here.
It shows that it has lot of fats and cholesterol.
And here it is saying hey, it is made up of whey protein and only vegetables.
So it was good.
And finally my output is basically one.
So this is that basically means this burger is good right.
So this is my training data.
So you can probably consider this as my training data okay.
Now with respect to training data, how do we take this particular sentence and train it in our memory.
What information needs to probably go to my memory cell or the long term memory cell and the short term
memory cell?
Okay.
Uh, see, already we have discussed about this entire representation, right?
Uh, with respect to the LSTM.
Now, I will just give you a simplistic, uh, a simplistic representation.
Now, with respect to LSTM, you know that there are two important line.
There are two important line, okay.
One is for the LSTM right.
So this is for the long term memory.
And there is also one more which is specifically for the short term memory.
Okay.
Short term memory.
You know this this short term memory how it is basically denoted by this is nothing but c t minus one.
This is nothing but c t minus one.
Let me just go ahead and write it down c t minus one.
And this is the previous hidden state HT minus one okay.
Here we give the input okay x of t.
As soon as we give the input the first important I'll, I'll just I'll put a box over here to make you
understand.
Okay.
So the first important gate is nothing but for gate.
Gate.
For gate gate.
Then the next important gate is my.
Input and or candidate gate.
Input and candidate gate.
Uh, can.
Sorry.
Input gate and candidate memory okay.
And finally third that you have is something called as output gate right.
So I'm not writing what operation is basically happening over here.
Right.
But I'll add this one.
So here you have this pointwise multiplication.
Here you have this pointwise addition okay.
Okay.
And considering this I will just connect this I will just connect this.
And finally you'll be able to see that I will be getting back to my output gate and coming back over
here.
So this is my CT.
Okay.
Now don't worry that I have missed some of the parts.
If you really want to see the complete diagram, it looks something like this.
But I've just combined this into some blocks and here I've actually shown it Okay.
And this is nothing, but this is my next state of HD.
Now, the reason why I'm showing you this, right?
Because I need to pass all these words with respect to time to this particular gate.
Okay.
So how does the training specifically happen?
See, when I have this text paragraph, you know that every word over here, every word, the first
step is that it needs to be converted into vectors.
It needs to be converted into vectors.
Okay.
First step we basically talk about it.
And this layer also we say it as embedding layer.
That basically means whenever I take a word first of all I need to convert this into vectors.
Some vectors Okay, now already we have discussed about word two vec okay.
So in word two vec what do we do?
See word two vec.
If I probably consider an example of word two vec as a vector okay.
In word two vec.
Let's consider that I am converting this word into three dimensions three dimension vector three dimension
vector.
Now, since I have these words, let's say see word two vec.
How does it work?
I hope you may be seeing you have already seen the previous videos of word two vec.
So here what we do is that with respect to word two vec we we take one input.
Let's say we have one word over here.
And with respect to this particular word, if I want to probably convert this into a vector okay.
If I really want to convert this into a vector, then, um, let's say the one of the word is, uh,
tasty.
Okay.
Now in order to convert this word into a vector.
So with the help of word two vec, what it does is that it takes some features one, two, three, like
we are just using three dimension.
So it will try to find out the relationship of this word with this particular feature.
Okay.
Now what feature it is, how it basically does it.
I have already shown you how the entire word two vec basically training happens.
Okay.
But you can just consider see it's mostly like a black box.
This is completely a black box.
You will not be able to understand or get to know what is basically happening internally, but it will
just try to select three words.
And based on this relation it will assign some vectors.
Right now most of my text is related to food.
It is related to good bad.
Something like this.
Right.
So I will just create three vectors okay.
Over here one will be finding the relationship with respect to good.
Then one will be finding the relationship with respect to bad and one will be finding the relationship
with respect to a healthy.
Okay, so let's say I have selected this three.
And based on this three, we will try to take each and every word and we'll try to find out one three
dimension.
Let's say as soon as I read the burger looks tasty.
So here when we say tasty I will be getting a vector for good somewhere as 0.9.
Um, for bad since it is not bad right now, so it will be 0.0 for healthy.
Still no information is given.
So let's say that this is just point one.
When we are saying tasty, it may be somewhat healthy.
So.
So as you all know in LSTM RNN what we do we keep on giving every input over here, right?
One input at a time.
Um, and let's say for tasty, this is my vectors that I'm specifically giving.
Right.
This is the vector that I'm giving and I'm considering.
Hey it is three dimension.
So similarly for every word what we do, we give this kind of dimension from here.
Right.
But at with respect to timestamp, first of all we will go ahead and let's say, uh, I go ahead and
say is that I went to Restaurant and order burger.
So every three, three, three dimensions will probably be going on over here.
Now with the help of forget get what will get, what will basically happen.
We need to find out like what information needs to be passed through this memory cell.
Right.
So here we have a memory cell.
So here you can see I went to the restaurant and ordered burger okay.
And ordered burger.
So once I give this entire information right now I know I don't want to skip anything from the previous
memory because nothing new has come.
But when I say the burger looked tasty and crispy, I know because this tasty and crispy is going to
be used somewhere.
So what I will do when these words come from this input candidate kit, right input gate and candidate
memory.
We will try to add this information like tasty and crispy.
We will add in the previous memory cell over here, because this tasty and crispy will be getting used.
Now when this tasty and crispy will be coming, right.
You'll be seeing that once I add in this input a candidate.
Get this.
All vectors will have more better values.
Let's say over here it will be 0.9 here.
Uh bad about bad.
It has not been told anything but about healthy.
Uh, is not good for health.
It is basically, uh, the next sentence, it is actually coming up.
Right.
So right now, if I say the burger look tasty and crispy, so it will be just point one with respect
to health.
So this at this point of time, this information is going over here.
And this will further get added to the memory because it is talking about tasty and crispy.
Now when the next line goes that is.
But the burger is not good for health.
So when this comes, since it is saying not good for health, right?
Not good for health.
Okay, fine.
This information will probably go now.
The vector is going okay, but here you will be able to see that after this particular operation.
Since we are talking about not good for health right now, my vectors will get changed to something
like this.
Now we are talking.
See?
Good is already there.
Point eight.
Okay, let's say the good will reduce a little bit, but bad if I talk about it will become point four
because bad is increasing.
Okay.
It is saying not good for health right.
Not good for health.
So if I say not good for health again this will become point zero right.
So here bad value is also increasing.
Good value has decreased right.
If I say good value may have decreased to point seven.
Point seven.
Okay.
Because here we are trying to find out the relationship of every word.
So if I say health with good bad and healthy right.
If I try to find out this relation.
So this value will keep on decreasing.
Right.
Then it says it has a lot of fats and cholesterol.
So when we go to this sentence and then we pass the word now what will happen?
This good value will still keep on decreasing because we are saying, hey, it has a lot of fats and
cholesterol.
So this becomes bad.
Bad value will increase.
Now the this healthy value, healthy value will also like increase or decrease, right.
Initially let's say uh over here when we say it is not good for health, it is completely zero.
Right.
And here we have said, okay, it has a lot of fats and cholesterol.
So it will be zero right?
Initially it can be one.
But this burger was made with whey protein and only vegetables.
When this sentence comes what will happen.
This vector will further change into something like let's say uh now we are talking about more good
things, right?
So this will increase, the bad part will decrease, and the good and the healthy part may become 0.9
because we are seeing that, hey, it is made up of whey protein and only vegetables, right?
So this vector will keep on changing and that is how word two vec work.
Because we will try to find out the relationship between this word to this particular features given
by word two vec okay.
And then we will keep on giving that particular vectors.
Now when this value changes we now know what information we need to remove from the previous state and
what new information needs to be added.
Because see, this tasty and crispy is again going to come somewhere.
So we are just going to add some information about tasty and crispy over here okay.
Let's say burger is not good for health.
And later on we don't get anything about good for health or bad for health.
Something like that.
Right?
Anything I will not be having a reference of not good or not bad over here.
Right?
So I may remove some of the information over there.
So when, when see over here you will be able to see that I am reducing some value.
I'm increasing some value.
Right.
How this is actually happening.
This is actually happening based on the sentence.
And this is only why when we reduce something that basically means we are removing something from the
memory, maybe, or we are increasing something in the memory itself.
Right.
So this is for your short term.
This is for your long term.
Right.
And this is how your entire forward propagation and the backward propagation will keep on happening
with updation of the weights.
Okay.
So I hope you are able to make sense.
Like how each and every vector changes and how your forget gate, input gate and output gate will keep
on working with the help of forget gate we will just be doing one work.
Is that what information I need to let through this particular memory and what information I need to
forget?
Make it forget.
Okay, now as a training will happen initially when I say, uh, the burger look tasty and crispy,
this entire information will go over here.
Since this is a new information right from the previous information, I may remove something.
Let's say over here also, we have a three dimensional vector, let's say about good bad or healthy.
So good was 0.0, bad was 0.2.
Healthy was let's say 0.1 previous cell state.
Now the burger looked tasty and crispy.
When this information is going now input gate what it is going to do.
It is going to convert this entire vector where the good value will increase, okay.
It will increase to 0.4.
Let's say I'm just taking it as an example.
When I talk about bad this will decrease okay.
And when I'm talking about healthy right.
This may have some value like 0.4.
Right.
Then when I pass this three this third sentence I need to add some more information.
I need to skip some more information.
Right.
So from this entire cell, what I will do when we go to the next timestamp, I may I may decrease this
value now.
Right.
Because now it is saying not good and not uh, not good for health.
So what will happen?
We will add some information.
We'll forget some information.
So if you're forgetting some information now we are saying that hey, this is not good.
The burger is not good for health.
So this value will become 0.2.
Similarly, this second one is specifically for bad.
This will increase to 0.4.
And if I say healthy since I have written previous point for but it says it is not good for health,
this may become 0.1.
Okay, so this how the entire, uh, the forget gate, the input gate, candidate gate, and the output
gate actually works.
But you really need to remember forget gate will help you forget some of the information based on the
context, switching only the important information that will be relevant for the long term context that
will be used and it will keep on going out.
Right.
So I hope you got this complete idea.
This was a quick, uh, way of making your understanding training data with LSTM, RNN.
Um, I tried my level best, uh, to explain you all this things.
Okay, so.
Yes, uh, this was it.
I will see you all in the next video.
Thank.