Hello guys.
So we are going to continue the discussion with respect to LSTM RNN.
Uh, now let's go ahead and understand about the entire architecture initially.
Then we will break down this into topic by topic and try to understand that how does an LSTM RNN work?
Okay, so the basic architecture representation which we saw in the previous video was this one.
Okay.
And uh here you could see that okay I have this x of T x of T is nothing, but it is the inputs which
is passed with respect to time.
Like t is equal to one.
One word is passed, t is equal to two other word is passed.
Uh, if I probably expand this architecture.
So we go to this particular next stage.
The entire LSTM RNN has this three important gates okay.
One is the forget gate.
Okay.
Now if you tell me which part is basically the forget gate, I will probably say from here to here.
So if I divide this into three layers, let's say this is my first layer.
Okay.
I'll not say layer but first part.
Okay.
So this entire part is all about forget gate.
Then the next important gates that I have is something called as input gate and candidate memory.
Okay.
So this is my second part.
And the third part is basically output gate.
So a single hidden neuron right a single hidden layer neurons with respect to our LSTM, RNN has this
three important gates.
One is the forget gate.
Then you have input and candidate gate uh candidate memory.
And then you have this output gate okay.
Now let's go ahead and discuss more about this.
So here uh let's discuss about this entire parameters okay.
So here in the first instance, I am giving my, uh, input over here x of T again with respect to timestamp,
I will be giving this t of minus one is the hidden state of the previous, uh, hidden neuron, right?
Uh, when I say previous hidden neuron, let's consider this as my neuron.
Right.
So, you know, with respect to neuron, uh, in our case, right.
A simple RNN provides one feedback loop, right?
But in the case of LSTM RNN, we will also have another feedback loop which will act as a long term
memory.
This is basically a short term memory, short term memory.
And this is nothing.
But this is a long term memory.
Long term memory.
So when I say HT minus one, this is nothing, but it is the hidden state with respect to short term
memory of the previous previous timestamp.
Okay.
So here you can see I'm giving input x.
Then I'm giving HT minus one HT of minus one.
That is nothing but the previous hidden state.
Then these are both combined.
Then we pass it to the sigmoid activation function.
And all this kind of operations will happen.
Okay, we'll discuss one by one and what exactly it is.
Okay.
Now, before this, uh, uh, with respect to any diagrams that we see, right, there are some notation
that is specifically used over here.
So let's consider this okay.
This is basically a neural network I will talk about.
See whenever you find out something like this here sigmoid is there here tanh is there here tan.
Uh here sigmoid is there.
This is basically called as a neural network operator layer.
Okay.
So this is specifically called as a neural network layer.
What does the neural network layer basically mean?
Nothing.
Uh, here you can just consider that it is one of my hidden neuron or hidden layer neuron, which has
some neurons over here.
And in all these neurons we have applied a sigmoid activation function.
So I have just applied a sigmoid activation function.
And here we are getting giving the input okay.
So if I probably consider this as an example.
So here I'm taking the input x of t.
And there is also one more input ht minus one.
We are giving both of them.
First of all we are combining this and then we are giving it okay.
So if I probably want to write this particular diagram in a better way so I can say, hey, uh, this
is my HT minus one, this is my X of T, I will take this input.
I'll combine it together and then I will send it to my.
I'll send it to my hidden neuron.
I'll send it to my hidden layer which will specifically have this input nodes, uh, this hidden nodes.
And on top of that I'm applying an activation function.
This is what it looks like if I probably expand this.
So over here, this entire thing, when we say we have this kind of notation, right, this is nothing,
but this is this notation.
It is basically a neural network neural network layer.
We will discuss more about it as I keep on expanding.
And once we go ahead and understand each and every gates, we will be able to understand.
Then the second one is that you'll be able to see this pink color operations, the pink color operations
that you have over here.
Let's take that particular color.
I will try to find the pink color over here okay.
So light pink okay.
Dark purple.
Pink.
Pink pink pink okay.
Let's go ahead and do this.
So let's say there is one operation which is like this.
This is nothing.
But this is called as a point wise multiplication operation.
And if I have this kind of operation, this is nothing but point wise addition operation.
Okay.
And one more that you will be able to see that something called as tan H.
So this is nothing but point wise tan H operation.
Now what does this basically mean?
Let's say if I have two vectors my one vector is one, two, three and my another vector is four, five,
six.
Let's consider that this is my x of T, or this is my t of minus one.
Or let.
Let's just consider this v one and v two vectors.
If I say point wise multiplication, that basically means we are just going to take each and every number
position wise, and we are just going to multiply it.
This is basically called as point wise multiplication.
So if I say point wise multiplication, if I perform this this uh, here I am specifically going to
get, let's say if I probably say this, I'm basically going to get one vector, which will be a multiplication
of one multiplied by four, two multiplied by five, and six multiplied by three.
Okay, so this is nothing, but this is a 18.
If I talk about point wise addition, that basically means my output will be nothing but four plus one,
five, five plus two, seven, six plus three nine.
And if I probably apply tan H as my point wise operation.
So here you'll be able to see I will just go ahead and, you know, we'll take one vector, let's say
for this particular vector I'm applying tan h.
So in short we are doing tan h of one.
Then we have tan h of two.
And then we will go ahead and calculate an h of three.
So this is what we are specifically doing.
And this is nothing.
But it is basically called as a point wise operation.
Okay.
So I hope you got an idea with respect to point wise operation, any arrows that you will be seeing
over here, this is nothing.
But this is called as vector transfer.
We are just transferring the vector.
So let's say in this line we are just transferring the vector from one state to the other state okay.
Then similarly you'll be able to see concatenate concatenate.
What does concatenate mean.
Uh so let's let's take this as an example okay.
So here you can see I, I have, I have this scenario where h t minus one and x of t is there.
And these are basically getting concatenated.
Concatenated is like combining combining two vectors, combining two vectors.
It's not like adding two vectors, but I'm trying to combine let's say my h t minus one.
It is represented by a three dimensional vector like this 123 and my x of T is a three dimensional vector
two, three, four.
Now if I say I'm combining, that basically means see, this is what, in short, is basically happening
when I'm combining, uh, in short, what I am doing, I will just combine this three vectors like this.
So this will be my HT minus one.
Okay.
Along with this I will go ahead and use my another vector x of T, right.
And then I will send this as an input to any other neurons.
Right.
So this is what we are specifically doing.
So when we say combining we are we are combining them together.
And then we are sending it to the hidden layer.
Let's say this is my hidden layer okay.
This is what we are trying to do okay.
When we say combining two vectors right.
And that is nothing but concatenate okay.
Now similarly copy basically means we can copy one vector to the other.
Uh, like we like let's say there is one vector.
Then we can also make a duplicate copy of that duplicate copy okay.
So these are some of the operations that we will be seeing.
Uh with respect to the LSTM architecture okay.
Now let's go ahead and let's first of all discuss about some of the important points okay.
Okay.
Now the first line that you will be seeing on top of it.
Right.
This line, this line is basically called as something called as memory cell.
So here I will just go ahead and write this as memory cell.
Now what does memory cell basically means.
This is for my long term memory Whatever information I want, I will be saving inside this.
Whatever information I don't want, I'll be removing from this.
That is the main task of this memory cell.
Okay, till now, just get to know about this.
CT minus one is nothing but the memory cell of the previous of the previous, um, timestamp and CT
is nothing.
But after we are removing some context, after we are removing some context and adding some context
and adding some context after that, whatever state of this memory cell will be, it will be C of T
for that particular timestamp.
Okay.
So I hope you are able to understand this basic things.
Now it's time to understand how this forget gate works, how this input and candidate memory gate works,
how this output gate works, and how the entire LSTM RNN works.
We'll be discussing about that.
Okay, so the first gate that we are going to discuss about is something called as forget gate.
Okay.
And this is what we are going to discuss in our next video.
So we will take this as an example.
And here I will break down each and every thing.
How does I forget get work.
So in short in our next video we will be discussing about this stage.
Right?
This stage which is highlighted over here and here also, you should be able to see that I am giving
this information.
This should also be considered okay.
This entire operation should be considered and this is my CT minus one.
Okay.
As I said, HT minus one is the hidden state.
So let me just go ahead and write some of the notation.
HT minus one is the hidden state of previous timestamp.
Let's say t is equal to one.
And right now we are working in t is equal to two.
X of t is nothing but word passed as input in the current timestamp.
Current timestamp okay.
And here we are concatenating it okay.
Now let's go ahead and discuss about this forget gate in our next video.
But till here I think you got a basic idea about the entire LSTM architecture and some of the important
points which is basically mentioned in this diagram.
So yes, this was it from my side.
I'll see you in the next video.
Thank you.