Hello guys.
So we are going to continue the discussion with respect to our decoders.
Already in our previous video, we have discussed about this really important topic, which is called
as masked Multi-head attention.
And we understood how different types of masking is specifically done and what is the purpose of the
masking.
Right.
Then after this, adding a normalization will specifically happen over here.
And when I say normalization layer normalization will basically happen.
And this is the same layer normalization like how we discussed in the encoder.
Right.
So from here you'll be able to see a residual connection is basically there which is also called as
skip connection.
So this skip connection information is also going right.
So I will also go ahead and say skip connection okay.
And with the help of this what will happen is that the uh it will get added to the output vectors.
That is probably coming from here.
Right.
And how did we calculate the output vector step by step?
Each and every thing is probably shown in the previous video.
Right?
Uh, we went till the softmax and we probably found out the softmax scores and these were the scores
with respect to the vector that I had right now.
What we are going to discuss about in this particular video is about encoder and decoder multi-head
attention.
Okay.
So this multi-head attention is also called as encoder and decoder.
Uh, why it is specifically called as encoder and decoder.
We'll just discuss about it.
And uh, you'll be able to see that two of the connection is specifically coming from the encoder output.
Right.
So here from the encoder output, when I say encoder output from here we are specifically taking the
vectors of key and values.
Okay.
So we are going to probably take the vectors of key and value that is coming outside this particular
encoder.
And we are going to give it to the decoder Multi-head attention.
So from here you will be able to see from the adding and normalization layer what information will be
going from here.
Whatever query vector is basically generated.
Right.
That will specifically be going.
Now you may be thinking why query from here and why key and values from the encoder output okay.
The reason is very very simple.
So here you'll be able to see from the encoder right from the encoder output.
Right.
What we are specifically sending.
We are sending a set of attention vectors.
Attention vector k and v.
Right.
And other than this, you'll be able to see when the output from this right masked Multi-head.
Attention.
So the output from masked multi-head attention.
Here we are sending.
Attention.
Vector.
Q, which is also called as query vector.
Now this query vector is the information that we are specifically doing over here.
Right?
By shifting the outputs to the right, then we are getting the output embedding.
Then we are probably getting the positional encoding okay.
And there is one very important thing.
Also I really want to discuss based on your input and output.
And just I'll discuss it in another five minutes.
Okay.
And why we are specifically doing it is very much simple because you can understand that these keys
and values, these are to be used by each decoder in its.
Encoder.
Decoder.
Attention layer.
And why specifically we are doing this?
Because this helps this entirely helps the decoder two focus on appropriate, appropriate places in
the input sequence.
So during the training time, I know here we will be giving our training output, right?
Whatever will be my Y and we need to predict y hat.
Right.
So over here, with respect to what will be the query of Y with respect to this key and values, that
is coming from the encoder output, that both will be given once a set of three vectors of k, v and
query, which is coming from this masked multi-head attention, it will be given to the multi-head attention,
which is also called as encoder decoder Multi-head attention.
And based on this, whatever is the necessary operation that specifically takes in multi-head attention,
right?
So in the multi-head attention, what will happen?
Self-attention will specifically happen inside that.
Then you go ahead and multiply.
K uh Q uh, the query with keys, then you do, uh, then you apply a softmax, then you multiply with
the values.
Then you probably calculate the scores that exactly we have discussed about each and everything one.
When we are discussed in the previous session, right where we have discussed about all the numericals,
what is going to happen, right?
Same thing is over here that is specifically going to happen.
The K and B are going to come from the encoder output, and query is basically going to go from here.
Right?
Then all the necessary operation will take place while the training.
And then after this again adding a normalization will basically happen.
And again here also you will be able to see whatever output I am getting over here, right.
With respect to the queries that will be also forwarded to the next, uh, layer that is called as adding
a normalization.
This is just to make sure that more added information is gone to the next layer.
Right.
So by this you will be able to understand that what Multi-head attention will specifically do.
And the major thing that is important for you all to understand that for this multi-head attention from
where the encoder output is probably coming up, what is the encoder output?
That is, the set of key value vectors.
And the query vectors will probably come from here.
Right.
And why do we do this.
See in at the end of the day when you have a data set right.
So when you have a data set in this data set you have some input and output right in input.
Let's say you have X1X2X3.
In the output you have Y1Y2Y3.
Right.
So while training I have to pass X1X2X3 over here.
Right.
So when we have this inputs we need to pass the inputs from here.
All the process will basically happen.
And with respect to that, while we are training this entire Transformers with respect to our input
and output, let's say the example that I'm actually going to take is about machine translation.
So here also we need to pass the output which will be my y right y.
Again output encoding will happen.
Positional encoding will happen.
Then after that, masked multi-head attention will happen, right?
We have discussed about what masked multi-head attention is all about, right?
But when we come over here, we need to make sure that based on this particular vectors, we need to
map the key and value vectors for this.
So the training mechanism will definitely know that how this query can be matched with this k and v
so that we will be able to, you know, focus on the appropriate places in the input sequence to probably
predict the next word.
Right.
And this is really, really much important to for you all to understand.
Okay.
Then after that you have this feed forward neural network and adding a normalization.
This is the same thing that what we discussed in the encoder output.
Now let me do one thing.
Let me go ahead and show you one of the most amazing diagram.
And by this you will get a clear idea how the entire training basically happens.
So again this is the block from um Jalandhar.
Uh, and beautifully Transformers has been explained over here.
What I have actually done is that I have, uh, put up more stress on the research paper to explain
you each and everything that is going behind the hood.
Okay.
So see over here.
So decoder side, what it happens is that we know the components of decoder work as well okay.
The encoder starts by processing the input sequence.
So from here it all the input sequence is basically start.
The output of the top loader is transformed into a set of attention vectors k and b.
So from here I will be getting this k encoder and b encoder right.
Then they are used by each decoder in its encoder and decoder attention layer.
Okay.
So that is where we are specifically using them inside our decoder where in the encoder and decoder
attention layer, which is nothing but encoder and decoder multi-head attention, which helps the decoder
focus on the appropriate places in the input sequence.
Okay, now let's see this okay at timestamp one okay, we need to pass this entire information.
See at timestamp one.
This is the word j e.
It goes to the encoder.
I mean all these three words je suis and into and then this vector is basically created.
Then from this we derive our key and value encoders.
Right.
And this is passed to our decoder.
For every decoder we specifically need to pass.
And initially we don't pass anything for our training data set.
Understand one thing okay.
So this is really important to understand.
See guys initially we don't pass any training data set okay.
From the second instance.
Once we generate this then we will be passing our training data set okay.
While the training phase I've also seen in some of the research paper where they say that, hey, the
first word needs to be passed.
And based on that, some prediction needs to be done.
But uh, again, in some research paper that I was seeing and in some of the blogs that I was seeing
here, a start token will specifically go, okay, so let me do one thing.
Let me open my epi pen so that I will be able to explain you here also.
Okay.
And you'll be getting a clear idea like how things basically happen.
But this diagram will actually showcase each and everything.
So here I have got my encoder.
Okay.
Now the next step.
This is my key and value encoder.
So here what will happen.
The first state start token will go okay.
Now this start token.
When it goes again this will be done with respect to embedding positional embedding everything.
It will go to the next step.
Then in our multi-head attention right.
This key value will be taken.
And whatever query vector is basically generated from here that will be given to the multi-head attention
along with this.
Right.
And similarly the next step will go once I is computed.
Now once I is computed, what will happen for timestamp two when we are doing the decoding?
Timestamp of two.
Something is going to happen in this way.
See this.
So for timestamp two see uh let's start this entirely from starting okay.
So from timestamp two what will happen I will come previous output is I.
Then it is going to go right.
Then all the steps inside the decoder will basically happen.
This will be your encoding and uh key and uh value vectors.
But with respect to every word that you are specifically getting over here, here positional embedding
will happen.
And with respect to this particular word your query vector will be created.
Okay.
Don't worry about linear and softmax.
I'll discuss more about this in the next video, but I hope you are able to understand this right.
In the first instance I will just pass a start token.
It will take this.
It will generate a query vector for that.
It will take this.
It will pass it to our Multi-head attention.
You know, what are the steps in Multi-head attention, right?
But at timestamp is equal to two.
Whatever output is basically generated over here, like I that will get passed over here like previous
output will be this.
Right.
And then again for this vector is basically created again Q uh key and value vector will be created.
Then we go to the next step where our next word will get predicted.
Similarly we'll go and take this previous word give it to the input encoder decoder.
Then again the third word will go and give it to the decoder.
And finally we'll get the output over here.
So this sentence that is probably given over here, it will give getting converted in this specific
way.
Right.
So I hope uh, you got an idea of this.
Right.
So let me just go ahead and explain it once again, okay.
So here let's say this is my encoder right.
You know what is there in the encoder right.
So in the encoder I pass all my words after doing the positional embedding.
Let's say this is my output.
From this we take all our key and value vectors.
We give it to our decoder in the decoder.
For the first phase in the training we give a start token.
Okay, then what will happen?
It will go to that masked multi-head attention.
Then it will go to the multi-head attention, which is also called as encoder decoder for this.
Okay.
And we will be going from here and query will be going from here.
Right.
Then we go to the next step.
So I've just drawn a rough diagram but I hope you're able to understand it over here.
Right.
So I will just close this again and let's do one thing.
Let's quickly take this image okay.
So I will copy this image and I will paste it over here okay.
So please make sure that you have this entire steps recorded okay.
So what we do in the first decoding timestamp we give all the words.
We generate all these things right?
We don't give anything from here.
Instead, what we'll do, we'll do some kind of padding.
We'll create a start token right initially.
And then we'll go and generate this output I right then I will be sent with respect to the previous
input.
Then we'll go ahead and compute the m.
So this will be m again M will be sent in my third timestamp right.
Similarly all the further timestamp things will basically be happening.
Okay.
So over here you can see at Time Stamp two whatever output is basically generated at time stamp one
I that is sent as a previous output.
Okay, so I hope you got an idea with respect to the multi-head attention.
And you know what happens inside Multi-head attention.
I don't need to explain that because already we have discussed in this this is the entire training mechanism.
Try to understand in this specific way training mechanism, okay.
The training mechanism initially what is going to go over here?
Then once we generate this particular output, this same output will be going as an input to the decoder
for the next time stamp.
Right.
So like that we go ahead and continue it.
So I hope you got an idea with respect to encoder and decoder multi-head attention.
Now it's time that we understand the most important thing that how do we calculate our output probabilities.
And that is what we are going to discuss in our next video.
So I hope you like this particular video.
I will see you all in the next video.
Thank you.