Hello guys.
So we are going to continue the discussion with respect to Transformers.
And in this video we are going to discuss about decoders.
Uh, already the plan of action that is probably given in the previous video.
We are going to discuss about this three main topics that is masked multi-head self-attention Multi-head
attention, which is also called as encoder decoder attention.
Why do we say it as encoder decoder attention will try to understand.
Then we'll also be understanding about the feed forward neural network.
But now if you see from the architecture here, you have your encoder, here you have your decoder.
The differences between encoder and decoder is that instead of just having multi-head attention, here,
you have masked multi-head attention.
So in this video, we will try to understand the importance of this masked multi-head attention.
Okay.
And we'll also be seeing that how this entire transformer is also get trained during the training time.
Okay.
So let's go ahead.
And as I said uh in the masked Multi-head attention, if you really want to understand, uh, we will
be going completely from the basics.
Okay.
Uh, like how we understood with respect to the encoders.
Okay.
So here you can see that I have a data set.
So this is my data set.
Now inside this data set let's say I have uh task which I really need to convert my English text into
Hindi text.
Okay.
So I want to do a language translation from English to Hindi.
So let's say there are three words like this in English.
And for that you have two words with respect to Hindi.
Now let's just consider one input data okay?
And with the help of one input data we'll try to understand.
So this diagram that you see this side is my encoder okay.
Just a single encoder.
And this side is my decoder.
Now, during the training time what will happen.
This parameters that is X1X2X3 will be going from here right.
Then we have our vectors.
This vectors will get combined with positional vectors right.
Sorry positional encodings.
Once it is done then we get this entire input embedding you here you can see right.
So let me just go and draw this.
So once I give my inputs over here, you will be able to see that I will be able to get this specific
vectors.
And this is added along with positional encoding.
So over here with respect to the inputs I will be sending X1X2X3.
And this will be sent parallelly right in the encoder.
Similarly, once this entire operation probably takes place over here where it probably calculates your
q k v, it probably goes and find out all the attention heads like z one to z eight according to the
research paper.
Then it will do the normalization here.
It will be passing this information over here, which is my residual.
Then it will go to the feed forward neural network and whatever output I'm getting over here, this
is ready to go to this multi-head attention.
But at the same time you will be able to see that my outputs over here y1 and y2 will be going right.
And when we say output shifted right, what does this basically mean?
This is a very important word.
Now let's say guys over here here I have three words.
Here we have only two words right.
It is always a good idea that whenever this kind of difference is there, we really need to make sure
that we have a fixed set of characters, right?
So for this initially one, while we are sending the output over here, we are going to make sure that
we also make this text padded.
Right.
So let's say now my padding will become like Y1Y2.
And over here I will be using a zero padding.
Okay.
So let's say that I have used a zero padding.
Why did I use it?
Because I wanted to make all the sentences of the same size.
In this example, I've just considered three words like kind of thing.
The length is actually three, but again, it depends on the data set.
How long also you can actually make this particular sentence.
So now when I say I have basically done the output shifted to right.
That basically means I have also added zero zero padding over here.
Now with respect to this zero padding, again, at the last you will be having zero.
I will be getting this embedding again.
This will go to my positional encoding.
It will create a positional encoding, will do the summation.
And now it will send the data to the masked multi-head attention.
So if you see from this particular point right in decoder, also this input embedding and positional
embedding will occur.
But here you are going to specifically do a kind of padding.
Let's say for an example I'm saying zero padding.
Why this is done to make your sequences length equal.
Perfect.
Then here you will be able to see we are going to the masked Multi-head attention.
Now what happens inside this masked multi-head attention?
Let's talk about it.
So I'm going to create this specific diagram.
This will be.
Masked.
Multi head.
Attention okay.
Perfect.
So here you have masked multi head attention okay.
Now with respect to masked multi head attention what are the steps is that is basically going to happen.
That is nothing.
But it is the combination of all these three steps.
Okay.
Linear projection for q k, v this will get calculated.
Then you have the scaled dot product attention.
It will get calculated.
We know this right in encoders also.
Then we will be understanding about this very important topic that is mask application.
And finally we'll go ahead and find out the mask Multi-head attention.
We'll understand why this mask application is required okay.
Step by step we will go ahead and we'll try to do it okay.
So uh first of all let's get an idea about why do we specifically do this mask masking work okay.
So which is basically written masked multi head attention okay.
So before that let me go ahead and take a very simple example.
I will show you with respect to all the steps that is basically happening in the decoder.
This is also happening parallelly.
Okay, first I will be getting the information over here.
So till here I hope everybody knows, but we need to understand what is basically happening over here
parallelly when we are training it.
Okay.
And here also all your information will be going all at once.
But with respect to the output, I will be getting one by one output as I get it with respect to words
by word.
Okay, let's say if I want to get y one hat y two hat.
So like this I will be getting the output one by one.
Okay, so let's take one basic example and I'm just going to discuss about this okay.
I'm not going to discuss about this because this is already done okay.
So let's take a very good example now in the masked multi head multi-head attention.
But before that, first of all, what will happen?
Let's say that I have a sequence.
Let's say, uh, my input sequence is something like this.
My output sequence is over here in my data set.
My input sequence is something like, uh, four, five, six.
Okay.
And let's say my four, five, six.
And I'll go ahead and write seven.
Okay.
This is my just my input sequence.
I'm numbering it.
Okay.
Then my output sequence is one, two, three.
Okay, so this is my training data that I have in my input and output okay.
Input and output.
Now let's do one thing okay.
Let's see what will happen step by step.
So first step is nothing but it is called as input embedding.
In my decoder.
The first step is nothing but input embedding and positional encoding.
Now, with respect to this positional encoding, what is going to happen?
It is very simple right.
So here let's say I know that my output is specified by this three characters.
Right.
Three characters.
I'm going to probably make this into four.
How I will be making padding over here.
Let's say I'm considering the maximum length is four over here in all this particular data set, so
I will just try to make it as four.
I have done it zero padding over here.
Now with respect to every every characters okay.
Let's say that every number is represented by a four dimension vector four dimension vector.
So here now how your output embedding will look like.
See output embedding basically means this output embedding shifted to right right.
This output admitting okay so here I will just go ahead and write my output embeddings will be.
Very simple.
Let's let's consider that uh one is represented by another vector.
Let's say this is
1.1.2.3.4
Okay, then I have my next embeddings.
.5.6.7.8.
Then I have my next embedding 0.9, 1.0, 1.1, 1.2.
Okay.
Then let's say that final one is zero, right?
So zero padding basically means it will be just zero only.
So I'm just going to write it down over here.
So if I just consider this my entire vector is represented like this okay.
My entire vector or my entire sentence is represented like this.
So this basically becomes my output embedding.
Okay.
Perfect.
Now you know that uh, my output embedding should specifically be going along with positional encoding.
Okay.
Over here.
So here you can probably see positional encoding is there.
Let's consider that um positional encoding is zero.
All the values are zero.
So I will be getting the same output embedding okay.
Now in the next step what we do we go to this masked multi-head attention.
Now there are some steps that is involved in masked multi-head attention.
So the steps line by line I will be saying the first step is nothing, but we go ahead and calculate
the linear projection.
Linear projection that is nothing.
But for q k v this also we do do in encoder.
And you know the importance of this right.
Then uh, in short we are just calculating q k v okay.
So once we do that then the next step that we have is nothing but scaled dot product.
Dot product.
And with the help of this we calculate the attention right.
Attention scores.
Right.
So that is what we specifically get with respect to the scores.
And for third step that you will be able to see is something called as mask application.
Mask application okay.
So this is what we are basically going to discuss.
And here we are going to discuss about two important mask application.
One is look ahead look ahead mask look ahead mask I will write it down.
And this is like um, one more is basically we did the zero padding.
Right.
So this is nothing, but it is the padding mask.
So we will discuss about both the specific mask.
What is this mask techniques okay.
And then probably whatever uh, steps is after this.
You know that right?
We finally need to get our contextual embedding so that the same thing we will be continuing and will
try to understand it.
Okay.
So now let me go back over here.
So this is my output embedding right now in the step two.
So this is my step one.
And remember the step one is we have added the positional encoding.
My positional encoding is nothing, but everything is zeros okay.
So four cross four matrix everything is zeros right.
So here based on this same matrix I will be getting over here.
Now coming to the step two.
Now in the step two what we basically do we do the linear projection for q k and v.
Okay.
So let's go ahead and create what we are doing in this.
We are creating query q.
Key k and value v.
This vectors we will try to create.
How do you create it By multiplying with w of q, w of k and w of k.
Let's go ahead and initialize w of q is equal to w of k is equal to w of v.
We will initialize this to identity matrix.
So if you are initializing this to identity matrix.
So anything that you multiply with identity matrix.
So I will just go ahead and calculate q.
This is nothing but output embedding.
Whatever output embedding I will just initialize it.
See, at the end of the day we will be learning this specific parameters.
This will be multiplied by w of q which will be nothing, but it will be output embedding only.
Right.
Because this is an identity matrix, right.
Similarly, if I go ahead and do the k this multiplied by w of k, which will be nothing but the same
output embedding.
Similarly, I have my V output embedding multiplied by w of v, which will be equal to the same thing.
Right?
So this is how we calculate the q k v okay.
At the end of the day the q k v is nothing, but it will be the same values.
So let me just quickly copy this.
So this is my final q kv for the first iteration let's say.
So my q k v is nothing, but it is equal to the same thing.
Right.
So till here I hope everybody is clear.
So this is how step by step it happens.
And probably when I go to that multi head attention which includes self attention and all this is the
first thing that is going to happen right.
I will be able to calculate it.
Now the third step uh over here we will go ahead and calculate our scaled.
product.
Scaled dot product.
Scaled.
Dot.
Product.
Attention.
Calculation.
Okay.
So we will go ahead and compute this okay.
Now in order to compute the attention scores it is very much simple.
How do we compute the score.
I will just write a shortcut formula q we multiplied by k transpose right.
And then we divide by square root of d of k right.
In this case d of k is nothing but four right dimension of k.
That basically means it is nothing but q dot product with k of t divided by two.
Right.
We specifically do this okay.
Now let's go ahead and do the further calculation for this scaled dot product attention calculation.
And here we will be taking the same data of q k and v okay.
So now we are going to do is that we are going to go ahead and calculate our scores.
Now in order to calculate our scores uh, understand when I say q multiplied by k of t right.
What exactly it is, it is the same matrix multiplied by the transpose of K.
Right.
So this will like see if this is q okay.
And let's say this is k right.
So first of all I'm just going to probably multiply this with this point one sorry point.
So it should be something like this.
.1.2.3.4.
Right.
So this will be my k transpose.
So that is the reason what you'll do.
Point one multiply by point one uh, plus point two multiply by point two plus point three.
Multiply by point three plus point four.
Multiply by point four.
Then next one will be point five.
Multiply uh point five uh multiplied by .1.6.
Multiply by point two.
Point uh point seven.
Multiplied by point three.
Then you have point four or sorry point eight multiplied by point four.
Then again you will go ahead and calculate the third one.
This will get multiplied by this.
Okay.
Then for the second time, what you really need to do, you need to take this point five deployed by
this, uh, point five multiplied by .1.6 multiplied by .2.7. deploy by .3.8.
Multiplied by point four.
Then you what you will do.
Then again, you will take this entire number and then multiply with the second one.
right?
So here you'll be able to see I will be creating my separate key pairs.
Right.
So this will be another key transpose where I'll say 0.5.6.7.8.
Right.
So this will be my another key transpose.
Then another key transpose will be this.
So finally, when you do the specific calculation and just to give you an idea how the calculation is
given, I have probably written down each and every steps over here, but I have done the calculation
over here.
My final calculation will be nothing but.
But this is simple, right?
See my q k v is all these numbers.
When I say q transpose of k of t, I have to take.
This is my one query, right?
This is my another query.
And similarly if I probably consider this as my keys, this is my another key.
Another key, another key, another key.
So with respect to every thing, I have to probably do the matrix multiplication or dot operation.
Right.
So finally you'll be able to see that I have already done the calculation.
This will basically be my score.
The score will be nothing but point three.
Again it will be uh, four cross four, matrix point seven, 1.10.0.
And then we will further do the calculation
0.71.93.10.0.
Right.
And then I will be having 1.1, 3.15.18, 0.0 here.
Also again you'll be getting zero point.
I have to get this entire zero because you will be able to see that last one is just zero, right?
Again, in order to show you how the calculation is specifically happening.
So I will just make a matrix over here, let's say.
So this is my first matrix.
My first matrix will be 0.1 multiplied by 0.1 plus .0.2.
It is a dot operation and you should understand this point two.
Multiplied by point two plus point three.
Multiplied by point three.
Plus point four.
Multiplied by point four.
Okay.
Then coming to the next next one.
Right.
Again what I will do I will take this point one again okay.
Point one.
And here I will I will take the same query right.
Same query.
And I will multiply with this.
So here I will go ahead and write point one multiplied by point five.
Then point two multiplied by point six plus point three multiplied by point seven plus point four multiplied
by point eight.
Right.
So this is done.
Then what I will go to my next one.
Next one is nothing.
But again if I do the transpose of keys because I have to for one token, I have to basically do this
multiplication or do this dot operation for all the other tokens, just to understand that how much
weight is it is basically adding, then this will.
Third, one query of key will be 0.9, 1.0, 1.1, 1.2.
So my third one will be something like 0.1 multiplied by 0.9.
Then I have point two multiplied by 1.0.
Then I have point three multiplied by 1.1 plus point four multiplied by 1.2.
Similarly, the last one will be nothing, but this will become transpose, right?
Then it will become point one multiplied by zero.
Then you have point two multiplied by zero.
Then you have point three multiplied by zero plus point 0.4 multiplied by zero.
So this is done right.
Overall, this is just for one query with respect to all the keys.
Right.
The last key will be nothing but 0000.
Then what we do we go with the next query.
This will try to multiply with the next thing.
Then I will be getting my next matrix.
Finally when I do this summation I'm going to get this values right point three.
So this entire multiplication if you do it is point three.
If I do entire multiplication and addition, this is point seven, 1.10.4.
Then again the next one which will be in my next, uh, which will be my next one, which will be my
next one.
Like this.
This will be my next one.
And total overall you'll be able to see this is also like four cross four all the multiplication.
And finally you'll be able to get the attention scores.
Okay so this is my attention score.
Uh, where we have specifically done scaled dot product attention calculation.
This is the first step.
See inside this right after linear projection of qkv we basically do this right now comes mask application.
Now what exactly is mask application we are going to discuss about.
So let's go ahead and discuss about it.
So here I'm just going to go ahead and write my last application.
Now first of all you need to understand what is the purpose of master application okay.
And this is probably the most important topic.
And I'll try to explain you this with uh, multiple examples.
Okay.
Uh, here, first of all I will just go ahead and write.
It helps mask application helps managing.
managing the structure.
Okay, the structure of the sequence says being processed okay and ensures
the model behaves correctly during training and inferencing.
Okay, now what are the main reasons of masking?
Okay.
so exactly what exactly is masking will get to know about it.
But you can just understand that it helps to manage the structure of the sequence being processed and
ensures the model behaves correctly.
Okay, now let's talk about the reasons why do we specifically do masking?
The first reason is handling
variable length sequences with padding mask.
Okay, now see guys, as you all know during the output, right?
I told you right when I'm actually calculating the output embedding, I'm also making sure that I do
zero padding so that both the sequence length are almost equal.
Right.
So one of the reason why we specifically do mask application is that because we need to handle variable
length sequence with pad masking.
What is the purpose behind this.
Okay.
You'll understand this.
What is the purpose.
The first purpose is that to handle sequences.
Sequences of different length in batch.
Different length in batch.
Please just wait for some time.
I will explain each and every thing what I'm actually saying to ensure.
The second point is to ensure that padding tokens.
Which are.
added?
Which are added to make
sequences of uniform length do not affect the model prediction.
Do not affect the model prediction.
So guys now let's talk about the second point.
To ensure that padding tokens, which are added to make sequences of uniform length, do not affect
the model prediction.
Now let me just explain you this particular point with an example okay.
Let's say in my input data I have a sequence.
Okay.
Let's the sequence is something like one two, three.
And in my output data, let's say this is my A sequence one.
In my output I have my sequence two.
In my sequence two I have something like four five.
And I have purposely done the padding over here.
Okay.
So here zero is nothing, but it is the padding token.
Okay.
We initially need to do the padding.
Okay.
Perfect.
Now without masking see if I.
If I just give this data like this.
Okay, then in my data, since this zero padding is there, this will influence.
Please note this point.
This will influence the attention mechanism okay.
The attention mechanism.
Now you may be thinking, Chris, how right now you are just thinking, okay, one zero is actually
added.
But just imagine in a real world problem statement when we do padding, right?
Let's say I want to probably I get the maximum length of the sentence is 100 okay.
Or 100 or 200.
Right.
Let's say and let's say in one of the output I just have two words Y1Y2 okay.
In one of the output.
Right.
I have just two words y one and y two.
So for this what we will do, we will do the padding.
And for this 98 zeros will get added when 98 zeros are getting added over here.
And when we probably send this zero padding to our attention self-attention mechanism.
Don't you think because of this zero it will influence the entire attention mechanism and this will
in turn what will happen lead to lead to incorrect or biased predictions.
Okay, this is the problem.
So that is the reason.
Over here we will do some kind of masking.
And this masking will be making sure that this token zero padding tokens are ignored.
So for this, a kind of masking that we are going to do is something called as padding mask.
Okay.
So here we are specifically going to say padding mask.
This padding mask is basically going to make sure that the tokens are ignored.
The tokens are ignored okay.
Okay, so in short, if you really want to understand about the masking over here.
Masking is done with two important types, two important in two different ways.
One is something called as padding mask and the other one is something called as look ahead mask okay.
And in transformer we take the combination of both of them.
But we'll try to understand what is the importance of them okay.
So let's take the sequences I have 123450.
So this is my output in my training data set.
This is my input in my training data.
This will be sent to encoder.
This will be sent to decoder.
But we are still focusing on this right now for this since I have a zero right.
There is a padding token called as zero right.
So for this how do we apply a, uh, padding mask.
So what we do over here is very simple.
Okay.
First of all, wherever I find a sequence of words, if it is, if it is not having any padding.
Right.
So what does padding mask does is that since we need to consider all the three sequences.
So padding mask for the sequence one will be one, one, one.
I'm just taking as an example okay.
So over here 123 because you don't have any zero padding over here.
So I will be having the padding mask of 111.
What about sequence two.
So sequence two padding mask will be 110 by 110.
I will talk about it because this sequence are available here.
I just have a zero padding.
So I'm just going to mention that 110 okay.
So this is basically my padding mask for my sequence two okay.
But we will just not stop here okay.
We will not stop here.
They will.
They'll also be another type of masking which is called as lookahead masking.
So let's talk about this lookahead masking.
Then again I'll go back to this pad padding mask okay.
So the second type of masking which we specifically use is something called as look ahead masking.
You'll understand it why we are just using 110 for zero zero padding token.
Uh, why we are creating a padding mask of this zero.
Okay.
Because this will probably get added and multiplied.
I'll show you okay how things will happen.
So in look ahead masking this is basically called as and this is used to maintain autoregressive property.
This is used to maintain autoregressive property.
Okay.
See one thing that you know right.
In decoder whatever decoder I have okay.
Let's say I'll draw it over here.
This is my decoder Okay.
Decoder word work is to.
Let's say if this is my encoder.
Decoder work is basically to.
Whenever we give any kind of input to our encoder, this information will pass to the decoder.
And finally I should be getting my input one by one.
Right.
111 like this one by one okay, so let's say if I say hey, how are you?
Okay.
So here you should uh, probably in Hindi will say, Cassey ho aap Cassey ho aap something like this.
So one one word should be coming up from this particular decoder right now.
You know that in order to find out the output.
Okay, let's say my three words output is over here.
If I want to get this word or if I want to get probably if I want to predict this word, I need to have
the information of this specific word.
Right.
I need to have some idea about this specific word.
Right.
I don't require this further word information and that is how decoder will probably work, right?
Similarly, if I want to probably get this specific word, I need to have the context of this particular
word itself.
Right.
So a very important point that I'm actually going to write over here.
While look ahead masking is specifically used.
And why do we say maintaining autoregressive property to ensure that each each position.
In the decoder.
In the decoder output sequence.
Can only attend to the previous position to the previous position, but not future position, but no
future position.
And that is what we want, right?
We don't want the future position.
We don't want the future information.
Instead, from the previous information I should be able to predict.
And that is why we basically use lookahead mask and why this is necessary for different different sequence
to sequence task like language modeling.
Or you can also say language translation right here.
Sequence is very much important.
Okay.
And to probably do the prediction I don't require the information of the future token in my decoder.
Right.
And that is how we predict the current token okay.
So I hope you have got an idea about lookahead mask.
Now let's uh, see some example and we'll try to find out, uh, how masking is basically done.
Uh, okay.
And then we will again come back to this specific example.
But let's see one simple example.
Okay.
Uh, with respect to uh, masking okay.
So here you will be able to see I will take an example okay.
So let's take an example.
So example is that uh let's say I have a sequence like four comma five comma zero okay.
Zero basically means zero padding.
Now for this what kind of padding mask?
I will create one comma, one comma zero.
Done.
Right.
Now let's consider that every every token is a three dimension okay.
Every token is basically three dimensional.
Four is represented by three dimension, five is represented by three.
Dimension zero is represented by three dimension.
So that is the reason.
What we can do is that we can also use this mask padding and we can extend it to three dimension.
So let's go ahead and extend it okay.
So guys now our task is basically to convert this 1D to 2D mask right.
And for that we need to extend padding mask to 2D okay.
Now how do I how do we go ahead and extend the padding mask to 2D.
Understand.
Over here whenever we have this one, that basically means it is a real token.
Zero is nothing, but it is a zero padded token.
Okay.
Now let's take this example 1450I have represented as 110.
Considering that this is a real token, this is a real token and this is just A00 token okay.
Now in order to convert this one d to two D mask, uh, you will be able to see that each row.
See each row in 2D mask will be a copy of the 1D mask.
Okay, now understand one thing, guys, uh, this is really important to just understand.
How do we apply 2D masking attention?
Okay, so first of all, I have already created my 1D mask.
So this is this is nothing, but it is a 1D mask.
One dimension mask.
Now in order to convert one dimension to two dimension each.
For each token in the sequence, the mask should indicate which token it can attend to.
Okay, now when I say which token it can attend to.
Over here, when I am probably doing the encoding for this.
If I'm converting this into vectors for should should probably take the context of this token, right?
Similarly, if I can probably take the context of this token.
They should not take the context of this zero, right?
I don't want my vector to get changed because of this zero.
Instead, I want my vector to get changed with respect to four and five.
Right?
So this two tokens I really want.
Right?
So what we are saying is that for each token.
For each token in the sequence in the sequence, the mask should indicate.
Which tokens it can attend to.
Okay.
So based on this, if my padding mask is one, the padding mask is this.
We will just go ahead and repeat this.
I know four is there.
Right.
So for four I will write 110.
That basically means four can the attention can be changed with respect to five.
Right.
Then if I go and this is for the first token right.
This is for the first token.
Similarly for the second token I will go ahead and write 110.
Why?
Because I am I'm I'm saying that hey five also needs to probably change the attention based on this
token.
Four.
Right.
So both this context should be there.
So that is the reason we have repeated 110110.
This token is basically for four.
This token is basically for five.
And if I probably consider with respect to this this can impact this.
This can basically impact this okay.
This can be a, uh, context.
I can also basically right over here, uh, token one.
Token one attends to token one comma two.
Similarly token two attends to two comma 1 or 1 comma two.
Right.
So this can probably get impacted.
Again I'm telling you what is the exact meaning why I'm saying it can attend to.
Okay, this is important for you all to understand, right?
When we say it can attend to.
Right?
Uh, in short, we we are basically saying that, hey, this token for can attend to token five in the
context of attention mechanism, attention mechanism, attention mechanism, we can change the context,
okay.
And we can change the representation.
But the final one, if I probably see with respect to the token zero, I don't want this to impact anything.
So my when I convert this one D mask to to 2D mask, this is how it is basically going to look at.
Right.
So this is my token one token two.
This both can probably change its own attention.
So I'm writing 110.
Similarly I have 110 over here then 000 over here okay.
So this is how things work with, uh, respect to this.
And I hope you got an idea with respect to this particular example.
Okay, but we'll not just stop over here, okay?
There are many more things that we really need to probably go ahead and discuss.
Okay.
And that is nothing.
But we need to also go ahead and apply one more padding okay.
And and how do we combine both of those padding.
We will try to understand that okay.
But I hope you got an idea.
idea over here.
So this is my extended padding mask.
Okay, then the next step, what we do is that we will go ahead and create our look ahead mask.
Now how do we calculate this.
Look ahead mask.
Look ahead mask is also based on which token can attend.
See this is completely dependent on the decoder output.
And here the sequence is very much important.
Now if I really want to create the look ahead mask, I know right when I probably get the first token,
I will keep it as one.
This will be zero zero because the further context I don't require.
Similarly, I will go ahead and write 110 the second token.
If I'm probably getting it, it should have the context of first, not the future.
And if I try to probably predict the third one, this should probably have the context of all the previous,
not the further one, right?
So since I'm working with three cross three.
So this will be my another three cross three matrix okay.
And uh, if I really want to specify this in a better way, it should be better that I create a separate
arrays for this or matrix for this.
So this will be 100.
Then I have the next one which is nothing but 110.
And I have nothing but 111.
Okay, since we are using three cross three, this is how it is basically going to look.
Now in this kind of masking you'll be able to see the top.
Everything will be zero right.
And in this masking, wherever the token is zero, I'll be making sure that I try to convert that into
an extended mask where this zero will be created in the similar way.
Okay.
Now we the next step will be that we combine.
We combine padding and look ahead, look ahead.
Mask.
Why do we combine with the help of padding mask?
You'll be able to see that we are removing all the zeros.
Right.
And how do we do this?
It is very much simple.
It is basically with the help of element wise.
Element wise.
Multiplication of two mask.
Multiplication of two mask okay.
Now in this scenario if I go ahead and combine my mask.
So combine mask is nothing.
But we will go ahead and do the calculation.
Uh, it's nothing, but it's a simple dot operation.
Okay, so here, uh, I will just go ahead and write.
If you just do the multiplication operation, final output that I'm actually going to get is 100.
Okay.
Then I have one comma one comma zero.
And then I have zero comma zero comma zero.
Okay.
So this is my final matrix.
And that is what we have done.
We have done the multiplication of two uh important metrics.
Uh mask matrix.
in short.
Okay.
And this is how you have actually got it right.
Then the next step in this will be that wherever the value is, zero.
Please listen to this very, very carefully.
Wherever in the combined mass the value is basically zero.
There we specify okay.
There we specifically specify infinity okay.
The reason I will tell you okay.
Why do we specify infinity.
But let us do one thing.
Let us do all the steps for this particular data that we have created right for this data.
So here I'm just going to copy this particular data because we are working on this.
And here we discussed about master application.
We saw example.
And finally we got the combined mask.
Now for this data we will go ahead and compute our mask okay.
So let's go ahead and compute our mask.
So here you could see uh this was my score right.
And uh with respect to this particular score.
Now let's go ahead and find out our look ahead mask.
So here I'm going to go ahead and compute my look ahead mask.
Now in order to get the look ahead mask.
It is very simple here how much?
I have four cross four right now with respect to four cross four again this will be 1000.
Then the next one will be 1100.
Then I have this 1110.
And then I have finally 1111 since it is a four dimension.
Right.
And here this is my complete look ahead mask.
Right.
And similarly if I want to get the padding mask okay.
So in order to get the padding mask and with respect to the padding mask, I know I need to have this
in the extended to 2D format.
Okay.
Single format.
It is very simple but 2D format.
What we really need to do over here is that I will go ahead and write one comma, one comma, one comma
zero, because this is not important.
Then again I have.
This is dependent on this.
This is in turn dependent on this dependent on this.
So I'll again go ahead and write this one.
Then I will be having my third one, one comma, one comma zero.
And then the fourth one will be nothing but zero comma zero comma zero comma zero.
Okay.
So this is my padding mask right now when I want to probably create my combined mask.
So it is nothing.
But if I want to probably find out my combined mask, it is the product of look ahead, look ahead mask
multiplied by, uh, multiplication by padding mask.
Now how are we going to do this particular product?
It is very simple.
Um, it's more like element wise multiplication.
This will get multiplied to this.
Like that.
Right.
So finally, if you go ahead and do this calculation, it is nothing but one multiplied by one, zero
multiplied by one zero multiplied by one zero multiplied by zero.
Okay.
Then the next one will be nothing but one.
Multiply by one, one multiplied by one, zero multiplied by one, and zero multiplied by zero.
Okay.
Then again one.
Multiply by one.
One multiplied by one.
Zeroth multiplied by zero.
So when we say multiplication, this is nothing but dot wise multiplication okay.
Uh, element wise multiplication.
Sorry.
One multiplied by one, one multiplied by one, one multiplied by one.
And here I have one multiplied by zero.
Okay.
So finally this is my output.
And the output is very simple.
Over here I will be getting one comma zero comma zero comma zero.
Then I have one comma, one comma zero comma zero.
Then I have one comma one comma one zero.
Then I have one comma, one comma, one comma.
Sorry one comma zero.
So this is my final combined padding.
This zero is there basically to ignore all the tokens.
Padded tokens.
And wherever there is one those are very important okay.
Now the next step which is really really important okay.
Which is about masked scores.
We go ahead and compute our mask scores.
And it is very simple.
The mask score is nothing, but wherever there are zeros that is going to just will just go ahead and
add.
You can just basically do something over here okay.
So let's, let's let's see this okay.
Now here I have my combined mask okay.
Now for getting this combined mask for applying this combined mask.
What we'll do we'll apply this combined mask to this entire matrix okay.
And the next step that you'll be seeing after this will be that we will try to convert this into a mask
over here.
And in this instead of zero here we are going to convert this into minus infinity y minus infinity.
I'll just talk about it in a while okay.
So here my next step will be converting everything into minus infinity.
The reason is very simple.
Very very simple.
Just think over it.
Till then I will go ahead and write this minus infinity.
Wherever there is zero that will get converted into minus infinity okay.
So this in turn I'm actually getting in the form of minus infinity.
And again we do the dot wise operation with this right.
Wherever there is minus infinity that will become minus infinity.
Okay, so final mass score, you'll be able to see that I will go ahead and just do the dot operation.
So I'm going to get point three minus infinity minus infinity minus infinity okay.
And then you have this .71.9 minus infinity minus infinity.
Then similarly you have 1.13.15.1.
And again you have minus infinity.
Then you have 0.0, 0.00.0.
Then again you have minus infinity.
So this is your final mask score.
Now.
The most important thing that you really need to understand is why you need to have this minus infinity.
So here you'll be able to see that, uh, one minor mistake that I did was that I told, hey, we are
replacing zero with minus infinity.
No, minus infinity is getting added to this specific zero value.
Okay.
And why this is actually done is that I will just go ahead and write the reason over here, okay.
When we apply or when we try to, uh, you know, add this specific value.
Okay.
It is basically to zero out or it is to remove the influence, zero out the influence when the softmax
activation function is applied.
Because the next step is applying softmax activation function.
Okay.
Now when we apply softmax to minus infinity you will be seeing the answer what you will be able to get.
Okay.
Uh, this is crucial.
This step is crucial to ensure that certain position.
Right.
Example.
If I probably consider padded tokens or future tokens in the case of lookahead, masking do not affect
the attention mechanism, okay.
And obviously the impact of zeros will also get removed, right?
When we apply a mask, we need to ensure that a certain tokens do not influence the attention mechanism.
We achieve this by setting the attention score for these tokens to a very high large negative numbers.
When the softmax function is applied to the scores, it converts this.
This entire value will be getting converted to zero.
Okay, ensuring that there is no much influence on the attention weights.
So attention weights will not get impacted.
Right?
Because after this, after we calculate, uh, the mass scores, what we are specifically going to do,
okay, we have to probably convert, uh, or apply a softmax activation function.
I hope everybody knows that.
Right.
So that is what we are specifically going to do over here.
Uh, over here, we are just going to go ahead and apply our softmax.
That is our next step.
So let's go ahead and do the softmax because after this we'll go ahead and add our values.
Right.
So softmax scores that we are going to get is nothing but softmax.
And here I'm going to use my masked scores right now with respect to the mask scores.
When I apply the softmax.
Uh, here you will be able to see that I will get one point.
Oh, I've done the calculation minus infinity is basically going to convert into zero.
Okay.
You can go ahead and write this or you can also check it out.
This will be .3.70.00.0.
Okay then I have .3.6.60.0.
And then I have 1.0, 0.0, 0.00.0.
So this is my entire score that I'm actually getting Okay.
And then, as you know, in the self-attention, the final step will be that we will do the weighted
sum of values, right.
So every number over here inside the softmax is going to get multiplied by, uh, this this this softmax
score okay is going to get multiplied by v okay.
So here my attention output will be nothing, but it will be softmax scores.
Multiplied by the vector v okay so guys this is the step by step mechanism that specifically happens
in the decoder which is similar to encoder.
But one additional step that you were able to find out was about masking.
Right.
And what is the importance of masking?
I will be giving you in this specific PDF in the word of writing, so that you'll be able to understand
masking in the transformer architecture is essential for several reasons.
It helps manage the structure of sequence being processed and ensures the model behaves correctly during
training and inferences.
Here are the key reasons for using masking.
One is basically to handle variable length sequence and padding masking because there will be a lot
many number of zeros to handle sequence of different length in a batch to ensure the padding tokens,
which are added to make sequences of uniform length do not affect the model prediction.
And second is basically to maintaining autoregressive property with lookahead mask, right?
The purpose is very simple to ensure that each position in a decoder output sequence can only attend
to the previous position or itself, but not future position.
This is crucial for sequence generation task like language modeling and translation, where the model
should not have the access to the future tokens while predicting the current token.
Okay, so these are some of the important steps with respect to the masking.
And now the next step that we are basically going to discuss about is add a normalization.
And then what is this exact line and why this particular line is also coming up.
Okay.
So that is what we are going to discuss about in this specific in the next video.
So yes, this was it for my side.
I'll see you all in the next video.
Thank you.