1
00:00:00,000 --> 00:00:01,000
Hello guys.

2
00:00:01,000 --> 00:00:05,000
So we are going to continue the NLP with Deep Learning series.

3
00:00:05,000 --> 00:00:11,000
Uh, already till now we have discussed about simple RNN right?

4
00:00:11,000 --> 00:00:15,000
And we saw the practical implementation already.

5
00:00:15,000 --> 00:00:15,000
Right.

6
00:00:16,000 --> 00:00:18,000
And we saw complete little deployment.

7
00:00:18,000 --> 00:00:24,000
So this entire implementation with deployment we have also seen inside the simple RNN, we also discussed

8
00:00:24,000 --> 00:00:31,000
about something called as embedding layer which was specifically used to convert your text into vectors.

9
00:00:31,000 --> 00:00:32,000
Right.

10
00:00:32,000 --> 00:00:42,000
Then, uh, we went ahead and understood about LSTM, uh, GRU variants of RNN.

11
00:00:42,000 --> 00:00:43,000
Right.

12
00:00:43,000 --> 00:00:49,000
And along with that other variants also we saw and I have also completed the practical implementation

13
00:00:49,000 --> 00:00:56,000
for this practical implementation For this right, we understood what were the problems in simple RNN.

14
00:00:56,000 --> 00:01:01,000
And then with respect to that, we also understood about LSTM and GRU variants.

15
00:01:01,000 --> 00:01:06,000
Now coming to the third one, which is uh, today what I'm actually going to discuss about is something

16
00:01:06,000 --> 00:01:11,000
called as bidirectional RNN.

17
00:01:11,000 --> 00:01:11,000
Okay.

18
00:01:11,000 --> 00:01:12,000
It can be RNN.

19
00:01:12,000 --> 00:01:20,000
It can be a LSTM RNN okay, I'll right RNN or LSTM RNN or gr rnn, but it will be bidirectional.

20
00:01:20,000 --> 00:01:21,000
Okay.

21
00:01:21,000 --> 00:01:29,000
Now first of all, I will just try to make you understand why bidirectional RNN or bidirectional LSTM

22
00:01:29,000 --> 00:01:29,000
RNN.

23
00:01:29,000 --> 00:01:29,000
Okay.

24
00:01:30,000 --> 00:01:33,000
And then we will probably go ahead and discuss about the architecture.

25
00:01:33,000 --> 00:01:35,000
And uh we will solve this okay.

26
00:01:35,000 --> 00:01:41,000
Now first of all, uh, let's go ahead and talk about our simple RNN right in our simple RNN, what

27
00:01:41,000 --> 00:01:46,000
we used to do like, uh, or simple RNN or LSTM, RNN, what we used to do.

28
00:01:46,000 --> 00:01:49,000
So here was my input here.

29
00:01:49,000 --> 00:01:52,000
I used to get my output and here was my hidden state.

30
00:01:52,000 --> 00:01:57,000
And with respect to that, uh, if I unfold this with respect to time, right.

31
00:01:57,000 --> 00:02:03,000
If I just unfold it so it would, it would look something like this.

32
00:02:03,000 --> 00:02:03,000
Right.

33
00:02:03,000 --> 00:02:03,000
right.

34
00:02:04,000 --> 00:02:11,000
So this will be my with respect to t is equal to two and I have t is equal to three.

35
00:02:13,000 --> 00:02:17,000
Then I have t is equal to four.

36
00:02:17,000 --> 00:02:25,000
Like this I may have a lot of um you know, the same hidden neurons, hidden states.

37
00:02:25,000 --> 00:02:25,000
Right.

38
00:02:25,000 --> 00:02:28,000
So over here you'll be able to see I will pass at t is equal to one.

39
00:02:28,000 --> 00:02:32,000
I may go ahead and pass X11.

40
00:02:32,000 --> 00:02:33,000
Then I may pass X12.

41
00:02:33,000 --> 00:02:36,000
Then similarly I may pass X13.

42
00:02:36,000 --> 00:02:39,000
And finally I may pass S1N right.

43
00:02:39,000 --> 00:02:41,000
This is with respect to the first sentence.

44
00:02:41,000 --> 00:02:42,000
Right.

45
00:02:42,000 --> 00:02:47,000
And you know that here also, uh, for every neuron right.

46
00:02:47,000 --> 00:02:48,000
We will also be getting an output.

47
00:02:48,000 --> 00:02:51,000
And finally whatever output we are specifically getting over here.

48
00:02:52,000 --> 00:02:57,000
Uh, let's say if I just want to predict one kind of output, if I just take an example with respect

49
00:02:57,000 --> 00:02:58,000
to sentiment analysis.

50
00:02:58,000 --> 00:03:01,000
So this will be my final output Y hat.

51
00:03:01,000 --> 00:03:08,000
And before I get the y hat here, I'm going to just apply a sigmoid activation function right here.

52
00:03:08,000 --> 00:03:11,000
I may go ahead and apply my sigmoid activation function.

53
00:03:11,000 --> 00:03:14,000
So let's go ahead and apply this right.

54
00:03:14,000 --> 00:03:21,000
So this will basically be my sigmoid if it if I just have a binary classification I used to get y hat.

55
00:03:21,000 --> 00:03:24,000
And then we used to basically go ahead and compute our loss.

56
00:03:24,000 --> 00:03:28,000
And based on our optimization we used to reduce this loss okay.

57
00:03:28,000 --> 00:03:31,000
And uh by default some initial weights.

58
00:03:31,000 --> 00:03:36,000
Also we used to transfer and here with respect to the weights what all weights we used, one was our

59
00:03:36,000 --> 00:03:38,000
input weight, one was our hidden weight.

60
00:03:38,000 --> 00:03:40,000
And this is also my hidden weight.

61
00:03:40,000 --> 00:03:42,000
This is also my hidden weight.

62
00:03:42,000 --> 00:03:47,000
And here your you will also be able to see that I will get this as my output.

63
00:03:47,000 --> 00:03:52,000
And I can just consider drawing it over here.

64
00:03:52,000 --> 00:03:52,000
Right.

65
00:03:52,000 --> 00:03:57,000
So let's say I hope everybody remembers this architecture right.

66
00:03:57,000 --> 00:04:05,000
So I may take this as my w o output and then pass to my sigmoid and get my y hat.

67
00:04:05,000 --> 00:04:06,000
Okay.

68
00:04:07,000 --> 00:04:11,000
And then what we should do is to basically go ahead and compute our loss.

69
00:04:11,000 --> 00:04:11,000
Okay.

70
00:04:12,000 --> 00:04:15,000
Now, uh, in this scenario.

71
00:04:15,000 --> 00:04:15,000
Right.

72
00:04:15,000 --> 00:04:18,000
And when we just consider this is basically my hidden state.

73
00:04:18,000 --> 00:04:21,000
So in the input also I will be giving some kind of weights.

74
00:04:21,000 --> 00:04:23,000
And this will also be my hidden weights.

75
00:04:23,000 --> 00:04:23,000
Right.

76
00:04:23,000 --> 00:04:25,000
So three types of weights.

77
00:04:25,000 --> 00:04:31,000
Uh we were specifically using input weights hidden weights and w o for output weights.

78
00:04:31,000 --> 00:04:32,000
Right.

79
00:04:32,000 --> 00:04:38,000
And with respect to this I had my output something like 010203 and oh four.

80
00:04:38,000 --> 00:04:38,000
Right.

81
00:04:38,000 --> 00:04:40,000
Whatever outputs we are basically creating.

82
00:04:40,000 --> 00:04:43,000
Now let's take one specific example.

83
00:04:43,000 --> 00:04:49,000
And before that, I just want to talk about various problem statements that you can solve with RNN.

84
00:04:49,000 --> 00:04:49,000
Okay.

85
00:04:49,000 --> 00:04:54,000
Like there is one kind of RNN which is called as one to many RNN.

86
00:04:55,000 --> 00:04:57,000
Okay, one to many RNN.

87
00:04:57,000 --> 00:04:59,000
Uh, I will just first of all categorize them.

88
00:04:59,000 --> 00:05:00,000
Okay.

89
00:05:00,000 --> 00:05:03,000
Then I have something called as many to one RNN.

90
00:05:04,000 --> 00:05:05,000
Okay.

91
00:05:06,000 --> 00:05:10,000
And the third one is like many to many RNN.

92
00:05:15,000 --> 00:05:16,000
Okay.

93
00:05:16,000 --> 00:05:17,000
Many to many RNN.

94
00:05:17,000 --> 00:05:18,000
Okay.

95
00:05:18,000 --> 00:05:21,000
And the fourth one is something like 1 to 1 RNN.

96
00:05:24,000 --> 00:05:27,000
So these are the different types of RNN.

97
00:05:27,000 --> 00:05:35,000
We specifically use okay one to many RNN, many to one RNN, many to many RNN and 1 to 1 RNN.

98
00:05:35,000 --> 00:05:40,000
Now, in order to explain you some of the examples, I'll first of all go and show you these are the

99
00:05:40,000 --> 00:05:45,000
variants or these are the types of RNN that I can say with respect to problem statement.

100
00:05:45,000 --> 00:05:46,000
Okay.

101
00:05:46,000 --> 00:05:49,000
And we will go ahead and discuss some of the types.

102
00:05:49,000 --> 00:05:54,000
And then we'll go back again to our LSTM and we'll try to discuss sorry.

103
00:05:54,000 --> 00:05:58,000
We'll go towards this bidirectional RNN and discuss more about it okay.

104
00:05:58,000 --> 00:06:03,000
So first of all let me just quickly go ahead and give some of the examples of one to many and many to

105
00:06:03,000 --> 00:06:06,000
one RNN and all with some real world examples itself.

106
00:06:06,000 --> 00:06:07,000
So guys.

107
00:06:07,000 --> 00:06:14,000
So here is one of the diagram that I am specifically using to explain you about 1 to 1, one to many,

108
00:06:14,000 --> 00:06:15,000
many to one and many to many.

109
00:06:15,000 --> 00:06:18,000
And there is also one more variant of many to many.

110
00:06:18,000 --> 00:06:19,000
Okay.

111
00:06:19,000 --> 00:06:25,000
So, uh, we will just go ahead and discuss about this, you know, and uh, let's take one example

112
00:06:25,000 --> 00:06:26,000
for at a time, okay?

113
00:06:26,000 --> 00:06:34,000
Now, with respect to this particular RNN, you should be able to see that there is one input and one

114
00:06:34,000 --> 00:06:37,000
output, okay, one input and one output.

115
00:06:37,000 --> 00:06:40,000
So we will discuss one example for this okay.

116
00:06:40,000 --> 00:06:42,000
So here I'm giving one input.

117
00:06:42,000 --> 00:06:45,000
And I am getting one output with respect to this particular RNN.

118
00:06:45,000 --> 00:06:45,000
Okay.

119
00:06:46,000 --> 00:06:50,000
Uh if I probably go ahead and see one to too many.

120
00:06:50,000 --> 00:06:51,000
Okay.

121
00:06:51,000 --> 00:06:52,000
One too many.

122
00:06:52,000 --> 00:06:57,000
So here in this one, too many, you'll be able to see that I'm giving just one input.

123
00:06:57,000 --> 00:07:04,000
And based on this one output, okay, it will basically give you several outputs okay.

124
00:07:04,000 --> 00:07:08,000
So here your output are many okay.

125
00:07:08,000 --> 00:07:12,000
Now in order to talk about an example over here okay.

126
00:07:12,000 --> 00:07:15,000
One input and many outputs okay.

127
00:07:15,000 --> 00:07:18,000
So that is the reason we say it as one to many.

128
00:07:18,000 --> 00:07:18,000
Okay.

129
00:07:19,000 --> 00:07:25,000
Let's say I give an example in Google like if you probably there is there is a concept of something

130
00:07:25,000 --> 00:07:26,000
called as image captioning.

131
00:07:27,000 --> 00:07:36,000
What does image captioning mean is that whenever I specifically give one image okay I when I specifically

132
00:07:36,000 --> 00:07:44,000
give one image image captioning, when I give one image, the model should be able to tell or give a

133
00:07:44,000 --> 00:07:46,000
text information about that particular image.

134
00:07:46,000 --> 00:07:54,000
Let's say if I am giving a image which has a dog and a cat, right?

135
00:07:54,000 --> 00:08:00,000
So here let's say there is 111 dog and there is one cat, right.

136
00:08:00,000 --> 00:08:05,000
So this model should basically be say after giving this particular image, there is a dog and cat in

137
00:08:05,000 --> 00:08:05,000
the image.

138
00:08:05,000 --> 00:08:05,000
Right.

139
00:08:05,000 --> 00:08:07,000
There is a dog and cat.

140
00:08:07,000 --> 00:08:11,000
So this is basically called as image captioning.

141
00:08:11,000 --> 00:08:12,000
In this we are just giving one input.

142
00:08:12,000 --> 00:08:15,000
But we are getting many outputs in form of words.

143
00:08:15,000 --> 00:08:16,000
Okay.

144
00:08:16,000 --> 00:08:22,000
Now similarly if I go ahead and see many to one okay in many to one, you will be able to see that over

145
00:08:22,000 --> 00:08:27,000
here I have multiple inputs and I have one output okay.

146
00:08:27,000 --> 00:08:35,000
Now this also example if I probably go ahead and in the Google search Google image search I search for

147
00:08:35,000 --> 00:08:35,000
a input.

148
00:08:35,000 --> 00:08:42,000
Let's say a cat eating food a cat eating food.

149
00:08:43,000 --> 00:08:49,000
So if you go ahead and search this in Google search image, you'll be able to get an images of dog eating

150
00:08:50,000 --> 00:08:52,000
sorry of cat eating food right?

151
00:08:53,000 --> 00:08:55,000
You'll be able to get this specific image.

152
00:08:55,000 --> 00:09:01,000
Now in this case you are able to generate one output with respect to multiple inputs, right?

153
00:09:01,000 --> 00:09:07,000
So you have your inputs are multiple, your inputs are many, and your output is one, right?

154
00:09:08,000 --> 00:09:10,000
Similarly here you have many to many.

155
00:09:10,000 --> 00:09:15,000
So this two examples that you can specifically consider is like your language translation.

156
00:09:16,000 --> 00:09:22,000
So in language translation, if I want to convert one language to the other I may give one many inputs.

157
00:09:22,000 --> 00:09:23,000
Right.

158
00:09:23,000 --> 00:09:29,000
And similarly I will be able to get many outputs right based on the sentence that we are actually working

159
00:09:29,000 --> 00:09:30,000
with.

160
00:09:30,000 --> 00:09:35,000
Now these are some of the types of RNN that everybody should probably know about.

161
00:09:35,000 --> 00:09:35,000
Okay.

162
00:09:35,000 --> 00:09:41,000
And the reason why I'm talking about this particular RNN, let's say if I have a use case, which is

163
00:09:41,000 --> 00:09:42,000
like many to many.

164
00:09:42,000 --> 00:09:45,000
Now let's go back to this particular use case.

165
00:09:45,000 --> 00:09:46,000
Okay.

166
00:09:46,000 --> 00:09:49,000
I'm just going to probably take this entire diagram.

167
00:09:49,000 --> 00:09:50,000
Okay.

168
00:09:50,000 --> 00:09:51,000
I will copy this again.

169
00:09:51,000 --> 00:09:53,000
Go back and explain it over here.

170
00:09:53,000 --> 00:09:59,000
Now in this particular case, if I am considering all the outputs right.

171
00:09:59,000 --> 00:10:02,000
If I'm considering all the outputs here, let's say this is my Y one.

172
00:10:02,000 --> 00:10:04,000
This is my Y two.

173
00:10:04,000 --> 00:10:05,000
This is my y three.

174
00:10:05,000 --> 00:10:06,000
This is my y four.

175
00:10:06,000 --> 00:10:10,000
With respect to all the inputs, if I'm considering all the outputs, if I require all the outputs,

176
00:10:10,000 --> 00:10:12,000
then I will specifically use many to many.

177
00:10:12,000 --> 00:10:17,000
And if I'm considering all the outputs here, I can basically go ahead and use sigmoid or softmax,

178
00:10:17,000 --> 00:10:21,000
any types of activation function like how I have actually applied over here.

179
00:10:21,000 --> 00:10:21,000
Okay.

180
00:10:22,000 --> 00:10:25,000
So why I am teaching you this?

181
00:10:25,000 --> 00:10:28,000
Let's take a specific example.

182
00:10:28,000 --> 00:10:28,000
Okay.

183
00:10:28,000 --> 00:10:30,000
So I will take a text example.

184
00:10:30,000 --> 00:10:36,000
Now that is where your bidirectional LSTM, RNN or bidirectional RNN will come.

185
00:10:36,000 --> 00:10:37,000
Okay so my name is Krish.

186
00:10:37,000 --> 00:10:44,000
I'll say hey uh Krish okay I will, I'll just write a sentence.

187
00:10:44,000 --> 00:10:48,000
Likes to eat one word.

188
00:10:48,000 --> 00:10:52,000
dash in Bangalore.

189
00:10:52,000 --> 00:10:56,000
So I'm giving this specific example.

190
00:10:56,000 --> 00:10:56,000
Okay.

191
00:10:56,000 --> 00:10:58,000
Krish likes to eat.

192
00:10:58,000 --> 00:10:59,000
Okay.

193
00:10:59,000 --> 00:11:00,000
One word.

194
00:11:00,000 --> 00:11:03,000
I need to predict this word in Bangalore.

195
00:11:03,000 --> 00:11:04,000
Okay.

196
00:11:04,000 --> 00:11:08,000
So let's say at x t is equal to one I give this word at t is equal to two.

197
00:11:08,000 --> 00:11:09,000
I give this word t is equal to three.

198
00:11:09,000 --> 00:11:11,000
I give this word t is equal to four.

199
00:11:11,000 --> 00:11:11,000
I give this word.

200
00:11:11,000 --> 00:11:11,000
Okay.

201
00:11:11,000 --> 00:11:14,000
Let's consider that I'm giving this okay.

202
00:11:14,000 --> 00:11:22,000
So I'll just say instead of writing each two I will just go ahead and write eats okay.

203
00:11:22,000 --> 00:11:25,000
Krish eats dash word okay.

204
00:11:25,000 --> 00:11:26,000
So first word goes over here.

205
00:11:26,000 --> 00:11:27,000
Second word goes over here.

206
00:11:27,000 --> 00:11:31,000
And I need to basically do the prediction for the third word okay.

207
00:11:31,000 --> 00:11:33,000
Prediction for the third word.

208
00:11:33,000 --> 00:11:35,000
And that is what I really need to predict it over here.

209
00:11:35,000 --> 00:11:42,000
Now in this scenario, please note okay, if I want to probably calculate this particular word, if

210
00:11:42,000 --> 00:11:52,000
I use or if I need to predict this word if I use simple RNN or LSTM, RNN or GRU and write in all these

211
00:11:52,000 --> 00:12:00,000
three cases, my this prediction will be dependent on this two words right on the previous context,

212
00:12:00,000 --> 00:12:01,000
right?

213
00:12:01,000 --> 00:12:06,000
But here you will be able to see this specific example.

214
00:12:06,000 --> 00:12:12,000
If I really need to do the prediction for this particular word, I also need the context of the upcoming

215
00:12:12,000 --> 00:12:13,000
words.

216
00:12:13,000 --> 00:12:17,000
Because if I write Krish eats dash in Bangalore.

217
00:12:17,000 --> 00:12:18,000
So in Bangalore what is famous?

218
00:12:18,000 --> 00:12:19,000
I will eat that right?

219
00:12:19,000 --> 00:12:21,000
Let's say there is also one more sentence.

220
00:12:21,000 --> 00:12:22,000
I will go ahead and write over here.

221
00:12:22,000 --> 00:12:28,000
Krish, eats something in Paris.

222
00:12:29,000 --> 00:12:35,000
So if I take this word now, you know that if my first real training data says that I eat dosa.

223
00:12:36,000 --> 00:12:40,000
Okay, if you don't know about dosa, I think you can go and search in the Google Krish eat dosa in

224
00:12:40,000 --> 00:12:41,000
Bangalore.

225
00:12:41,000 --> 00:12:44,000
Because dosa is very famous right now.

226
00:12:44,000 --> 00:12:47,000
Obviously in my training data I will have dosa, right?

227
00:12:47,000 --> 00:12:49,000
And over here you'll be seeing that in my training data.

228
00:12:49,000 --> 00:12:51,000
I may have something like this.

229
00:12:51,000 --> 00:12:55,000
I may want to eat pizza in Paris because pizza may be famous in Paris.

230
00:12:55,000 --> 00:12:56,000
Right.

231
00:12:56,000 --> 00:13:03,000
So here you will be able to see that this word, if we are trying to predict it, does not only depends

232
00:13:03,000 --> 00:13:07,000
on the context of the previous word, but it also depends on the context of the forward word.

233
00:13:07,000 --> 00:13:12,000
Because when the country changes or when the city changes, you will be able to see that this specific

234
00:13:12,000 --> 00:13:13,000
word may also change.

235
00:13:14,000 --> 00:13:20,000
So in this scenario, if I really want to predict a word in between here, I need to also have the context

236
00:13:20,000 --> 00:13:22,000
of the upcoming word.

237
00:13:22,000 --> 00:13:25,000
But in this scenario I don't have that if I use simple WRN.

238
00:13:26,000 --> 00:13:26,000
WRN.

239
00:13:26,000 --> 00:13:27,000
WRN.

240
00:13:27,000 --> 00:13:29,000
I am not able to do that.

241
00:13:29,000 --> 00:13:30,000
I'm not able to get it.

242
00:13:30,000 --> 00:13:33,000
So how do we fix this particular problem?

243
00:13:33,000 --> 00:13:40,000
So fixing this specific problem is what bidirectional RNN is actually doing.

244
00:13:40,000 --> 00:13:45,000
Now in the case of bidirectional RNN what will specifically happen.

245
00:13:45,000 --> 00:13:46,000
We will discuss.

246
00:13:47,000 --> 00:13:50,000
So let's go ahead and take this again okay.

247
00:13:51,000 --> 00:13:55,000
I will just go ahead and rub this for now okay.

248
00:13:56,000 --> 00:13:58,000
So here you will be able to see this.

249
00:13:58,000 --> 00:14:04,000
I don't know what has happened, but let's go ahead and draw this again okay I will go ahead and draw

250
00:14:04,000 --> 00:14:05,000
this.

251
00:14:05,000 --> 00:14:07,000
So first of all I will go ahead and draw a simple RNN.

252
00:14:07,000 --> 00:14:10,000
And then I'll try to convert that into a bidirectional RNN.

253
00:14:10,000 --> 00:14:16,000
And how with the help of further context we will probably predict the words okay, so let's say this

254
00:14:16,000 --> 00:14:23,000
is my LSTM or this is my RNN with respect to unfolding with respect to time.

255
00:14:24,000 --> 00:14:24,000
Okay.

256
00:14:31,000 --> 00:14:32,000
And here you have this.

257
00:14:32,000 --> 00:14:33,000
Okay.

258
00:14:33,000 --> 00:14:36,000
Now in this I pass my input with respect to this.

259
00:14:36,000 --> 00:14:39,000
I get my this here is my input.

260
00:14:39,000 --> 00:14:40,000
Here is my input.

261
00:14:40,000 --> 00:14:45,000
And you know that since we are trying to solve a many to many problem, it can be many to one.

262
00:14:45,000 --> 00:14:46,000
It can be any kind of problems Okay.

263
00:14:46,000 --> 00:14:50,000
So here what we are basically going to do here is my output okay.

264
00:14:50,000 --> 00:14:57,000
Now now you know that if I have a sentence like this right.

265
00:14:57,000 --> 00:15:03,000
Krish eats dash in Bangalore.

266
00:15:06,000 --> 00:15:06,000
Right.

267
00:15:06,000 --> 00:15:08,000
I need to predict this particular word.

268
00:15:09,000 --> 00:15:13,000
In this case I will give my X11X12X13.

269
00:15:13,000 --> 00:15:13,000
Like this.

270
00:15:13,000 --> 00:15:17,000
Up till the X1N, whichever is the last word right here.

271
00:15:17,000 --> 00:15:21,000
You know that what we will be having here, we'll be having our input weights here.

272
00:15:21,000 --> 00:15:24,000
Hidden weights here, input weights here.

273
00:15:24,000 --> 00:15:25,000
Hidden weights here.

274
00:15:25,000 --> 00:15:27,000
Input weights here.

275
00:15:27,000 --> 00:15:28,000
Hidden weights okay.

276
00:15:28,000 --> 00:15:30,000
Similarly here uh input weights.

277
00:15:30,000 --> 00:15:33,000
And here finally I'll get my output weight okay.

278
00:15:33,000 --> 00:15:33,000
Okay.

279
00:15:33,000 --> 00:15:37,000
And with respect to this output weight, it will pass to the sigmoid activation function.

280
00:15:37,000 --> 00:15:38,000
Whatever activation function.

281
00:15:38,000 --> 00:15:44,000
Now in this case what we do we pass one by word 111 word at every time.

282
00:15:44,000 --> 00:15:48,000
Step at t is equal to one, t is equal to two, t is equal to three, t is equal to four.

283
00:15:48,000 --> 00:15:52,000
So this is my RNN a simple RNN right.

284
00:15:53,000 --> 00:15:56,000
Or if I take a LSTM RNN this will probably be changing inside that.

285
00:15:56,000 --> 00:15:59,000
But the flow will be almost like this, right?

286
00:16:00,000 --> 00:16:02,000
Now what does bidirectional LSTM says?

287
00:16:02,000 --> 00:16:05,000
Hey, if I want.

288
00:16:05,000 --> 00:16:08,000
So let's say this is Y1Y2Y3Y4.

289
00:16:09,000 --> 00:16:09,000
Okay.

290
00:16:09,000 --> 00:16:19,000
It says that hey if y three wants the context of the upcoming words, it is better that you use one

291
00:16:19,000 --> 00:16:28,000
or RNN and make sure that instead of passing the words from front, you pass all the words from back.

292
00:16:28,000 --> 00:16:32,000
Okay, so what it is saying, hey, go ahead and try to create another RNN.

293
00:16:32,000 --> 00:16:37,000
So here I will go ahead and create another RNN which will look something like this.

294
00:16:37,000 --> 00:16:38,000
So let's say this is my RNN.

295
00:16:39,000 --> 00:16:40,000
This is my RNN.

296
00:16:42,000 --> 00:16:43,000
This is my another RNN.

297
00:16:45,000 --> 00:16:46,000
This is my another RNA.

298
00:16:46,000 --> 00:16:50,000
Okay, now I will consider all this RNN.

299
00:16:51,000 --> 00:16:57,000
Then what I will do along with this inputs I will also pass this input here.

300
00:16:57,000 --> 00:16:58,000
Okay.

301
00:16:58,000 --> 00:17:01,000
And here I am going to get my output.

302
00:17:01,000 --> 00:17:04,000
It will get combined to this particular output.

303
00:17:04,000 --> 00:17:10,000
Along with this you will also be seeing that I will pass this information one by one.

304
00:17:10,000 --> 00:17:12,000
I will pass this information.

305
00:17:12,000 --> 00:17:14,000
See, this needs to be also getting connected, right.

306
00:17:14,000 --> 00:17:19,000
Without without the other uh, other uh other RNN.

307
00:17:19,000 --> 00:17:22,000
If it is not connected, then it will be of no use, right?

308
00:17:22,000 --> 00:17:28,000
So what we will do, we will try to connect this RNN with this, this RNN with this.

309
00:17:29,000 --> 00:17:34,000
Then we'll connect this RNN with this, like how it is connected over here with respect to the hidden

310
00:17:34,000 --> 00:17:35,000
state okay.

311
00:17:35,000 --> 00:17:40,000
And whatever RNN is basically coming out as an output.

312
00:17:40,000 --> 00:17:44,000
This will get combined to this Y3 okay.

313
00:17:44,000 --> 00:17:48,000
So this will get combined to y3.

314
00:17:48,000 --> 00:17:50,000
Let me just complete this drawing.

315
00:17:50,000 --> 00:17:54,000
Then you'll be able to understand more and why we are specifically doing this.

316
00:17:55,000 --> 00:18:01,000
And you should know that all these words that I'm actually passing, right.

317
00:18:01,000 --> 00:18:05,000
All these words, this should also get connected to this.

318
00:18:05,000 --> 00:18:07,000
It should also get passed to this.

319
00:18:07,000 --> 00:18:09,000
So here are all my words.

320
00:18:10,000 --> 00:18:10,000
Right?

321
00:18:13,000 --> 00:18:14,000
Right.

322
00:18:14,000 --> 00:18:21,000
So this is the entire architecture of a bidirectional, uh, RNN, or it can be a LSTM RNN.

323
00:18:21,000 --> 00:18:26,000
If I just change this hidden layer from RNN into LSTM, RNN becomes bidirectional LSTM rnn okay, now

324
00:18:26,000 --> 00:18:29,000
see see focus on this entire architecture.

325
00:18:29,000 --> 00:18:36,000
First of all, I will use this white colored RNN or LSTM RNN in the forward propagation by passing this

326
00:18:36,000 --> 00:18:36,000
word.

327
00:18:36,000 --> 00:18:38,000
So this will be my X11.

328
00:18:38,000 --> 00:18:39,000
This will be my X12.

329
00:18:39,000 --> 00:18:49,000
This will be my X13X14X15 okay so I will be passing every word over here one by one right in this direction.

330
00:18:50,000 --> 00:18:55,000
In the other RNN I will be passing from backward direction every word.

331
00:18:55,000 --> 00:19:03,000
So here you can see my last word at t is equal to one will pass get passed over here right from in the

332
00:19:03,000 --> 00:19:04,000
backward direction.

333
00:19:04,000 --> 00:19:07,000
Then my next word will get passed over here.

334
00:19:07,000 --> 00:19:09,000
Right then my next word will get passed over here.

335
00:19:09,000 --> 00:19:14,000
So with respect to timestamp in the reverse direction, if I go from this direction, this will be t

336
00:19:14,000 --> 00:19:15,000
is equal to one.

337
00:19:15,000 --> 00:19:16,000
This will be.

338
00:19:16,000 --> 00:19:17,000
This will be t is equal to two.

339
00:19:17,000 --> 00:19:19,000
This will be t is equal to three.

340
00:19:19,000 --> 00:19:20,000
This will be t is equal to four.

341
00:19:20,000 --> 00:19:23,000
And I am just reversing the sentence.

342
00:19:23,000 --> 00:19:25,000
So first Bangalore will get passed over here.

343
00:19:25,000 --> 00:19:28,000
Then in will get passed over here.

344
00:19:28,000 --> 00:19:30,000
Then we will also go ahead and calculate the y hat.

345
00:19:30,000 --> 00:19:30,000
Right.

346
00:19:30,000 --> 00:19:33,000
And this will basically get concatenated.

347
00:19:33,000 --> 00:19:35,000
This will get combined right.

348
00:19:35,000 --> 00:19:37,000
So I hope you are able to understand this entire flow.

349
00:19:37,000 --> 00:19:40,000
Now what is the use of this.

350
00:19:40,000 --> 00:19:43,000
Let's say if I want to do the prediction for y three right.

351
00:19:43,000 --> 00:19:48,000
Let's say I want to do this prediction of this blank crush eats dash okay.

352
00:19:48,000 --> 00:19:49,000
Some blank statement.

353
00:19:49,000 --> 00:19:50,000
So here you can see that.

354
00:19:50,000 --> 00:19:58,000
Now to predict this I will be having the context of x one to I will be having the context of X11.

355
00:19:58,000 --> 00:20:04,000
Along with this, you'll also be seeing that I will also be having the context of this particular words,

356
00:20:04,000 --> 00:20:11,000
because my this RNN, when it is running from the reverse direction, that entire context of X14 and

357
00:20:11,000 --> 00:20:13,000
X13 will be available.

358
00:20:13,000 --> 00:20:18,000
And when it says okay, cricket and Bangalore is also coming from this particular direction with respect

359
00:20:18,000 --> 00:20:23,000
to the vectors, it will definitely be able to understand that, okay, the food that is probably famous

360
00:20:23,000 --> 00:20:25,000
in Bangalore that will only get predicted.

361
00:20:25,000 --> 00:20:32,000
Okay, so this is the entire funda behind bidirectional RNN, right?

362
00:20:32,000 --> 00:20:34,000
I can go ahead and create my hidden layers over here.

363
00:20:35,000 --> 00:20:36,000
Right?

364
00:20:36,000 --> 00:20:37,000
It is very simple.

365
00:20:37,000 --> 00:20:38,000
What we are doing.

366
00:20:38,000 --> 00:20:40,000
We are combining things.

367
00:20:40,000 --> 00:20:41,000
We are just combining.

368
00:20:44,000 --> 00:20:45,000
Whatever output we are basically getting.

369
00:20:45,000 --> 00:20:50,000
If this is my output, I may apply a sigmoid activation function over here or this is my output.

370
00:20:50,000 --> 00:20:55,000
I may apply sigmoid activation function, but here I am getting the context of the previous word.

371
00:20:55,000 --> 00:20:58,000
And I'm also getting the context of the upcoming words.

372
00:20:58,000 --> 00:21:00,000
And based on that, I am able to do the prediction.

373
00:21:01,000 --> 00:21:08,000
So if I know Bangalore, if I know Krish, if I know Paris, if I know Krish, then this RNN will be

374
00:21:08,000 --> 00:21:13,000
able to perform well when compared to the simple RNN and LSTM RNN.

375
00:21:13,000 --> 00:21:17,000
And that is the biggest difference with respect to the simple RNN, LSTM, RNN.

376
00:21:18,000 --> 00:21:23,000
Now one assignment that uh, I definitely want to give it to you.

377
00:21:23,000 --> 00:21:29,000
Okay I know it is little bit difficult assignment I'll give you you need to come up.

378
00:21:29,000 --> 00:21:30,000
Just keep on exploring.

379
00:21:30,000 --> 00:21:31,000
See the research paper okay?

380
00:21:31,000 --> 00:21:36,000
You need to come up with forward propagation.

381
00:21:36,000 --> 00:21:40,000
Forward propagation equations.

382
00:21:40,000 --> 00:21:44,000
Like how we wrote the equation for LSTM and simple RNN.

383
00:21:44,000 --> 00:21:50,000
And similarly what will be the basic equations for this forward propagation I need to know this first

384
00:21:50,000 --> 00:21:50,000
of all okay.

385
00:21:50,000 --> 00:21:55,000
So here you will be able to see I am combining this with this.

386
00:21:55,000 --> 00:21:55,000
Right.

387
00:21:55,000 --> 00:21:59,000
So this will also be there will be some weights right which I'm actually passing.

388
00:21:59,000 --> 00:22:02,000
This will be my X11 I'll be passing to this okay.

389
00:22:02,000 --> 00:22:04,000
It will not be y of I over here w of I.

390
00:22:04,000 --> 00:22:06,000
But this will get connected.

391
00:22:06,000 --> 00:22:08,000
So here I am passing this input.

392
00:22:08,000 --> 00:22:13,000
Also I'm passing this input also along with this this input I will be passing over here.

393
00:22:14,000 --> 00:22:14,000
Right.

394
00:22:14,000 --> 00:22:17,000
This is the same words input I'm actually giving.

395
00:22:17,000 --> 00:22:21,000
That basically means that the same word which I'm passing over here, that word will also go over here.

396
00:22:21,000 --> 00:22:22,000
Right.

397
00:22:22,000 --> 00:22:26,000
So just see what will basically happen with respect to every output.

398
00:22:26,000 --> 00:22:31,000
What will happen if I'm getting 01020304.

399
00:22:31,000 --> 00:22:33,000
Because this is also getting combined to this.

400
00:22:33,000 --> 00:22:33,000
Right?

401
00:22:33,000 --> 00:22:37,000
So just come up with the forward propagation equation.

402
00:22:37,000 --> 00:22:41,000
See research papers come with different different ways and try to do this okay.

403
00:22:41,000 --> 00:22:43,000
This will basically be an assignment to understand.

404
00:22:43,000 --> 00:22:47,000
But this is how your entire bidirectional RNN actually works.

405
00:22:47,000 --> 00:22:48,000
Okay.

406
00:22:48,000 --> 00:22:54,000
So I hope you're able to understand, uh, I tried to simplify this as much as I could, but again,

407
00:22:54,000 --> 00:23:00,000
the main aim is over here that we need to provide the forward context to the output for doing the prediction.

408
00:23:00,000 --> 00:23:03,000
It will actually help us to do the proper prediction.

409
00:23:03,000 --> 00:23:04,000
So yes, this was it for my side.

410
00:23:04,000 --> 00:23:06,000
I will see you all in the next video.

411
00:23:06,000 --> 00:23:06,000
Have a great day.

412
00:23:06,000 --> 00:23:07,000
Thank you.

413
00:23:07,000 --> 00:23:07,000
Take care.