1
00:00:00,000 --> 00:00:00,000
Hello guys.

2
00:00:00,000 --> 00:00:04,000
So we are going to continue our discussion with respect to NLP in Deep learning.

3
00:00:04,000 --> 00:00:07,000
In my previous video I was asking you a question.

4
00:00:07,000 --> 00:00:11,000
Can we use A and N to solve this problem with respect to the sequential data?

5
00:00:11,000 --> 00:00:11,000
Right?

6
00:00:11,000 --> 00:00:17,000
Now when I said that I know you may have got confused like what kind of problem statement we are solving.

7
00:00:17,000 --> 00:00:19,000
So let us go ahead and let us see the problem statement okay.

8
00:00:19,000 --> 00:00:26,000
So let's consider that, uh, I have a data set and this is my data set over here.

9
00:00:27,000 --> 00:00:32,000
And with respect to this particular data set, let's say I have something called as text.

10
00:00:32,000 --> 00:00:34,000
And this is my output okay.

11
00:00:34,000 --> 00:00:41,000
The problem statement that we are solving is something called as sentiment analysis okay.

12
00:00:42,000 --> 00:00:49,000
Sentiment analysis let's say one of the data set is like the sentence is something like this.

13
00:00:49,000 --> 00:00:52,000
The food is good okay.

14
00:00:53,000 --> 00:00:55,000
And let's say here I have one.

15
00:00:55,000 --> 00:01:02,000
The next statement that I have or next sentence in my data set, I'm writing the food is bad.

16
00:01:02,000 --> 00:01:06,000
And here I have 0 or 1 basically means good.

17
00:01:06,000 --> 00:01:08,000
Zero basically means bad okay.

18
00:01:08,000 --> 00:01:11,000
And then third one that I have is nothing.

19
00:01:11,000 --> 00:01:15,000
But the food is not good.

20
00:01:16,000 --> 00:01:17,000
I have zero over here.

21
00:01:18,000 --> 00:01:23,000
Let's say that in my training data set, I have this all sentence.

22
00:01:23,000 --> 00:01:24,000
Okay.

23
00:01:24,000 --> 00:01:28,000
Now, how do I solve this with the help of an.

24
00:01:29,000 --> 00:01:29,000
Yes.

25
00:01:29,000 --> 00:01:32,000
Can we solve it with the help of artificial neural network?

26
00:01:32,000 --> 00:01:33,000
Right.

27
00:01:33,000 --> 00:01:36,000
So let's consider that over here.

28
00:01:36,000 --> 00:01:37,000
First of all we will go ahead and see.

29
00:01:37,000 --> 00:01:45,000
And again, uh, first step in this is that regarding your text pre-processing, if I really want to

30
00:01:45,000 --> 00:01:47,000
solve it with the help of an.

31
00:01:47,000 --> 00:01:54,000
So in a text pre-processing, what we do is that we have to convert our text into vectors.

32
00:01:54,000 --> 00:02:00,000
And we have seen lot of various ways like bag of word, one hot encoding TF-IDF.

33
00:02:00,000 --> 00:02:00,000
And.

34
00:02:00,000 --> 00:02:00,000
All right.

35
00:02:01,000 --> 00:02:04,000
So let's consider that I take this particular data set okay.

36
00:02:04,000 --> 00:02:08,000
And I, I will first of all go ahead and design my neural network also okay.

37
00:02:08,000 --> 00:02:10,000
And that is artificial neural network.

38
00:02:10,000 --> 00:02:14,000
But before that let's see that how many unique words I have in this.

39
00:02:14,000 --> 00:02:22,000
Let's say I have one unique word two, three, four, five and six okay.

40
00:02:22,000 --> 00:02:24,000
I have six unique words.

41
00:02:24,000 --> 00:02:24,000
Okay.

42
00:02:24,000 --> 00:02:29,000
And in this I will not consider is because I consider this as a stop word okay.

43
00:02:29,000 --> 00:02:35,000
So I'm not just going to consider is so over here the number of unique word is 1234, five.

44
00:02:35,000 --> 00:02:39,000
I'll also not consider the because this is a stop word okay.

45
00:02:39,000 --> 00:02:41,000
So let me just go ahead and use this.

46
00:02:41,000 --> 00:02:46,000
All remaining words 1234 okay so four words I have.

47
00:02:46,000 --> 00:02:55,000
So that basically means my vocabulary size vocabulary in this particular case is for vocabulary size

48
00:02:55,000 --> 00:02:55,000
Okay.

49
00:02:56,000 --> 00:02:58,000
Now in this vocabulary size I have four.

50
00:02:58,000 --> 00:03:02,000
That basically means my entire corpus word corpus is nothing but four.

51
00:03:02,000 --> 00:03:03,000
Okay.

52
00:03:03,000 --> 00:03:09,000
So as you all know, if I want to convert sentence by sentence, uh, from text to vectors.

53
00:03:09,000 --> 00:03:12,000
So you have something like bag of words, right?

54
00:03:12,000 --> 00:03:13,000
In bag of words.

55
00:03:13,000 --> 00:03:15,000
What I, what what we actually do.

56
00:03:15,000 --> 00:03:16,000
Right.

57
00:03:16,000 --> 00:03:20,000
Uh, we basically try to convert every sentence into a vector.

58
00:03:20,000 --> 00:03:20,000
Right.

59
00:03:20,000 --> 00:03:21,000
How do we convert it?

60
00:03:21,000 --> 00:03:24,000
Let's say over here I have all these words.

61
00:03:24,000 --> 00:03:26,000
So I'll take this food.

62
00:03:26,000 --> 00:03:27,000
Good.

63
00:03:27,000 --> 00:03:28,000
Right.

64
00:03:28,000 --> 00:03:33,000
So I'll say food good bad and not okay bad and not.

65
00:03:34,000 --> 00:03:39,000
And with respect to sentence one, what we do is that wherever food is present right in this particular

66
00:03:39,000 --> 00:03:42,000
sentence, I will make that as one and remaining all as zero.

67
00:03:42,000 --> 00:03:47,000
So food and good where it is present, that will be one one and remaining all will be zero.

68
00:03:47,000 --> 00:03:48,000
So here for sentence one.

69
00:03:48,000 --> 00:03:51,000
If I convert this into vectors, it will be 1100.

70
00:03:51,000 --> 00:03:52,000
Okay.

71
00:03:52,000 --> 00:03:56,000
Then in the sentence two you will be able to see over here the foot.

72
00:03:56,000 --> 00:03:58,000
Foot is there bad is there.

73
00:03:58,000 --> 00:03:59,000
So this will basically become one.

74
00:03:59,000 --> 00:04:02,000
The remaining all will become zero and this will become one and this will be zero.

75
00:04:02,000 --> 00:04:06,000
Okay, I'm just using the technique of bag of words to convert text into vectors.

76
00:04:06,000 --> 00:04:10,000
Similarly, the third one is saying that the food is not good, right?

77
00:04:10,000 --> 00:04:13,000
So food is not good with respect to sentence three.

78
00:04:13,000 --> 00:04:16,000
So I have this one not is there and good is there.

79
00:04:17,000 --> 00:04:20,000
So this three will basically be having one.

80
00:04:20,000 --> 00:04:25,000
So this is just a simple way of converting a word into vectors right.

81
00:04:25,000 --> 00:04:30,000
And here you can see that I have some fixed fixed vector size.

82
00:04:31,000 --> 00:04:33,000
Right I have this fixed vector size.

83
00:04:33,000 --> 00:04:34,000
So this is one way.

84
00:04:34,000 --> 00:04:34,000
Okay.

85
00:04:35,000 --> 00:04:42,000
Now one very important information that we can see from this particular sentences and the text that

86
00:04:42,000 --> 00:04:44,000
we have converted that into vectors okay.

87
00:04:44,000 --> 00:04:49,000
One is like when I probably create my artificial neural network okay.

88
00:04:50,000 --> 00:04:55,000
Here you will be able to see this are my inputs that I have to give to my neural network.

89
00:04:55,000 --> 00:04:56,000
Right.

90
00:04:56,000 --> 00:04:59,000
These are my inputs that I give to my neural network.

91
00:04:59,000 --> 00:05:05,000
And since these all are text data, these all are text data.

92
00:05:05,000 --> 00:05:09,000
Text data basically means the sequence information is important.

93
00:05:10,000 --> 00:05:15,000
Sequence information is important.

94
00:05:15,000 --> 00:05:20,000
Again I'm stressing on this, but here do you think this all words are in sequence.

95
00:05:20,000 --> 00:05:23,000
No Food is coming fast based on the frequency.

96
00:05:23,000 --> 00:05:25,000
Food is over here.

97
00:05:25,000 --> 00:05:27,000
Maximum number of time food is present.

98
00:05:27,000 --> 00:05:27,000
Right?

99
00:05:27,000 --> 00:05:31,000
So food is basically related and first, then good you have, then bad you have, then not you have.

100
00:05:31,000 --> 00:05:34,000
So these words are exactly not in sequence.

101
00:05:35,000 --> 00:05:35,000
Right.

102
00:05:35,000 --> 00:05:40,000
And when we use this particular technique and we when we try to train it with a artificial neural network,

103
00:05:40,000 --> 00:05:43,000
first of all we are using that sequence over here.

104
00:05:43,000 --> 00:05:48,000
When we lose the sequence that basically means the meaning of the sentence is lost.

105
00:05:50,000 --> 00:05:55,000
Meaning of the sentence is lost because the context information is lost, right?

106
00:05:56,000 --> 00:06:01,000
So that is the reason why I say the sequential information is super important and we are not able to

107
00:06:01,000 --> 00:06:02,000
manage this.

108
00:06:02,000 --> 00:06:04,000
This is the major one problem here.

109
00:06:04,000 --> 00:06:10,000
If I go ahead and use even use tf IDF write in tf IDF, I may do the calculation and I may get different

110
00:06:10,000 --> 00:06:11,000
different values, right?

111
00:06:12,000 --> 00:06:18,000
Uh, even in word two vec, if I use word two vec, I may be able to fix this into some dimensions,

112
00:06:18,000 --> 00:06:19,000
right?

113
00:06:19,000 --> 00:06:22,000
Just by considering every word will be converted into one dimensions, right?

114
00:06:22,000 --> 00:06:28,000
The dimension size may be 300, it may be different, different sizes, but at the end of the day,

115
00:06:28,000 --> 00:06:31,000
we are losing some sequence information right over here.

116
00:06:32,000 --> 00:06:34,000
Now very much important thing.

117
00:06:34,000 --> 00:06:37,000
If I'm training my neural network, right.

118
00:06:37,000 --> 00:06:39,000
Let's say this is my Ann.

119
00:06:39,000 --> 00:06:43,000
Let's say over here, I can see that the maximum number of words are four.

120
00:06:43,000 --> 00:06:47,000
So I will give my input size as one, two, three, four.

121
00:06:47,000 --> 00:06:48,000
Right.

122
00:06:48,000 --> 00:06:50,000
I go ahead and create my neural network.

123
00:06:50,000 --> 00:06:53,000
Let's say in my neural net hidden layer I have three neurons.

124
00:06:53,000 --> 00:06:57,000
And finally, since I have just one output, this basically is connected.

125
00:06:57,000 --> 00:07:00,000
This input layer is connected to all the hidden neurons.

126
00:07:00,000 --> 00:07:03,000
This is basically connected to this particular output layer.

127
00:07:03,000 --> 00:07:04,000
Right.

128
00:07:04,000 --> 00:07:06,000
And this is also connected to every hidden neurons.

129
00:07:07,000 --> 00:07:10,000
This is also connected to every hidden neurons.

130
00:07:11,000 --> 00:07:11,000
Right.

131
00:07:12,000 --> 00:07:15,000
So this here you can see that hey fine.

132
00:07:15,000 --> 00:07:20,000
This is my how my artificial neural network looks like the first problem you have already discussed.

133
00:07:20,000 --> 00:07:22,000
That sequence information is getting lost.

134
00:07:22,000 --> 00:07:27,000
Okay, now second problem here with respect to the text, right.

135
00:07:27,000 --> 00:07:28,000
The food is good.

136
00:07:28,000 --> 00:07:28,000
Yeah.

137
00:07:28,000 --> 00:07:34,000
I will be sending all this information like let's say my first sentence is basically 1100.

138
00:07:34,000 --> 00:07:38,000
So I'll be sending my 1100 over here.

139
00:07:38,000 --> 00:07:39,000
Right.

140
00:07:39,000 --> 00:07:44,000
This this is what is the entire information that I'll be sending over here right while training.

141
00:07:45,000 --> 00:07:47,000
This will be my input okay.

142
00:07:47,000 --> 00:07:47,000
Okay.

143
00:07:50,000 --> 00:07:50,000
Just a second.

144
00:07:54,000 --> 00:07:58,000
So this will be my input that is basically getting sent okay.

145
00:07:58,000 --> 00:08:01,000
And here you can see this is my input layer.

146
00:08:02,000 --> 00:08:04,000
And this is my hidden layer one.

147
00:08:04,000 --> 00:08:05,000
And this is my output layer.

148
00:08:06,000 --> 00:08:07,000
And finally I get my y hat.

149
00:08:07,000 --> 00:08:14,000
Now what is one problem over here that you see in case of whenever we work with text data.

150
00:08:17,000 --> 00:08:26,000
It is always a good idea that whenever I have sequence of information, whenever I have sequence of

151
00:08:26,000 --> 00:08:30,000
information, let's say over here, I go ahead and write.

152
00:08:30,000 --> 00:08:31,000
The food is good.

153
00:08:34,000 --> 00:08:42,000
To solve any use cases, we have to give one word at a time, one word at a time.

154
00:08:42,000 --> 00:08:46,000
So let's say during timestamp one I just give this the word.

155
00:08:46,000 --> 00:08:49,000
Then forward propagation should happen and backward propagation two should happen.

156
00:08:49,000 --> 00:08:52,000
Then in my second timestamp the next word is food.

157
00:08:53,000 --> 00:08:55,000
I have to give that in my third timestamp.

158
00:08:55,000 --> 00:08:57,000
The next word is is.

159
00:08:57,000 --> 00:08:58,000
Then in the next timestamp.

160
00:08:58,000 --> 00:09:00,000
My next word is good.

161
00:09:00,000 --> 00:09:06,000
So it is always a good idea that whenever you have sequential data, you really need to give based on

162
00:09:06,000 --> 00:09:08,000
a specific timestamp, not all at one.

163
00:09:08,000 --> 00:09:14,000
Now, in this particular scenario, I'm giving this entire sentence one all the words all at once as

164
00:09:14,000 --> 00:09:15,000
my input.

165
00:09:15,000 --> 00:09:16,000
So here also input is going here.

166
00:09:16,000 --> 00:09:17,000
Also input is going here.

167
00:09:17,000 --> 00:09:18,000
Also input is going here.

168
00:09:18,000 --> 00:09:19,000
Here also input is going.

169
00:09:19,000 --> 00:09:25,000
And by this the sequence information will also not be maintained.

170
00:09:25,000 --> 00:09:25,000
Right.

171
00:09:25,000 --> 00:09:31,000
And that is the reason the neural network will not be able to efficiently train themselves.

172
00:09:31,000 --> 00:09:32,000
Right?

173
00:09:32,000 --> 00:09:38,000
In the case of sequence of data, sequential data, it is always a good idea that we pass one word at

174
00:09:38,000 --> 00:09:40,000
a time to our neural network.

175
00:09:41,000 --> 00:09:41,000
Right?

176
00:09:42,000 --> 00:09:44,000
And this is a major problem, right?

177
00:09:44,000 --> 00:09:49,000
Because just by using A and n what we are doing, we are just directly giving all the information.

178
00:09:49,000 --> 00:09:52,000
Then we do forward propagation, then we do backward propagation.

179
00:09:53,000 --> 00:09:53,000
And why?

180
00:09:53,000 --> 00:09:58,000
I'm saying you should not give all the words at all at once, because let's say that if I want to solve

181
00:09:58,000 --> 00:10:03,000
a problem of, uh, uh, language translation, in the case of language translation, I have to give

182
00:10:03,000 --> 00:10:05,000
one word by word, right?

183
00:10:05,000 --> 00:10:07,000
If you know about Google translation, right.

184
00:10:07,000 --> 00:10:15,000
If you have used Google Translation, then let's say I want to convert from Hindi to English, or let's

185
00:10:15,000 --> 00:10:16,000
say English to French.

186
00:10:18,000 --> 00:10:23,000
As soon as I give one word right, let's say I say the, then it should automatically give you that

187
00:10:23,000 --> 00:10:25,000
French the word right.

188
00:10:25,000 --> 00:10:30,000
Then if I keep on adding words, then it should similarly go ahead and translate in that way.

189
00:10:30,000 --> 00:10:35,000
So it is always necessary that you provide this information in a sequential way based on the time stamp.

190
00:10:35,000 --> 00:10:37,000
Not all at once.

191
00:10:37,000 --> 00:10:39,000
So this is one more major problem.

192
00:10:39,000 --> 00:10:43,000
And that is the reason we cannot just directly use A and N are two important information.

193
00:10:43,000 --> 00:10:46,000
First of all, it is not able to maintain the entire sequence.

194
00:10:46,000 --> 00:10:50,000
And the second thing is that it is giving all the words to the neural network all at once for a specific

195
00:10:50,000 --> 00:10:51,000
sentence.

196
00:10:51,000 --> 00:10:51,000
Okay.

197
00:10:52,000 --> 00:10:59,000
And that is where we will be using a new kind of neural network, which was proposed by researcher.

198
00:10:59,000 --> 00:11:01,000
That is nothing but a simple RNN.

199
00:11:02,000 --> 00:11:05,000
So simple RNN will basically be used.

200
00:11:05,000 --> 00:11:11,000
And if you don't know the full form of RNN, it is nothing but recurrent neural network.

201
00:11:12,000 --> 00:11:16,000
So we will be using recurrent neural network to solve this problem.

202
00:11:18,000 --> 00:11:24,000
So here in the case of neural network our data will be sequential.

203
00:11:27,000 --> 00:11:33,000
Now, just to give you a brief idea, how does a neural recurrent neural network look like?

204
00:11:33,000 --> 00:11:35,000
Okay, so first of all I will go ahead and take an.

205
00:11:35,000 --> 00:11:41,000
So in the case of an I will be having a input.

206
00:11:42,000 --> 00:11:47,000
So let's say this is my input and this will get as an output okay.

207
00:11:47,000 --> 00:11:49,000
So this is my hidden layer.

208
00:11:50,000 --> 00:11:51,000
This is my hidden layer.

209
00:11:51,000 --> 00:11:52,000
Okay.

210
00:11:52,000 --> 00:11:56,000
Now with respect to this I'm giving my input over here.

211
00:11:56,000 --> 00:11:57,000
Some processing will happen in the hidden layer.

212
00:11:57,000 --> 00:11:58,000
And finally I'll get the output.

213
00:11:58,000 --> 00:12:03,000
If I want to expand this, let's say in my input layer I have two inputs.

214
00:12:03,000 --> 00:12:05,000
Let's say this is my two inputs.

215
00:12:05,000 --> 00:12:06,000
Let's say in my hidden layer.

216
00:12:06,000 --> 00:12:10,000
This is my three, uh, three neurons in my hidden layer.

217
00:12:10,000 --> 00:12:12,000
And finally I have my output layer.

218
00:12:12,000 --> 00:12:14,000
So I can also represent this in this way.

219
00:12:14,000 --> 00:12:18,000
So this will be connected over here to all the neurons.

220
00:12:18,000 --> 00:12:20,000
This will be connected to all the neurons over here.

221
00:12:21,000 --> 00:12:22,000
Right.

222
00:12:22,000 --> 00:12:26,000
And finally this will be connected to my output layer okay.

223
00:12:26,000 --> 00:12:29,000
So this is my output layer and this is my input layer.

224
00:12:31,000 --> 00:12:32,000
This is my input layer.

225
00:12:32,000 --> 00:12:36,000
So here this representation that I have actually shown is nothing but this.

226
00:12:36,000 --> 00:12:38,000
Only I have my input, I have my output.

227
00:12:38,000 --> 00:12:41,000
And in middle I have this hidden layers with hidden neurons.

228
00:12:41,000 --> 00:12:43,000
That is how A and N looks like.

229
00:12:43,000 --> 00:12:51,000
Okay, now in the case of RNN, how it is different from A and N and why it is suitable for you can

230
00:12:51,000 --> 00:12:53,000
just consider for a sequential data.

231
00:12:53,000 --> 00:12:58,000
Because if I go ahead and see the neural network now the neural network looks something like this.

232
00:12:58,000 --> 00:13:01,000
So initially I will be having my input layer.

233
00:13:02,000 --> 00:13:05,000
Then I have my hidden layer and then I have my output layer.

234
00:13:05,000 --> 00:13:11,000
This is perfectly fine, but along with this we also have a feedback loop.

235
00:13:12,000 --> 00:13:18,000
Okay, now what does this feedback loop basically says that whatever output I am getting from my hidden

236
00:13:18,000 --> 00:13:25,000
layer neurons, it will be sent to that particular neuron and to all the other neurons in that hidden

237
00:13:25,000 --> 00:13:25,000
layer.

238
00:13:26,000 --> 00:13:29,000
Okay, so if I go ahead and expand this now see see the magic?

239
00:13:29,000 --> 00:13:30,000
Okay.

240
00:13:30,000 --> 00:13:33,000
Now if I go ahead and expand this, this will look something like this.

241
00:13:33,000 --> 00:13:37,000
Let's say I have two input uh, two inputs.

242
00:13:37,000 --> 00:13:43,000
Then I have three hidden layer, sorry, three neurons in a hidden layer.

243
00:13:43,000 --> 00:13:45,000
And finally I have one output.

244
00:13:45,000 --> 00:13:46,000
Right.

245
00:13:46,000 --> 00:13:50,000
Because to solve this problem also, I'll be requiring this neural network itself to solve this kind

246
00:13:50,000 --> 00:13:52,000
of data set problem.

247
00:13:52,000 --> 00:13:52,000
Right.

248
00:13:52,000 --> 00:13:55,000
Because here I have this output as 100.

249
00:13:55,000 --> 00:13:55,000
Okay.

250
00:13:55,000 --> 00:14:01,000
So here you'll be able to see that I will be having one final output over here okay.

251
00:14:01,000 --> 00:14:05,000
Let's say we go ahead and connect this okay.

252
00:14:05,000 --> 00:14:07,000
We go ahead and connect this.

253
00:14:07,000 --> 00:14:08,000
Now this is important okay.

254
00:14:08,000 --> 00:14:10,000
Because I'm expanding it okay.

255
00:14:10,000 --> 00:14:15,000
So this is very simple expanding from input till the hidden layer is very simple.

256
00:14:15,000 --> 00:14:18,000
And connecting hidden to the output also is very simple.

257
00:14:18,000 --> 00:14:20,000
This is nothing, but this is my an only.

258
00:14:20,000 --> 00:14:22,000
Okay, this is my an only.

259
00:14:22,000 --> 00:14:24,000
Now, now see the magic?

260
00:14:24,000 --> 00:14:27,000
Okay, now see the magic.

261
00:14:28,000 --> 00:14:31,000
I will just go ahead and connect this.

262
00:14:31,000 --> 00:14:36,000
I will just go ahead and connect this okay.

263
00:14:36,000 --> 00:14:37,000
So let's connect this.

264
00:14:37,000 --> 00:14:41,000
So this become my input one input to this is my hidden layer.

265
00:14:41,000 --> 00:14:44,000
Now obviously Ann is looking similar like RNN.

266
00:14:44,000 --> 00:14:47,000
Now we have to think about this feedback loop.

267
00:14:47,000 --> 00:14:56,000
Now this feedback loop will be like this hidden layer will be sending its output back to its node okay.

268
00:14:56,000 --> 00:15:02,000
Not only that, this hidden layer will also send the feedback to this particular node.

269
00:15:02,000 --> 00:15:02,000
Along with this.

270
00:15:02,000 --> 00:15:05,000
This hidden layer will also send the feedback to this particular node.

271
00:15:06,000 --> 00:15:08,000
Okay, that is what it is saying.

272
00:15:08,000 --> 00:15:09,000
Sending the information.

273
00:15:09,000 --> 00:15:13,000
Sending the output back to itself, right to all the hidden layer it will be sending.

274
00:15:13,000 --> 00:15:13,000
Okay.

275
00:15:13,000 --> 00:15:18,000
Similarly, you will be able to see that the second hidden layer, a second neuron in the hidden layer

276
00:15:18,000 --> 00:15:22,000
will also sending in the loop, and it will also send to this hidden layer.

277
00:15:22,000 --> 00:15:25,000
Sorry to this hidden neuron along with this, to this hidden neuron.

278
00:15:25,000 --> 00:15:28,000
So we are just passing the information to each and every hidden neuron.

279
00:15:28,000 --> 00:15:34,000
And finally here also you have in this neuron here you have the self loop uh feedback loop.

280
00:15:34,000 --> 00:15:37,000
And then this information will also be sent to this.

281
00:15:37,000 --> 00:15:40,000
And it will also be sent to this.

282
00:15:40,000 --> 00:15:47,000
So in short, whatever output we are getting after, we probably take the inputs along with the weights

283
00:15:47,000 --> 00:15:48,000
that are initialized.

284
00:15:48,000 --> 00:15:50,000
Then we add a bias.

285
00:15:50,000 --> 00:15:52,000
Then we add an activation function.

286
00:15:52,000 --> 00:15:53,000
Whatever output is basically coming.

287
00:15:53,000 --> 00:15:56,000
We also share it with all the other hidden layers.

288
00:15:56,000 --> 00:15:57,000
Okay.

289
00:15:57,000 --> 00:16:00,000
All the all the other hidden neurons in the hidden layer.

290
00:16:00,000 --> 00:16:01,000
Okay.

291
00:16:01,000 --> 00:16:07,000
And the and with respect to this particular neural network, all the inputs that is passed, let's say

292
00:16:07,000 --> 00:16:08,000
this is my input.

293
00:16:09,000 --> 00:16:12,000
The food is good.

294
00:16:13,000 --> 00:16:14,000
The food is good.

295
00:16:15,000 --> 00:16:20,000
Here we will be doing some kind of numbering based on sentence.

296
00:16:20,000 --> 00:16:21,000
Okay.

297
00:16:21,000 --> 00:16:23,000
Sentence and words.

298
00:16:23,000 --> 00:16:25,000
Let's say this is my first sentence.

299
00:16:25,000 --> 00:16:30,000
So I will go ahead and note this as X11X11 basically means first sentence and first word.

300
00:16:30,000 --> 00:16:32,000
Then you have X12 first sentence and second word.

301
00:16:32,000 --> 00:16:37,000
Then you have a uh the first sentence and third word and first sentence and fourth word.

302
00:16:37,000 --> 00:16:44,000
In the case of RNN, we pass a input at each time stamp.

303
00:16:44,000 --> 00:16:46,000
That basically means that T is equal to one.

304
00:16:46,000 --> 00:16:48,000
We will be passing.

305
00:16:48,000 --> 00:16:50,000
We will be passing the input as X11.

306
00:16:51,000 --> 00:16:54,000
So here we'll be seeing at t is equal to one.

307
00:16:54,000 --> 00:16:56,000
We will be passing X11.

308
00:16:56,000 --> 00:17:00,000
Then as t is equal to two we will be passing X12.

309
00:17:00,000 --> 00:17:03,000
That basically means when X11 is basically passed okay.

310
00:17:03,000 --> 00:17:10,000
Whichever information it has with respect to the word that we are having right when it is passed, right.

311
00:17:10,000 --> 00:17:12,000
It goes to all these hidden neurons.

312
00:17:12,000 --> 00:17:18,000
It does the calculation and whatever output is there, it will be shared with all the other hidden neurons

313
00:17:18,000 --> 00:17:19,000
that are present in the hidden layer.

314
00:17:19,000 --> 00:17:24,000
Then in the T2 step, what we do is that we basically go ahead and write x.

315
00:17:24,000 --> 00:17:27,000
We go and pass this next word that is food.

316
00:17:27,000 --> 00:17:27,000
Okay.

317
00:17:27,000 --> 00:17:30,000
So here we initially pass the then food.

318
00:17:30,000 --> 00:17:31,000
Then food will be basically passed.

319
00:17:31,000 --> 00:17:33,000
All the operation will take place.

320
00:17:33,000 --> 00:17:36,000
Then teeth tree is equal to 3X13 will be passed.

321
00:17:36,000 --> 00:17:40,000
T is equal to 4X1 will will passed right.

322
00:17:40,000 --> 00:17:45,000
That is how each and every words are passed from a specific sentence based on timestamp.

323
00:17:46,000 --> 00:17:47,000
In an.

324
00:17:47,000 --> 00:17:52,000
We used to pass each and every thing at at the initial stage while doing the forward propagation.

325
00:17:52,000 --> 00:17:55,000
But in case of RNN, we just pass one word at a time.

326
00:17:56,000 --> 00:18:02,000
Okay, now in my next video here, I've just shown you the basic example of.

327
00:18:03,000 --> 00:18:05,000
Just to make you understand the difference between A and RNN.

328
00:18:05,000 --> 00:18:06,000
Okay.

329
00:18:06,000 --> 00:18:09,000
And if I want to probably expand this okay.

330
00:18:09,000 --> 00:18:10,000
Expand this.

331
00:18:10,000 --> 00:18:11,000
This will also look something like this.

332
00:18:11,000 --> 00:18:11,000
Okay.

333
00:18:11,000 --> 00:18:16,000
I'll just give you one more brief, uh, unfolding technique that we specifically use.

334
00:18:16,000 --> 00:18:17,000
It looks something like this.

335
00:18:17,000 --> 00:18:21,000
So this is my this is my neuron okay.

336
00:18:21,000 --> 00:18:22,000
This is my hidden layer.

337
00:18:23,000 --> 00:18:30,000
So now what happens in RNN we create a self-loop okay.

338
00:18:30,000 --> 00:18:31,000
Sorry.

339
00:18:31,000 --> 00:18:36,000
So this is nothing but this is a self-loop okay.

340
00:18:36,000 --> 00:18:38,000
Here also we pass some information okay.

341
00:18:38,000 --> 00:18:44,000
Now if we unfold this with respect to time stamp, understand this is my hidden layer okay.

342
00:18:44,000 --> 00:18:46,000
So here I will just go ahead and write like this.

343
00:18:46,000 --> 00:18:49,000
So this will be my input.

344
00:18:49,000 --> 00:18:50,000
This will be my output.

345
00:18:50,000 --> 00:18:52,000
And then I have this.

346
00:18:52,000 --> 00:18:53,000
Then this will go.

347
00:18:53,000 --> 00:18:57,000
This output will go to the next time stamp okay.

348
00:18:58,000 --> 00:19:00,000
Then again you have this and you have this.

349
00:19:00,000 --> 00:19:03,000
So that basically means during time stamp t is equal to one.

350
00:19:03,000 --> 00:19:07,000
Here I'm going to pass my X11 word of my sentence one.

351
00:19:07,000 --> 00:19:08,000
Sorry.

352
00:19:08,000 --> 00:19:11,000
My sentence one is something like this and my output is 0 or 1, right?

353
00:19:11,000 --> 00:19:12,000
Let's say this is one.

354
00:19:12,000 --> 00:19:14,000
So what I will do in my t is equal to one.

355
00:19:14,000 --> 00:19:19,000
I'll be passing X11 and then at t is equal to two I will be passing X12.

356
00:19:19,000 --> 00:19:24,000
At t is equal to three I will again be passing

357
00:19:26,000 --> 00:19:29,000
X13 okay.

358
00:19:29,000 --> 00:19:30,000
At t is equal to three.

359
00:19:30,000 --> 00:19:32,000
And finally at t is equal to four.

360
00:19:32,000 --> 00:19:34,000
I will be passing my last word.

361
00:19:34,000 --> 00:19:40,000
So based on the number of time stamps you'll be seeing that each and every word will be passed.

362
00:19:40,000 --> 00:19:41,000
Okay.

363
00:19:41,000 --> 00:19:43,000
Now what operation is going to happen that we'll discuss.

364
00:19:43,000 --> 00:19:44,000
Right.

365
00:19:44,000 --> 00:19:49,000
But here you can see that whatever output is coming, it is also being shared with all the hidden neuron.

366
00:19:49,000 --> 00:19:51,000
But at t is equal to two.

367
00:19:51,000 --> 00:19:55,000
I will be sending my second word and it will also have the first word context.

368
00:19:55,000 --> 00:20:01,000
Right this the entire RNN will have the first word context also, and that is how the previous context

369
00:20:01,000 --> 00:20:05,000
information will also be retained by the RNN.

370
00:20:05,000 --> 00:20:05,000
Right?

371
00:20:05,000 --> 00:20:08,000
Please observe this right at t is equal to one.

372
00:20:08,000 --> 00:20:09,000
I'm passing X11 right.

373
00:20:09,000 --> 00:20:13,000
So when we pass the first word all this proposition will happen.

374
00:20:13,000 --> 00:20:15,000
Let's consider that I'm just going to pass one neuron.

375
00:20:15,000 --> 00:20:17,000
So this will be only one input okay.

376
00:20:17,000 --> 00:20:18,000
Not two one input.

377
00:20:18,000 --> 00:20:24,000
So once I do this everything will be happening once uh the input will be multiplied by all the weights

378
00:20:24,000 --> 00:20:24,000
over here.

379
00:20:24,000 --> 00:20:26,000
Then the bias will be added.

380
00:20:26,000 --> 00:20:28,000
We'll talk about how the forward propagation will happen.

381
00:20:28,000 --> 00:20:31,000
But once I get this particular output, it needs to be shared with all the other neurons.

382
00:20:31,000 --> 00:20:34,000
So it has been shared with all the neurons over here.

383
00:20:34,000 --> 00:20:34,000
Right.

384
00:20:34,000 --> 00:20:38,000
All this information, all this output is basically shared with this particular neuron.

385
00:20:38,000 --> 00:20:44,000
Now this neuron based on the time stamp and this neuron is the same hidden neuron, right.

386
00:20:44,000 --> 00:20:46,000
Based on a time stamp T is equal to two.

387
00:20:46,000 --> 00:20:48,000
Has the previous information of the first word.

388
00:20:48,000 --> 00:20:51,000
That is, the right now in t is equal to timestamp two.

389
00:20:51,000 --> 00:20:54,000
I'm getting the second word food, so it has the previous context.

390
00:20:54,000 --> 00:20:55,000
It has the current context.

391
00:20:55,000 --> 00:20:57,000
And again this operation will take place.

392
00:20:57,000 --> 00:20:58,000
What kind of operation.

393
00:20:58,000 --> 00:21:00,000
I'll talk more about it as we go ahead.

394
00:21:00,000 --> 00:21:06,000
But this is just a brief idea to talk about how our how a simple RNN architecture looks like and how

395
00:21:06,000 --> 00:21:07,000
it is different from an.

396
00:21:08,000 --> 00:21:15,000
Okay, so, uh, in my next video, uh, I will break down this entire entire diagram again, okay.

397
00:21:15,000 --> 00:21:19,000
And show you like how detaily it will be looking like.

398
00:21:19,000 --> 00:21:23,000
And along with that, we'll discuss about this architecture more in a better way.

399
00:21:23,000 --> 00:21:25,000
This is an important architecture.

400
00:21:25,000 --> 00:21:28,000
So I will be taking more couple of classes to make you explain each and everything.

401
00:21:28,000 --> 00:21:29,000
Okay.

402
00:21:29,000 --> 00:21:31,000
So yes, this was it for my side.

403
00:21:31,000 --> 00:21:34,000
I will see you all in the next video, and we'll continue the discussion with respect to the architecture

404
00:21:34,000 --> 00:21:35,000
of RNN.

405
00:21:35,000 --> 00:21:36,000
Thank you.