1
00:00:00,000 --> 00:00:00,000
Hello guys.

2
00:00:00,000 --> 00:00:05,000
Now we are going to create our simple RNN neural network.

3
00:00:05,000 --> 00:00:09,000
And then we will go ahead with training this entire training data.

4
00:00:09,000 --> 00:00:13,000
And then we'll be seeing that how we can also apply early stopping in this.

5
00:00:13,000 --> 00:00:17,000
And uh yes there will be lot many things that we are going to do right now.

6
00:00:17,000 --> 00:00:18,000
So let's go ahead.

7
00:00:18,000 --> 00:00:25,000
And now, first of all, in order to create our simple RNN, uh, we will go ahead and create model

8
00:00:25,000 --> 00:00:27,000
dot uh model equals to sequential model.

9
00:00:27,000 --> 00:00:32,000
So this is the first thing that I will be creating, as you know, that in my sequential model, the

10
00:00:32,000 --> 00:00:35,000
first thing that I'm actually going to add is something called as embedding layer.

11
00:00:35,000 --> 00:00:38,000
So in order to add I'll just go ahead and write model dot add.

12
00:00:39,000 --> 00:00:42,000
And here I'm going to call my embedding layer.

13
00:00:42,000 --> 00:00:48,000
Inside this embedding layer I have to give my vocabulary size which is initialized in this max underscore

14
00:00:48,000 --> 00:00:49,000
features.

15
00:00:49,000 --> 00:00:54,000
And you know that uh, I also have to use some feature dimension for my feature representation in the

16
00:00:54,000 --> 00:00:55,000
embedding layer.

17
00:00:55,000 --> 00:01:03,000
Let's consider that I'm going to use 128 features, uh, you know, for, for, for, uh, our dimension.

18
00:01:03,000 --> 00:01:04,000
Right.

19
00:01:04,000 --> 00:01:08,000
Like how in the embedding we used some dimension over here.

20
00:01:08,000 --> 00:01:08,000
Right?

21
00:01:08,000 --> 00:01:13,000
We, we represented ten, but since we have a very big data set, I will just go ahead and represent

22
00:01:13,000 --> 00:01:15,000
it to 128.

23
00:01:15,000 --> 00:01:17,000
You can probably put different numbers.

24
00:01:17,000 --> 00:01:18,000
Also, it is up to you.

25
00:01:18,000 --> 00:01:26,000
Okay then finally I will also go ahead and give my input underscore length okay.

26
00:01:26,000 --> 00:01:29,000
And this will be nothing but my max length okay.

27
00:01:29,000 --> 00:01:32,000
Max length which I have already created.

28
00:01:33,000 --> 00:01:35,000
This is nothing but the length of the sentence.

29
00:01:35,000 --> 00:01:37,000
So this is nothing.

30
00:01:37,000 --> 00:01:39,000
But it is our embedding layer.

31
00:01:39,000 --> 00:01:45,000
And this is responsible for taking all the words, converting that particular word into a vector of

32
00:01:45,000 --> 00:01:46,000
this dimensions.

33
00:01:46,000 --> 00:01:49,000
Okay, that is mentioned over here.

34
00:01:49,000 --> 00:01:53,000
Then we will go ahead and add a simple RNN.

35
00:01:53,000 --> 00:01:57,000
So for that I will just go ahead and use simple RNN.

36
00:01:57,000 --> 00:01:59,000
This is also present in the layer section.

37
00:01:59,000 --> 00:02:04,000
And then inside the simple RNN let's say I'm going to use 128 neurons.

38
00:02:04,000 --> 00:02:06,000
So this is the number that I have come up with.

39
00:02:06,000 --> 00:02:08,000
You can select any other numbers.

40
00:02:08,000 --> 00:02:13,000
And I've also shown you an an how you can go ahead and find out the optimal numbers.

41
00:02:13,000 --> 00:02:13,000
Right.

42
00:02:13,000 --> 00:02:17,000
And here I'm going to use the activation function like ReLU okay.

43
00:02:18,000 --> 00:02:26,000
Finally we'll go ahead and add a dense layer because I need to have only one output.

44
00:02:26,000 --> 00:02:32,000
And my activation function will be over here as sigmoid.

45
00:02:32,000 --> 00:02:37,000
Okay, so these are all the things that is available over here.

46
00:02:37,000 --> 00:02:41,000
Uh, then uh, we will go ahead and execute it.

47
00:02:41,000 --> 00:02:48,000
Once we execute it, you will be able to see my model summary, see if I have a multi-class classification

48
00:02:48,000 --> 00:02:52,000
in the output layer, that many number of nodes, how many number of outputs you specifically have or

49
00:02:52,000 --> 00:02:56,000
want to have, and the activation function can be changed to softmax.

50
00:02:56,000 --> 00:02:57,000
That's it.

51
00:02:57,000 --> 00:02:59,000
Only that is the change that you really need to do.

52
00:02:59,000 --> 00:03:01,000
Now if I go ahead and see the model summary.

53
00:03:01,000 --> 00:03:03,000
So this is how the model summary looks like.

54
00:03:03,000 --> 00:03:06,000
And these are the total number of parameters.

55
00:03:06,000 --> 00:03:06,000
Okay.

56
00:03:07,000 --> 00:03:20,000
Uh now quickly we will go ahead and create an instance of early stop stopping callback.

57
00:03:20,000 --> 00:03:21,000
Right.

58
00:03:21,000 --> 00:03:23,000
So let's go ahead and do this.

59
00:03:23,000 --> 00:03:38,000
So first of all I need to import from TensorFlow dot Keras dot callbacks import early stopping okay.

60
00:03:38,000 --> 00:03:44,000
Then I'm going to initialize this early stopping and I'm going to monitor.

61
00:03:44,000 --> 00:03:47,000
So I will just go ahead and use this early stopping.

62
00:03:47,000 --> 00:03:51,000
I'm going to monitor my validation loss.

63
00:03:51,000 --> 00:03:53,000
So I'll go ahead and write Val underscore loss.

64
00:03:54,000 --> 00:04:00,000
And here I'm going to use my patients let's say the patients I'm going to use it for five epochs.

65
00:04:01,000 --> 00:04:09,000
And I'm going to say hey whenever you find the best uh best, uh, validation loss or, uh, best accuracy.

66
00:04:09,000 --> 00:04:11,000
One, you can restore the weights from there.

67
00:04:11,000 --> 00:04:16,000
So for that, I'll just go ahead and restore best weights to true.

68
00:04:17,000 --> 00:04:23,000
So this is done and this I will initialize it to my variable which is called as early stopping.

69
00:04:24,000 --> 00:04:29,000
So let's go ahead and execute this I'll go ahead and write early stopping.

70
00:04:29,000 --> 00:04:31,000
And this is my values.

71
00:04:31,000 --> 00:04:31,000
Okay.

72
00:04:33,000 --> 00:04:39,000
Now finally train the model with early stopping.

73
00:04:40,000 --> 00:04:40,000
Okay.

74
00:04:40,000 --> 00:04:44,000
So here I'm just going to go ahead and write model dot fit.

75
00:04:44,000 --> 00:04:51,000
And inside this let's go ahead and write my x underscore train y underscore train.

76
00:04:51,000 --> 00:04:55,000
And then I'm going to use an epochs of ten.

77
00:04:55,000 --> 00:05:03,000
And finally you'll be able to see that my batch size, the batch size that I am actually going to use

78
00:05:03,000 --> 00:05:06,000
over here is 32, let's say.

79
00:05:06,000 --> 00:05:12,000
And finally I have my validation underscore split.

80
00:05:12,000 --> 00:05:17,000
Let's say I'll say use 20% of that in the validation underscore split.

81
00:05:17,000 --> 00:05:19,000
And finally I have my callbacks.

82
00:05:21,000 --> 00:05:26,000
And this callbacks I can assign it to my early stopping okay.

83
00:05:26,000 --> 00:05:33,000
So whichever we have initialized and all these things we will keep it in history, uh, the model history

84
00:05:33,000 --> 00:05:33,000
itself.

85
00:05:33,000 --> 00:05:36,000
And then we will go ahead and execute it.

86
00:05:36,000 --> 00:05:38,000
So here you can see some error.

87
00:05:38,000 --> 00:05:39,000
You must compile.

88
00:05:39,000 --> 00:05:42,000
Okay I forgot to compile things over here.

89
00:05:42,000 --> 00:05:43,000
The reason is very simple.

90
00:05:44,000 --> 00:05:45,000
Uh, okay.

91
00:05:45,000 --> 00:05:49,000
After this model dot summary, I will just go ahead and compile it.

92
00:05:49,000 --> 00:05:58,000
The reason of compiling is that because I need to assign what what is the optimizer function that I

93
00:05:58,000 --> 00:05:59,000
need to use.

94
00:05:59,000 --> 00:06:00,000
So I will just go ahead and write.

95
00:06:00,000 --> 00:06:03,000
My optimizer is equal to Adam okay.

96
00:06:03,000 --> 00:06:13,000
And my loss since loss over here is the loss that I'm actually going to use is for a binary classification.

97
00:06:13,000 --> 00:06:18,000
So I will just go ahead and write binary underscore cross entropy okay.

98
00:06:18,000 --> 00:06:23,000
And you can go ahead and see the Keras APIs if fit for multi-class classification this will basically

99
00:06:23,000 --> 00:06:27,000
become sparse categorical cross entropy and metrics.

100
00:06:27,000 --> 00:06:30,000
I'm just going to use my accuracy.

101
00:06:31,000 --> 00:06:32,000
Done.

102
00:06:32,000 --> 00:06:34,000
Now we'll go ahead and use my early stopping.

103
00:06:34,000 --> 00:06:36,000
And finally let's go ahead and train it.

104
00:06:37,000 --> 00:06:41,000
Now this is obviously going to take some amount of time if you do not have a powerful machine.

105
00:06:41,000 --> 00:06:46,000
So I would suggest please go ahead and do it in Google Colab.

106
00:06:46,000 --> 00:06:50,000
I'm doing this in my local because I definitely have a very powerful workstation.

107
00:06:50,000 --> 00:06:54,000
I have, uh, Nvidia RTX GPUs.

108
00:06:54,000 --> 00:06:59,000
Uh, so that is the reason I will be able to train it with, uh, with a good speed, because I have

109
00:06:59,000 --> 00:07:01,000
over here 25,000 records.

110
00:07:01,000 --> 00:07:02,000
Right.

111
00:07:02,000 --> 00:07:07,000
So we will wait, uh, and we will wait till all the epochs gets completed.

112
00:07:07,000 --> 00:07:13,000
And then once this training is done, we will save this specific model and we'll see whether the accuracy

113
00:07:13,000 --> 00:07:14,000
is increasing or not.

114
00:07:14,000 --> 00:07:16,000
So here you can see loss is decreasing.

115
00:07:16,000 --> 00:07:18,000
Uh, accuracy is also increasing.

116
00:07:18,000 --> 00:07:23,000
So I think after we complete this we'll be able to get a good, uh, accuracy.

117
00:07:23,000 --> 00:07:30,000
And after that, uh, we will go ahead and, uh, create, uh, Streamlit web app, and then we will

118
00:07:30,000 --> 00:07:35,000
try to see that how, uh, the prediction is basically done, you know, in as an end to end project.

119
00:07:35,000 --> 00:07:37,000
So right now epoch two is going on.

120
00:07:37,000 --> 00:07:38,000
It will take some time.

121
00:07:38,000 --> 00:07:39,000
So let's see.

122
00:07:39,000 --> 00:07:41,000
Let's wait for some time till this happens.

123
00:07:42,000 --> 00:07:42,000
Okay.

124
00:07:42,000 --> 00:07:43,000
I'll see you in some time.