So finally, guys, uh, I'm quite excited that we will finally train our LSTM, RNN, and, uh, this,
uh, step by step will see again how to probably go ahead and train it already.
We have done the data preprocessing and data ingestion part.
Uh, now let's go ahead and do this.
So first of all I will go ahead and import from TensorFlow okay.
Dot Keras dot models import sequential okay.
So we are going to specifically use sequential.
Along with this I will go ahead and write from TensorFlow dot Keras dot layers import embedding okay
embedding.
And then along with that we will also use LSTM.
Then I'll go ahead and use dense right.
We require an embedding layer.
We require an LSTM layer.
We require a dense layer.
And let me go ahead and use one more.
Something called as dropout.
Dropout is specifically to disable some of the neurons while training so that you know your entire model
does not overfit.
Now I'll go ahead and define the model okay.
So first of all, I'll go ahead and initialize with sequential okay.
Then I will go ahead and add my embedding layer.
I hope I've actually covered everything as such.
You know in the course itself.
Embedding.
And then I have my total underscore words uh, comma.
Let's say, um, yeah.
The number of dimension that I'm going to take is 100.
And my input length should be input underscore length should be my max sequence length.
Okay.
This is what we are specifically using for our embedding layer.
Then let's go ahead and add model dot add model dot add with respect to LSTM, I'm going to add my LSTM
layer.
Let's say I will go ahead and use 150 and along with this 150 neurons okay.
We'll also go ahead and write return underscore sequences.
Sequences is equal to true okay.
Uh along with this I'm also going to go ahead and add my dropout layer.
So let me just go ahead and add some dropout layer where I'm saying, uh, just go ahead and make sure
that you disable 20 percentage of the neurons hidden neurons when you are training the neural network.
Along with this, let me just go ahead and add one more LSTM layer.
If you want, you can add one more.
Just see.
Let's see whether it will be able to perform well.
And here I'm going to use the number of neurons as 100 okay.
Then finally I will go ahead and add model dot add.
And quickly let's go ahead and create my dense and the total words.
And here I'm going to use my activation function.
Uh activation is nothing but softmax since it is an multi.
See over here I'm using softmax.
The reason is very simple because um, with respect to the softmax, uh, here my activation function
because my output.
Right.
It is multi-class.
It is not a single class.
Right.
So that is the reason I'm using softmax.
Otherwise I would have used uh, sigmoid.
Okay.
Now we will go ahead and compile the model.
So this will be the next step for compiling.
So I will go ahead and write model dot compile.
And you know the loss that I'm actually going to use is nothing but categorical cross entropy since
it is multi-class okay.
Cross entropy.
And the optimizer that I'm actually going to use.
Optimizer will be nothing, but it will be Adam.
Okay.
And the metrics that I'm actually going to use is nothing but accuracy.
Metrics will be accuracy.
Okay, so finally I can go ahead and check my model dot summary.
So let me go ahead and write model dot summary.
So overall once I execute this you'll be able to see I'm getting some kind of errors.
So what is the error.
Let's see.
Uh got an unexpected keyword argument drop out okay.
I should be writing this in this way.
It should is it is not a keyword argument, but instead it is a, uh, like how we have a separate layer
like that.
Okay.
So now you can see over here my embedding layer is my first layer, then LSTM, and dropout LSTM second
layer.
And then finally this is my output layer.
Right.
And total number of trainable parameters is how much right.
So this is some around 12 lakhs 19,418.
Right.
So these are my total trainable parameters.
Now it's time that we go ahead and train our entire model.
Right now to train the model uh I will just go ahead and write like this.
So let's go ahead and train the model.
And here I'm just going to use my history okay.
Is equal to model dot fit.
And with respect to this I will go ahead and write my X train comma y train.
Uh let's use around at least 50 epochs.
And this is going to take some time.
It's not going to happen very fast.
And if I also go ahead and use this 50 epochs right.
It is also not going to probably give you the 90 percentage error.
And for that I think we need to run it for one hour.
Okay.
But obviously I will just try to show you with respect to 50 epochs.
So my validation data is basically having my X test.
And here I will use one more bracket.
And this will basically be my Y underscore text.
And I will just go ahead and use my verbose is equal to one so that it gets displayed.
Okay, so this is my history, uh, with respect to the model fit.
Uh, now quickly let me just go ahead and execute it.
So now my execution has started.
Uh, input zero layer with the layer.
None.
Comma 14.
So, guys, uh, over here, one issue, uh, over here that we are specifically getting it is saying
that, hey, input zero of layer sequential one is incompatible with the layer expected shape none comma
14.
But, uh, you know, we have given this font shape none comma 13.
The reason is very simple because here, uh, I've given this max sequence length.
Uh, here I will just give it as minus one.
Okay.
So that is the reason I think that is happening.
So let's go ahead and execute this quickly.
And here we will go ahead and start the epoch okay.
So please make sure that you see this particular code.
And I'm also going to keep the error similarly like that so that you will be able to fix it okay.
Um, again uh, if I probably see my max sequence length over here, it is basically starting from zero,
right.
So when we are starting the indexes from zero, uh, then we uh, to get the actual length of the indexes,
I'm just doing it as minus one.
So here you will be able to see my epoch has basically started.
I'm in epoch two.
Uh, my accuracy is 0.03 right now.
I hope so, it should keep on getting increases.
So this is just 3% accuracy.
And obviously you'll be getting this accuracy because hamlet dot txt is very complex.
It is not that simple okay.
So uh now you'll be able to see that it'll slowly increase.
Now see from .038 it has gone to .046.
Then .55 I hope I've used everything right.
Accuracy is the parameter.
Okay, perfect.
But it will increase.
You just need to run it for many, many epochs, at least 100 epochs.
If you run, you will be able to gain some very good accuracy.
So now, uh, now the accuracy, the loss is also decreasing over here.
Here also the loss is decreasing.
So let's let this continue this specific epoch.
And uh, I think uh, once the training gets completed, then I will show you the final accuracy.
So with respect to only 50 epochs, you can keep on increasing the number of epochs to 100 and do it
and anyhow.
Or you can also go ahead and apply this early stopping if you really want.
Okay.
So with respect to that you can go ahead and apply early stopping if you want.
Okay.
Um but again I did not apply it over here.
If you want to apply I will just give you the code with respect to early stopping also.
Okay.
Uh, how to probably do with this early stopping and where to add the particular parameter as you go
ahead.
Right.
So for if you really want to just implement early stopping quickly, I will just make a code over here.
And I will define this code over here itself.
Right.
So first of all let me just copy and paste this okay.
Okay.
Now first of all it will tell you to hey go ahead and import this early stopping.
So we will go ahead and do this.
And in the callback.
So Tensorflow.keras or callback will be implementing this early stopping uh when you are doing the fit,
you know, uh, over here when you are basically giving this particular, um, this, this, this,
uh, when you're doing this particular fit, right at that point of time, you can add this early.
Early stopping, right?
You can just say, hey, my callback is something like this, right?
I will I will be able to add this particular callback.
Okay.
That's it.
Then automatically the early stopping will be enabled.
But I think, uh, this loss will keep on decreasing as we go ahead.
So let us wait till this entire training gets happened and then, uh, we will go ahead and do the prediction.
Okay.
So yeah, in some time I will see you all.
Thank you.