Hello guys.
So we are going to continue the discussion with respect to an.
Now finally, we are going to discuss about a very important problem statement or question that comes
in everybody's mind.
Whenever you are implementing any application with deep learning neural networks, that is determining
the optimal number of hidden layers and neurons for an artificial neural network.
Okay, now this can be really challenging and it often requires experimentation.
However, there are some guidelines and methods that you can help in making an informed decision, right?
So here are some pointers.
In order to probably come to a conclusion that how many optimal number of hidden layers and neurons
you should probably use for an an um.
So the first point over here, it says that please go ahead and start simple.
Begin with a simple architecture and gradually increase complexity if needed okay.
The second thing is that you can also perform grid search or random search.
I hope everybody knows about this hyperparameter tuning using grid search or random search to try different
architectures so you can make different architectures.
And with the help of grid search or random search, you can actually do it.
Then you also have cross validation.
Use cross validation to evaluate the performance of different different architecture, like how we used
to do in machine learning.
And here is one more approach which is called as heuristics and rules of thumb.
Some heuristics and empirical empirical rules can provide starting points, such as.
The number of neurons in the hidden layer should be between the size of the input layer and the size
of the output layer, right between that particular size.
That many number of hidden neurons can be there in the hidden layer.
Okay.
And a common practice is to start with one two hidden layer.
So, uh, obviously there are so many statements that I've discussed about, but, uh, we will try
one approach where we'll also perform hyperparameter tuning.
Um, and uh, the hyperparameter tuning will be done in such a way that I will take various number of
neurons, various number of hidden layers, and then we will try each and every architecture and try
to see that which is the best model, which is probably coming out of it.
Okay.
So, uh, first of all, what I am actually going to do for this, uh, I will be using one very important
library, which is called as Keras.
Okay.
So here instead of TensorFlow, I'll also use one more library which is called as Keras because in this
Keras there is one very important module which is called as Keras classifier.
Okay.
And that actually helps you to perform this entire hyperparameter tuning.
Uh, you know, just to find out which is the best model.
So I'll quickly go ahead and install this library.
Okay.
So I'll go ahead and right pip install minus our requirements.
Oops.
That H5 file it it went ahead you know.
So requirement dot txt.
So let me just go ahead and do this.
And I hope uh, I should be able to get the installation of Keras.
Let's see right now it has not given me anything.
But uh let's go ahead and import things.
So here I will be importing some very important libraries.
One is Gridsearchcv.
See, one more additional library that we are going to use is Gridsearchcv standard scalar Labelencoder
Onehotencoder is done.
We are also going to import pipeline.
Then we are going ahead with Keras classifier which is present in Keras dot wrapper dot scikit learn.
Along with that, we are going to import TensorFlow TensorFlow Keras model for sequential, dense and
early stopping.
Okay so this is almost same.
Nothing changes over here.
But I think I'll be getting an error over here.
Let's see if the installation is not taken place okay.
So here you can see uh model not found.
No model Keras dot wrapper.
So let me go ahead and see and see in the Keras whether we have this wrappers or not okay, I don't
know.
We need to probably check it out.
Okay.
Pip install r requirements.txt.
Okay.
Keras dot wrapper.
So let me just go ahead in Google and search for it.
So pip install Keras dot wrappers or scikit learn.
So uh for this here you have this particular module okay.
So I will go ahead and use this module.
model.
So always remember whenever you find this kind of errors please make sure that you have a good sense
wherein you will be able to find out which library we are talking about.
So let's see.
Sky Keras I think if this is if this is the library that we are searching for, I think it should work.
Okay.
But if it does not work, then again we'll look for a solution, but we'll look for a solution together.
Okay, so here you can see it has been successfully installed.
Now I think it should work okay.
Oh let me restart the kernel.
Okay.
So let me see.
I think we should not be getting an error now.
Okay, we are still getting an error, So let's search for this itself.
Right.
Let's search for this.
So here what I will do I will go ahead and open my browser.
I'll search for no modules.
Keras dot wrapper.
Okay.
Um.
Oh.
So here it says this works for me.
Another approach you can try.
Uh, okay.
It comes with TensorFlow 2.12.
Okay.
Um, from Skype I okay.
I can also use this.
Let's see, I have also installed this right.
Instead of using this I will use this.
But I think now this should work okay.
So I've used sky Keras.
And there I have actually done this installation okay now this is done.
Now the next thing will be that I will go ahead and quickly import our pandas.
Uh, use pandas and read our CSV file.
And here I'm actually going to write my churn underscore modeling dot CSV.
Here you'll be seeing how I specifically experiment with things.
Okay so here I'm actually going to get my data okay.
Then you know what.
All things you really need to do okay.
So I will just try to copy and paste it over here.
And I hope I don't have to make you understand.
I'm going to apply label encoder one hot encoder.
Then this is my entire data frame.
Then we are concatenating.
We are dividing.
We are getting the independent and dependent features.
And after this we are also going to do the train test split okay.
Because this is not that important.
Because this all things we have repeated again and again.
Okay.
And then finally we are dumping all the pickle file.
Or we are saving all the pickle file so quickly.
I will copy and paste it over here.
Let me go ahead and write Import pickle because I will also be requiring import pickle okay.
Done.
So let's go ahead and execute this.
Now here you will be able to see my pickle file will get created.
So this has got executed.
Now we are going to create a function.
Now see here is where I will write my coding completely from scratch.
So define a function to create a model okay.
And try different parameters.
Try different parameters okay.
Uh and here we are specifically going to use Keras classifier okay.
So I will go ahead and write definition create underscore model.
Okay.
Uh, here uh, what I'm actually going to do is that, um, by default I will be I will be providing
two parameters whenever we create a model.
One is how many number of neurons are required.
Let's say by default I'm giving 32 neurons if I'm not passing any value and how many layers I want with
respect to the hidden layer.
By default I'll give one hidden layer.
Okay.
So I'll create this function and I'll say, hey, go ahead and create.
My model is equal to sequential model okay.
And then what I can do on top of this particular model I can go ahead and add one dense layer.
So I'll say hey go ahead and add our dense layer.
And dense layer by default will be this entire neurons okay.
And uh, along with this I'll say, hey go ahead and apply my activation function, which is nothing
but ReLU.
Then here you'll be able to see that I'm also going to use my input underscore shape is equal to x underscore
train dot shape.
Please observe very carefully what I'm actually doing over here.
Okay.
So this is my input shape.
So in the first layer I'm basically adding this.
Right now I need to probably check.
Do I need to add more layers or not okay.
If I give 2 to 3 layers then we should also add it right.
So based on this parameter I'll write a generic code.
Right?
I'll say hey, if I'm giving 2 to 3 layers, if I'm giving three layers also along with this first layer.
This first layer hidden layer will be there.
Okay.
And here you'll also be seeing that at least this will be there with, with respect to the number of
neurons that we have.
Okay.
Then a generic representation, I'll say hey for dash in range of layers minus one, because I have
to probably create that many number of hidden layers.
Now remaining hidden layers is layer minus one.
I will go ahead and write hey go ahead and add model dot, add and use the dense and here use the neurons.
The neurons will be same what I've actually passed over there and the activation function.
Activation function will be nothing, but it will be ReLU okay.
So this is just by for loop.
We will be keep on adding this particular layers.
Finally our output layer I will go ahead and write model dot add.
Here we are going to use my dense.
And here I will use one comma.
Activation is nothing but it will be my sigmoid.
Okay.
So I'm basically going to add this.
And finally I'll go ahead and write model dot compile.
And let me just go ahead and use my optimizer which is nothing but Adam okay.
Loss since it is a binary loss.
So I'll go ahead and use binary underscore cross entropy.
And finally you'll also be seeing that I'll be able to add matrix.
And the matrix is nothing, but it will be accuracy.
Okay.
And finally I return the model.
This is what we are doing inside this function okay.
So whenever I usually call this particular function I'm just going to give this parameters uh neurons
and layers.
And based on that I need to add how many number of hidden layers and all first layer is added by default.
And we have also added output layer by default.
We have also added uh, which compiler, which optimizer and loss we are specifically going to use.
Okay.
Now we will go ahead and create create a Keras classifier okay.
So I'll go ahead and write hey model is equal to Keras classifier.
Let's go ahead and call this function.
Now this Keras classifier is responsible in creating the entire entire neural network.
And so I'll say hey go ahead and create call this create model function.
I will say hey go ahead and pass epochs as 50.
I'll say let's pass.
The batch size is equal to ten.
And here I'm specifically going to use verbose is equal to zero okay.
Some information with respect to this.
Now this is fine with respect to my model.
My model is a Keras classifier, which is going to probably call this function and create an entire
model.
And then you need to run.
The model needs to be run for 50 epochs.
Batch size is ten.
And all all the information now is very much important thing.
So here I will go ahead and define my grid search parameters in the key value pair.
So this will be neurons like neurons which how many neurons I want to play with every layer.
It can be 1632 64 128 layers.
I'll go ahead with one, two, three.
If you want to add more, you can go ahead and I will play.
I'll keep on playing with each and every parameter over here.
So let's go ahead and uh, let's remove the batch size because I just want to play with one batch size
that is ten okay.
So that I don't have to probably play with more things.
And my training will also happen fast.
This is just to show you and epochs I'll play with 50 and 100.
Right now Gridsearchcv is going to probably take this entire model.
It is going to take this entire param grid, and it is just going to perform this cross validation that
we are doing.
And once we write grid dot fit, I'm probably going to get the best number of parameters.
So I'll probably after the fit operation happens we can just go ahead and print it.
I can write grid underscore result dot best underscore score and best underscore parameter parameter
will be like how many neurons layers or epochs I need to probably take place.
Okay.
So here our training will start and gridsearchcv what it does.
Every combination it will probably take, it will go ahead and see what loss is there.
And whichever has the highest accuracy it will take that particular parameter.
So let's go ahead and execute this.
So it is going to take time uh that fast.
It is not going to happen.
All this things because here now the combination is many, right?
Uh, okay, I did not okay.
I had to probably print it.
Um, at least will not be able to see it.
Okay.
Over here.
So, uh, that is the reason it is just showing all these things.
Uh, okay, I'm getting some error.
So, guys, the error that we we got.
Right.
So what I will be doing is that along with this parameters, I will also be adding this layers equal
to one and neurons is equal to 32.
Because in this create model I'm actually passing this particular values.
Right.
Neurons 32 and layer one.
Right.
So that parameters.
Also I'll be passing in my Keras classifier.
Along with this I also saw one more issue that um, is with respect to let me just go ahead and have
a look, uh, where it is regarding binary cross entropy.
So how many time I have basically written this?
Okay, so I think now it should work fine.
Let's run this.
And finally I'll also run this.
Okay, now the entire fitting will specifically happen.
And here you can see three folds for each of 16 candidates.
Totally 48 fits.
That is basically going to happen.
And um, now the training has already started.
I think, uh, you know, uh, once the epoch starts with respect to this, we should be able to see
more information.
Um, but, uh, just by using this verbose is equal to one, we'll be able to see some information otherwise.
And right now when I wrote this n underscore jobs is equal to minus one.
Right.
It is probably going to use all the cores out there okay of my system.
But again uh, if you are trying to use do this in your local, uh, and if your local is not a good
system, then it may take some amount of time.
Okay.
Otherwise, please go ahead and do it in Google Colab.
You know, Google Colab is any point of time you will have some kind of GPUs that will be available.
So what we'll do, we'll wait for some time and uh, then uh, we will continue the discussion.
So here you can see.
Oh perfect.
The epochs has started and it's running.
It's running like anything.
See quickly.
This is the powerful of uh this is the power of a powerful workstation.
Right?
I have a powerful workstation, I have GPUs, I have all these things, uh, done as a setup over here.
And this is mainly to show you, like, how things work over here.
So here you can see accuracy is there, loss is there.
And finally, the best accuracy that you got is nothing but 85% using epochs 100, layer one and neuron
16.
So it is saying that, hey, just go ahead and use one hidden layer and uh, go ahead and use uh, 16
neurons and just try to do it and try to find it.
Right.
So when you use this now you can go ahead and create your own, uh, custom entire ah, uh, your entire
Ann, with this many number of layers.
And by this you will be able to find out the best parameter.
So I hope you were able to understand this.
Once you get this information, then constructing an An becomes very much easy, right?
So this is what we have actually discussed about determining the optimal number of hidden layers and
neurons for an artificial neural network.
Again you can try out with different different parameters, but we can use Keras classifier with respect
to this.
So yes this was it for my side.
I will see you all in the next video.
Thank you.