Hello and welcome to Digital in Digital Delivery, understand?
Vanishing gradient problem.
Let us begin.
When we do back propagation that is moving backward in network and calculating gradient of laws with
respect to Veidt, then the gradient tends to get smaller and smaller as we keep on moving backward
in the network.
That means the neutrons in earlier layers launched very slowly as compared to the neutrons that are
in literally us, the earlier layers in the network.
This law is to train.
So the process of reducing gradient as we go backward in network is called as vanishing gradient problem.
Let us understand the importance of earlier, least in neural network, early, early hours in the network
are important because they are responsible to learn and detect simple pat downs and they are actually
the building blocks of our network.
If these earlier layers will do improper and inaccurate results, then how can we expect that the next
layers to perform nicely and produce accurate results?
So this is the importance of earlier layers in neural network.
Let us discuss dissolution to varnishing gradient problem.
We do not use sigmoid Antonette as an activation function which caused this vanishing gradient problems,
instead of that, we use RELU based activation functions in training deep neural network model to avoid
such complications.
And we can improve the accuracy.
So theoretically, this is all about the vanishing gradient problem and its solution.
I will see you in the next door deal till then.
Happy learning.