Hello guys.
So we are going to continue our discussion with respect to NLP in Deep learning.
In my previous video I was asking you a question.
Can we use A and N to solve this problem with respect to the sequential data?
Right?
Now when I said that I know you may have got confused like what kind of problem statement we are solving.
So let us go ahead and let us see the problem statement okay.
So let's consider that, uh, I have a data set and this is my data set over here.
And with respect to this particular data set, let's say I have something called as text.
And this is my output okay.
The problem statement that we are solving is something called as sentiment analysis okay.
Sentiment analysis let's say one of the data set is like the sentence is something like this.
The food is good okay.
And let's say here I have one.
The next statement that I have or next sentence in my data set, I'm writing the food is bad.
And here I have 0 or 1 basically means good.
Zero basically means bad okay.
And then third one that I have is nothing.
But the food is not good.
I have zero over here.
Let's say that in my training data set, I have this all sentence.
Okay.
Now, how do I solve this with the help of an.
Yes.
Can we solve it with the help of artificial neural network?
Right.
So let's consider that over here.
First of all we will go ahead and see.
And again, uh, first step in this is that regarding your text pre-processing, if I really want to
solve it with the help of an.
So in a text pre-processing, what we do is that we have to convert our text into vectors.
And we have seen lot of various ways like bag of word, one hot encoding TF-IDF.
And.
All right.
So let's consider that I take this particular data set okay.
And I, I will first of all go ahead and design my neural network also okay.
And that is artificial neural network.
But before that let's see that how many unique words I have in this.
Let's say I have one unique word two, three, four, five and six okay.
I have six unique words.
Okay.
And in this I will not consider is because I consider this as a stop word okay.
So I'm not just going to consider is so over here the number of unique word is 1234, five.
I'll also not consider the because this is a stop word okay.
So let me just go ahead and use this.
All remaining words 1234 okay so four words I have.
So that basically means my vocabulary size vocabulary in this particular case is for vocabulary size
Okay.
Now in this vocabulary size I have four.
That basically means my entire corpus word corpus is nothing but four.
Okay.
So as you all know, if I want to convert sentence by sentence, uh, from text to vectors.
So you have something like bag of words, right?
In bag of words.
What I, what what we actually do.
Right.
Uh, we basically try to convert every sentence into a vector.
Right.
How do we convert it?
Let's say over here I have all these words.
So I'll take this food.
Good.
Right.
So I'll say food good bad and not okay bad and not.
And with respect to sentence one, what we do is that wherever food is present right in this particular
sentence, I will make that as one and remaining all as zero.
So food and good where it is present, that will be one one and remaining all will be zero.
So here for sentence one.
If I convert this into vectors, it will be 1100.
Okay.
Then in the sentence two you will be able to see over here the foot.
Foot is there bad is there.
So this will basically become one.
The remaining all will become zero and this will become one and this will be zero.
Okay, I'm just using the technique of bag of words to convert text into vectors.
Similarly, the third one is saying that the food is not good, right?
So food is not good with respect to sentence three.
So I have this one not is there and good is there.
So this three will basically be having one.
So this is just a simple way of converting a word into vectors right.
And here you can see that I have some fixed fixed vector size.
Right I have this fixed vector size.
So this is one way.
Okay.
Now one very important information that we can see from this particular sentences and the text that
we have converted that into vectors okay.
One is like when I probably create my artificial neural network okay.
Here you will be able to see this are my inputs that I have to give to my neural network.
Right.
These are my inputs that I give to my neural network.
And since these all are text data, these all are text data.
Text data basically means the sequence information is important.
Sequence information is important.
Again I'm stressing on this, but here do you think this all words are in sequence.
No Food is coming fast based on the frequency.
Food is over here.
Maximum number of time food is present.
Right?
So food is basically related and first, then good you have, then bad you have, then not you have.
So these words are exactly not in sequence.
Right.
And when we use this particular technique and we when we try to train it with a artificial neural network,
first of all we are using that sequence over here.
When we lose the sequence that basically means the meaning of the sentence is lost.
Meaning of the sentence is lost because the context information is lost, right?
So that is the reason why I say the sequential information is super important and we are not able to
manage this.
This is the major one problem here.
If I go ahead and use even use tf IDF write in tf IDF, I may do the calculation and I may get different
different values, right?
Uh, even in word two vec, if I use word two vec, I may be able to fix this into some dimensions,
right?
Just by considering every word will be converted into one dimensions, right?
The dimension size may be 300, it may be different, different sizes, but at the end of the day,
we are losing some sequence information right over here.
Now very much important thing.
If I'm training my neural network, right.
Let's say this is my Ann.
Let's say over here, I can see that the maximum number of words are four.
So I will give my input size as one, two, three, four.
Right.
I go ahead and create my neural network.
Let's say in my neural net hidden layer I have three neurons.
And finally, since I have just one output, this basically is connected.
This input layer is connected to all the hidden neurons.
This is basically connected to this particular output layer.
Right.
And this is also connected to every hidden neurons.
This is also connected to every hidden neurons.
Right.
So this here you can see that hey fine.
This is my how my artificial neural network looks like the first problem you have already discussed.
That sequence information is getting lost.
Okay, now second problem here with respect to the text, right.
The food is good.
Yeah.
I will be sending all this information like let's say my first sentence is basically 1100.
So I'll be sending my 1100 over here.
Right.
This this is what is the entire information that I'll be sending over here right while training.
This will be my input okay.
Okay.
Just a second.
So this will be my input that is basically getting sent okay.
And here you can see this is my input layer.
And this is my hidden layer one.
And this is my output layer.
And finally I get my y hat.
Now what is one problem over here that you see in case of whenever we work with text data.
It is always a good idea that whenever I have sequence of information, whenever I have sequence of
information, let's say over here, I go ahead and write.
The food is good.
To solve any use cases, we have to give one word at a time, one word at a time.
So let's say during timestamp one I just give this the word.
Then forward propagation should happen and backward propagation two should happen.
Then in my second timestamp the next word is food.
I have to give that in my third timestamp.
The next word is is.
Then in the next timestamp.
My next word is good.
So it is always a good idea that whenever you have sequential data, you really need to give based on
a specific timestamp, not all at one.
Now, in this particular scenario, I'm giving this entire sentence one all the words all at once as
my input.
So here also input is going here.
Also input is going here.
Here also input is going.
And by this the sequence information will also not be maintained.
Right.
And that is the reason the neural network will not be able to efficiently train themselves.
Right?
In the case of sequence of data, sequential data, it is always a good idea that we pass one word at
a time to our neural network.
Right?
And this is a major problem, right?
Because just by using A and n what we are doing, we are just directly giving all the information.
Then we do forward propagation, then we do backward propagation.
And why?
I'm saying you should not give all the words at all at once, because let's say that if I want to solve
a problem of, uh, uh, language translation, in the case of language translation, I have to give
one word by word, right?
If you know about Google translation, right.
If you have used Google Translation, then let's say I want to convert from Hindi to English, or let's
say English to French.
As soon as I give one word right, let's say I say the, then it should automatically give you that
French the word right.
Then if I keep on adding words, then it should similarly go ahead and translate in that way.
So it is always necessary that you provide this information in a sequential way based on the time stamp.
Not all at once.
So this is one more major problem.
And that is the reason we cannot just directly use A and N are two important information.
First of all, it is not able to maintain the entire sequence.
And the second thing is that it is giving all the words to the neural network all at once for a specific
sentence.
Okay.
And that is where we will be using a new kind of neural network, which was proposed by researcher.
That is nothing but a simple RNN.
So simple RNN will basically be used.
And if you don't know the full form of RNN, it is nothing but recurrent neural network.
So we will be using recurrent neural network to solve this problem.
So here in the case of neural network our data will be sequential.
Now, just to give you a brief idea, how does a neural recurrent neural network look like?
Okay, so first of all I will go ahead and take an.
So in the case of an I will be having a input.
So let's say this is my input and this will get as an output okay.
So this is my hidden layer.
This is my hidden layer.
Okay.
Now with respect to this I'm giving my input over here.
Some processing will happen in the hidden layer.
And finally I'll get the output.
If I want to expand this, let's say in my input layer I have two inputs.
Let's say this is my two inputs.
Let's say in my hidden layer.
This is my three, uh, three neurons in my hidden layer.
And finally I have my output layer.
So I can also represent this in this way.
So this will be connected over here to all the neurons.
This will be connected to all the neurons over here.
Right.
And finally this will be connected to my output layer okay.
So this is my output layer and this is my input layer.
This is my input layer.
So here this representation that I have actually shown is nothing but this.
Only I have my input, I have my output.
And in middle I have this hidden layers with hidden neurons.
That is how A and N looks like.
Okay, now in the case of RNN, how it is different from A and N and why it is suitable for you can
just consider for a sequential data.
Because if I go ahead and see the neural network now the neural network looks something like this.
So initially I will be having my input layer.
Then I have my hidden layer and then I have my output layer.
This is perfectly fine, but along with this we also have a feedback loop.
Okay, now what does this feedback loop basically says that whatever output I am getting from my hidden
layer neurons, it will be sent to that particular neuron and to all the other neurons in that hidden
layer.
Okay, so if I go ahead and expand this now see see the magic?
Okay.
Now if I go ahead and expand this, this will look something like this.
Let's say I have two input uh, two inputs.
Then I have three hidden layer, sorry, three neurons in a hidden layer.
And finally I have one output.
Right.
Because to solve this problem also, I'll be requiring this neural network itself to solve this kind
of data set problem.
Right.
Because here I have this output as 100.
Okay.
So here you'll be able to see that I will be having one final output over here okay.
Let's say we go ahead and connect this okay.
We go ahead and connect this.
Now this is important okay.
Because I'm expanding it okay.
So this is very simple expanding from input till the hidden layer is very simple.
And connecting hidden to the output also is very simple.
This is nothing, but this is my an only.
Okay, this is my an only.
Now, now see the magic?
Okay, now see the magic.
I will just go ahead and connect this.
I will just go ahead and connect this okay.
So let's connect this.
So this become my input one input to this is my hidden layer.
Now obviously Ann is looking similar like RNN.
Now we have to think about this feedback loop.
Now this feedback loop will be like this hidden layer will be sending its output back to its node okay.
Not only that, this hidden layer will also send the feedback to this particular node.
Along with this.
This hidden layer will also send the feedback to this particular node.
Okay, that is what it is saying.
Sending the information.
Sending the output back to itself, right to all the hidden layer it will be sending.
Okay.
Similarly, you will be able to see that the second hidden layer, a second neuron in the hidden layer
will also sending in the loop, and it will also send to this hidden layer.
Sorry to this hidden neuron along with this, to this hidden neuron.
So we are just passing the information to each and every hidden neuron.
And finally here also you have in this neuron here you have the self loop uh feedback loop.
And then this information will also be sent to this.
And it will also be sent to this.
So in short, whatever output we are getting after, we probably take the inputs along with the weights
that are initialized.
Then we add a bias.
Then we add an activation function.
Whatever output is basically coming.
We also share it with all the other hidden layers.
Okay.
All the all the other hidden neurons in the hidden layer.
Okay.
And the and with respect to this particular neural network, all the inputs that is passed, let's say
this is my input.
The food is good.
Here we will be doing some kind of numbering based on sentence.
Okay.
Sentence and words.
Let's say this is my first sentence.
So I will go ahead and note this as X11X11 basically means first sentence and first word.
Then you have X12 first sentence and second word.
Then you have a uh the first sentence and third word and first sentence and fourth word.
In the case of RNN, we pass a input at each time stamp.
That basically means that T is equal to one.
We will be passing.
We will be passing the input as X11.
So here we'll be seeing at t is equal to one.
We will be passing X11.
Then as t is equal to two we will be passing X12.
That basically means when X11 is basically passed okay.
Whichever information it has with respect to the word that we are having right when it is passed, right.
It goes to all these hidden neurons.
It does the calculation and whatever output is there, it will be shared with all the other hidden neurons
that are present in the hidden layer.
Then in the T2 step, what we do is that we basically go ahead and write x.
We go and pass this next word that is food.
Okay.
So here we initially pass the then food.
Then food will be basically passed.
All the operation will take place.
Then teeth tree is equal to 3X13 will be passed.
T is equal to 4X1 will will passed right.
That is how each and every words are passed from a specific sentence based on timestamp.
In an.
We used to pass each and every thing at at the initial stage while doing the forward propagation.
But in case of RNN, we just pass one word at a time.
Okay, now in my next video here, I've just shown you the basic example of.
Just to make you understand the difference between A and RNN.
Okay.
And if I want to probably expand this okay.
Expand this.
This will also look something like this.
Okay.
I'll just give you one more brief, uh, unfolding technique that we specifically use.
It looks something like this.
So this is my this is my neuron okay.
This is my hidden layer.
So now what happens in RNN we create a self-loop okay.
Sorry.
So this is nothing but this is a self-loop okay.
Here also we pass some information okay.
Now if we unfold this with respect to time stamp, understand this is my hidden layer okay.
So here I will just go ahead and write like this.
So this will be my input.
This will be my output.
And then I have this.
Then this will go.
This output will go to the next time stamp okay.
Then again you have this and you have this.
So that basically means during time stamp t is equal to one.
Here I'm going to pass my X11 word of my sentence one.
Sorry.
My sentence one is something like this and my output is 0 or 1, right?
Let's say this is one.
So what I will do in my t is equal to one.
I'll be passing X11 and then at t is equal to two I will be passing X12.
At t is equal to three I will again be passing
X13 okay.
At t is equal to three.
And finally at t is equal to four.
I will be passing my last word.
So based on the number of time stamps you'll be seeing that each and every word will be passed.
Okay.
Now what operation is going to happen that we'll discuss.
Right.
But here you can see that whatever output is coming, it is also being shared with all the hidden neuron.
But at t is equal to two.
I will be sending my second word and it will also have the first word context.
Right this the entire RNN will have the first word context also, and that is how the previous context
information will also be retained by the RNN.
Right?
Please observe this right at t is equal to one.
I'm passing X11 right.
So when we pass the first word all this proposition will happen.
Let's consider that I'm just going to pass one neuron.
So this will be only one input okay.
Not two one input.
So once I do this everything will be happening once uh the input will be multiplied by all the weights
over here.
Then the bias will be added.
We'll talk about how the forward propagation will happen.
But once I get this particular output, it needs to be shared with all the other neurons.
So it has been shared with all the neurons over here.
Right.
All this information, all this output is basically shared with this particular neuron.
Now this neuron based on the time stamp and this neuron is the same hidden neuron, right.
Based on a time stamp T is equal to two.
Has the previous information of the first word.
That is, the right now in t is equal to timestamp two.
I'm getting the second word food, so it has the previous context.
It has the current context.
And again this operation will take place.
What kind of operation.
I'll talk more about it as we go ahead.
But this is just a brief idea to talk about how our how a simple RNN architecture looks like and how
it is different from an.
Okay, so, uh, in my next video, uh, I will break down this entire entire diagram again, okay.
And show you like how detaily it will be looking like.
And along with that, we'll discuss about this architecture more in a better way.
This is an important architecture.
So I will be taking more couple of classes to make you explain each and everything.
Okay.
So yes, this was it for my side.
I will see you all in the next video, and we'll continue the discussion with respect to the architecture
of RNN.
Thank you.