Hello guys.
So we are going to continue the discussion with respect to NLP.
Already we have already seen that.
Now what is our next step?
You know after text pre-processing where we have specifically performed stemming Lemmatization Stopwords.
And we have cleaned the data.
Right.
Now the main thing is that we really need to convert the text into vectors.
And there are multiple ways.
The first way that we are going to discuss about is something called as one hot encoding.
Okay.
Now we'll try to understand how how with the help of one hot encoding, we are converting the words
into a vectors itself.
So let's take this specific example.
So here I have a text and a specific output right.
Our main aim is that first of all we need to take this particular text.
And we need to convert this into vectors okay.
Now with respect to this obviously, uh, I'm also not going to again lower all the sentences or words,
but let me focus on more on like how you can basically implement this one hot encoding.
And what is the theoretical intuition behind it and how the vectors are actually created.
So, uh, to begin with over here, let's say that this is my text, the document one is the food is
good, the food is bad.
Pizza is amazing.
Okay.
So this is basically pea size.
Amazing okay.
Now how do you find out how many unique vocabulary is there.
Because see the unique vocabulary will play a very important role.
Okay, while creating your vectors.
Okay.
So to begin with, what I'm actually going to do is that we just need to find out how many unique words
are there.
So let me go ahead and write it down.
And obviously you know that if I combine all these three documents it becomes a paragraph or corpus.
So let me just go ahead and let me just write down all the unique, unique documents.
So there is one unique, uh, sorry, not document vocabulary.
The is there food is there right.
Is is there right and good is there.
So over here you can see good is also there.
Then from the second sentence you can see the food is getting repeated.
So I'm just going to write bad okay.
And if I go probably to the third sentence or the documents here you will be able to see pizza is there
which is a unique word or unique vocabulary.
Is is again getting repeated.
And finally we have something called as amazing.
So all these words that are probably present over here, this are my unique vocabulary.
Right.
Vocabulary.
So these are just nothing but the unique words, unique words that are available in this entire paragraph
or sentence, or in this particular data set.
Now, based on this unique words one hot encoding means what?
Okay, now let's say if I probably consider this particular document 1D1 okay.
So let's say this is my D one.
Now what does d one do.
Is that like if you are probably applying one hot encoding we will convert all this specific words into
this vector representation.
That basically means let's say if I'm considering the word, then the vector representation for the
word will be 1000000.
Why?
Because one is present over there.
That basically means the is present in this specific word.
So this is basically becoming one.
If I consider for the next word that is food, then how will how will be able to represent it?
It will be 0100000.
Right?
So here clearly you will be able to see wherever those specific word is, it will be represented by
a vector which will be of a dimension v.
Okay.
Now what is this v v is nothing but the words that are present over here.
So here you will be able to see one word, two word three word for word, five word, six word, seven
word.
So the unique vocabulary I have seven.
So it is going to be represented by the seven vectors where for those specific word it will be represented
as one and remaining all will be represented as zero.
Right.
So now if I probably consider how can I represent this D one that is, the food is good with respect
to this particular vectors okay.
So now understand that in this particular food uh, there are four words, right.
1234.
So over here with respect to D one the first word will have a representation of 10000.
So how many zeros are there.
123456.
Right.
123456.
Right.
So this will be my first word okay then coming to the second word.
Now the second word over here is food.
Right.
So here you'll be able to see 0100000.
Right.
So this will be my second word.
Now if I come to my third word it is basically is right.
So is is basically represented by 0010000.
So this is my third word.
And coming to the fourth word which is good.
So here it will become 0001000.
Right.
So this is my sentence one one hot code, one hot code representation with respect to this particular
text.
So this text if I apply one hot encoding I'm going to get this kind of representation.
So guys now if I talk about the shape of this D one sentence which has four words right.
So here you can see the first word representation is this one.
The second word representation is this one vector representation.
Third word vector representation is this one.
And fourth word vector representation is this one.
So if I probably see the shape right it is basically four cross seven four words 1234 okay.
And seven seven uh vocabulary size right 1234567 okay.
Now what we are going to do, we are also going to do the same thing for D two.
Okay.
So let's go ahead and try with respect to D two.
I would suggest just pause the video, try it by yourself and then probably continue the video.
Anyhow I'll be showing you the entire uh representation again okay.
So over here you'll be able to see that, which is my first word.
Again, the in D2I have the food is bad right.
So again this two are same.
This three words are same.
So I'm going to basically write the same representation.
It will be one 0 or 1 000I think there are six zeros 123456 zeros okay.
So this will be my first word.
Coming to the second word.
This is this one okay I'm just going to write this over here okay.
So this is my second word.
So here I'm going to write 010000I hope the size is 71234567.
Perfect.
So coming to the third word now, here you'll be able to see 0010000.
Okay.
Now here you can see this food is the food is it is a similar word like D one only.
So the first three is just a replica coming to the last word right now if I say last word over here
you can see good.
Over here you can see bad.
Now whenever you have bad this particular word will become one, right?
So you'll be having 000000000 and then 100.
So I'll just go ahead and write 000 again 0100.
Perfect.
So this is how the D two is basically represented.
And again over here you'll be able to see the shape will be somewhere around four cross seven.
Okay.
Now uh, yes.
Uh, this was the technique.
Simple technique of one hot encoding.
Like how we converted a text into a vectors.
Uh, over here you can see each and every word is basically represented by a one hot encoded based on
the vocabulary size, whatever the vocabulary size is basically present.
So here you can have you can see that I have a vocabulary size of seven.
So based on that one hot representation is given for each and every word.
Okay.
Now, uh, going forward in the next video, we are going to discuss about all the advantages and disadvantages
of this.
And again, one hot encoding is not getting used for this NLP use cases.
We have techniques like Bag of Word and TF-IDF.
We'll also understand what are the advantages and disadvantages.
Why should we not use this.
And then we'll be trying to understand the bag of words and TF-IDF.
Okay.
So yes, uh, this was it.
I will see you all in the next video.
Thank you.