Hello guys.
So we are going to continue our discussion with respect to one hot encoding and NLP.
And I hope you have understood that how we can convert words into vectors and probably the entire process.
I have actually explained to you over here in this video, we are going to talk about the advantages
and disadvantages of using one hot encoding.
So let me go ahead and let me write it down.
So here are the advantages okay.
And here are the disadvantages.
Okay.
Now first of all, the basic advantage is that it is very easy to implement with Python programming
language, right?
Easy to implement with Python.
Now why I'm saying you that because in sklearn we have a specific library where we can easily implement
this.
And it is basically called as one hot encoder Okay.
And in pandas, if you are familiar with pandas library, we have something called as PD dot get underscore
dummies right.
So this function will basically help you to create this entire one hot encoding based on the words right.
And again the second advantage and obviously uh, we'll see this kind of examples as we go ahead.
But again I'm not going to implement specifically separately as a video in one hot encoding because
we don't mostly use this NLP in NLP technique.
And I'll tell you why.
Because there are a lot of disadvantages also.
Now let's go ahead and talk about the disadvantage.
The first disadvantage over here.
You see that over here.
At the end of the day if I probably consider right the food is good right.
At the end of the day I'm getting 1000.
And for the next one I'm getting this 010 zeros.
And there's so many number of zeros and ones in, in arithmetic, in in linear algebra we basically
say this as sparse matrix.
Okay.
So this basically creates a sparse matrix right.
Now what is sparse matrix exactly.
To talk about.
Sparse matrix is that in an array in a matrix you have lot of ones and zeros okay.
And what is the problem with respect to sparse matrix?
I can also say this as we can also convert this into an array also.
So we can also say this as sparse arrays.
But understand what is the disadvantage with respect to sparse matrix.
Whenever we apply any machine learning algorithm this specific machine learning algorithm, you know,
after we convert the text into vectors, right?
In most of the machine learning algorithms, this leads to something called as overfitting.
Right now, what exactly is overfitting?
Overfitting is a process wherein you get a very good accuracy with the training data, but with respect
to any new data, it will not be able to give you a very good accuracy.
So this sparse matrix usually leads to something called as overfitting.
Right.
So this is the one of the major disadvantage.
Now let me talk about the next disadvantage over here okay.
Now over here let's say that I have this words like the food is good, the pizza is the sorry, the
food is good, the food is bad.
Pizza is amazing right now.
See one thing over here, right?
In any machine learning algorithm, whenever you give the data.
Right.
And in this case also if I probably say this vocabulary right.
These are all my features.
The food is good.
Bad pizza.
Amazing.
Right now over here you can see that most of the time my inputs for every word that is getting converted
into a vector the size is seven, right?
1234567 right.
And based on the number of words here, you will be seeing that I'm having four words.
So I'm getting four cross seven here.
Also I'm having four words which I'm actually getting four cross seven.
But if I consider with respect to the third statement or third sentence, right.
So if I probably consider D D3 right.
And if I start creating a one hot encoded format for this, so what it will be see is there.
Right.
So pizza where it is one at this particular instance.
So I will probably create something like this 0000010.
Right.
So this is pizza right.
Is again we have is over here is is nothing.
But this will be one and remaining all will be zero.
So I'll just write 0010000.
Right.
And the third statement is amazing, which will be the last one as far as I remember from the vocabulary.
So this is how my D3 looks right in one hot encoded format.
But one one thing you really need to understand over here the size is three cross seven.
Right now in machine learning you need to understand one thing.
Whenever we perform NLP or let's say any machine learning use case, the number of features should be
fixed with respect to the length.
But here you can see this is four cross seven.
This is four cross seven.
This is three cross seven.
So I cannot train I cannot train my this particular data for a machine learning algorithm.
Because over here we still don't have a fixed text size.
Right.
So this is one of the major, major disadvantage unless and until and with the help of One-hot encoding,
we are not getting a fixed text size right over here I have got four cross seven four cross seven.
If this was also four cross seven I could have trained it right.
So over here I'll say that for ML algorithm we need we need fixed size.
We need fixed size input Right.
And right now it is not there.
So this is again a major disadvantage right.
We'll be seeing in the upcoming lectures how with the help of Bag of words tf IDF will be getting a
fixed size of words.
Okay.
Now the third one that you will be seeing.
Most of the times we are finding zeros and ones, right?
Zeros and ones in zeros and ones most of the times.
If if a specific word is there, that will become one and remaining all will be zero.
But if I talk about the semantic meaning between these two words like dough and food, right, we are
not able to exactly calculate how far, how equal it is or how similar that specific word is.
And this process is basically called as semantic.
So here I will say that no semantic meaning is getting captured, No semantic meaning is getting captured.
Now let me talk about this in a very good example.
Okay.
Let's say I have something like this food pizza burger.
Okay.
Now you know that let's say that in my vocabulary I have three words.
So for the food representation I will basically write it as 100 for pizza.
Let's say I'm going to write it as 010, and for burger, I'm just going to write it as 001.
Now understand these are my vectors.
And right now, since there are three features, I'm basically having three vectors.
Now, if you have probably heard of something called as cosine similarity, or if you really want to
find out the distance between this vector to this vector and then this vector to this vector, if I
probably consider this, let's say I'm just going to draw a three dimension okay.
Let's say this is being determined by something like burger.
And this is being determined by something like food.
And this is being determined by something like pizza.
Now if I talk about all these things right, like let's say in the case of food it is 100.
So in this axis I will probably be getting one.
So so this will be represented by one comma zero comma 0 in 3 dimension.
And if I talk about PC 010.
So this will be my another point.
Let's say I'm just going to denote this by another point zero comma one comma zero.
And let's let's talk about the third point over here.
The third point will be somewhere here it'll be in the same distance.
So this will be zero comma zero comma one.
Now if I probably calculate the distance between this all these things it will almost be equal, right?
So it is not being able to tell the exact difference between foot pizza and burger.
It is obviously considering that, okay, all these words are in equal distance.
So we cannot understand that how this particular word is different from this.
Right.
This is super important to understand.
So in short, what I am actually trying to say over here is that no semantic meaning is basically getting
captured.
That basically means in this particular sentence, I'm not able to understand which is the most important
word, how this word is related to this word, or how this word is much more similar to this word.
So that information is not getting captured, because at the end of the day, I'm getting ones and zeros.
Okay.
So this was the uh, third major disadvantage.
Now talking about the fourth disadvantage.
And this is also very super important.
And this particular concept is something called as out of vocabulary.
Out of vocabulary.
And what does this basically mean Ovi.
Okay.
Now what what is this all about.
Let's say that right now I have this many vocabularies of word.
Let's say for the after I train my model now I want to test it for my new data set.
So for testing on my new data set this will be my test data Let's say I will say burger is bad and I
need to predict this.
So this is my test data and I need to predict this.
Now you know that over here with respect to this particular sentences okay.
Burger is nowhere present in this particular, uh, vocabulary.
So what will be the problem?
We will not find out a way.
We will not have any way to represent this burger in the form of vectors, right?
Will not be able to form a vectors over here because anyhow, we don't have a vocabulary word.
So what will happen in this particular case.
And obviously we will not be able to perform this one hot encoding because in our vocabulary I just
have this many number of words.
So this will not work when a new word is actually coming, wherein it is not present in the vocabulary
with respect to the test data.
And obviously this is again a major disadvantage, right?
So this technique is basically called as out of vocabulary.
So this was in short about the advantages and disadvantages of this.
Understand sparse matrix basically means many ones and zeros.
It leads to overfitting with respect to various machine learning algorithm.
Then for any machine learning algorithm I definitely require a fixed size input.
Right now we are not able to get it.
That basically means all my sentences should be of fixed size.
It will not work.
No semantic meaning.
Sorry meaning is getting captured.
I have already told you because if you try to calculate the distance, all these words are equal distance.
It is not saying that whether Pisa is very much similar to foot or burger, how much similar it is to
foot, right, that that similarity is not there and based on that semantic meaning is not getting captured
because here either we are getting ones and zeros, so we are not able to provide.
Right over here you can see that much more maximum information, which is the most most important word.
That information is also not getting captured.
So this was in short about the advantages and disadvantage.
I hope, uh, you have understood this.
And again, the next, uh, technique with respect to converting word to vectors is something called
as bag of words.
We'll try to see that how it is fixing some of the disadvantage from here.
And uh, yes, one more, one more major disadvantage.
I'll see you that.
Let's say that if your right now I just have three sentences.
Let's say I have just seven vocabulary.
Let's say tomorrow I will be having 50 k unique vocabulary size.
Vocabulary size.
Now what will happen in this particular case.
Right.
I'll be getting so many number of ones and zeros.
So in short again this is actually leading into sparse matrix.
But in a real world scenario I'll just not be having three sentences right.
I'll be having bigger sentences and I'll also be having many sentences as such.
So this is also one of the thing in any interview, if they ask you, you probably need to talk about
or explain like this with respect to advantages and disadvantages.
So I hope you have understood this.
I will see you all in the next video.
And in the next video I'm going to discuss about Bag of Words.
Thank you.