Hello guys.
So we are going to continue a discussion with respect to natural language processing for machine learning.
In this video we are going to discuss about word embeddings.
And this was probably the topic that I should have covered long back, but I am deliberately keeping
this particular topic at this point of time.
Why?
Because we have discussed so many topics wherein we focused on converting word into vectors.
So I now you'll be getting a very clear idea about what exactly word embeddings is.
And here I have given you a Wikipedia definition.
So this is a very simple Wikipedia definition.
I have taken it from Wikipedia.
So the entire credit goes to Wikipedia over here.
So over here you can see the definition that in natural language processing word embedding is a term
used for representation of the words right for text analysis, typically in the form of real valued
vectors, that encodes the meaning of the word such that the word are closer in the vector space, are
expected to be similar in the meaning.
So let's say that I have two words king and queen.
Okay, or forget about king and queen.
Let's say that I have two words, and the two word is like happy and excited, right?
Let's say I have this two specific word.
Now when I have this two specific word with the help of word embedding techniques, what we do is that
we convert this particular word into vectors.
And let's say if I try to plot this vectors in a two dimensional graph okay.
And in if I really want to convert this into two dimensional graph, we have techniques like PCA or
other techniques, which is an unsupervised technique to do dimensionality reduction.
So once I probably plot this let's say happy and excited are coming near to each other based on this
particular vectors.
It basically indicates both are similar word.
Okay, let's say that I have one more word like angry.
Now in this case with the help of word embeddings.
If I'm trying to convert this into vectors, the tentative thing is that obviously happy is the opposite
to angry.
So angry will be somewhere here if I probably try to plot this particular, uh, vectors over here.
Why?
Why it is coming so far?
Because it is an opposite word.
So the distance between this word will be quite high, whereas the distance between this particular
word will be quite less.
So this indicates that this both the words are similar, whereas this both words are opposite to each
other.
Right.
And this is all possible because of efficient conversion of the word into vectors.
Right.
And how we are doing this again with the help of word embeddings.
But the techniques we have learned till now is it is something like one hot encoded.
We have learnt about bag of words, we have learned, TF-IDF we have learnt and all these techniques
are a part of word embeddings.
So if I properly clearly show you the division, you know in the first step if I have the word embedding
techniques.
So let me just again go ahead and write it.
Word embedding techniques are specifically of two types.
So this is my first type and this is my second type.
The first type is based on count or frequency.
Count or frequency.
And the second type is based on deep learning trained models.
Please hear to this very properly because this deep learning trained models will give you very good
accuracies.
Okay, now in count of frequency we have learned about three different types.
One is one hot encoded.
Second one is something called as bag of words.
And the third one is something called as tf IDF.
Right.
So all these techniques we have learned right.
And we know the advantages and disadvantages.
But here we are focusing more on count of frequency.
Right.
But the major one, which is having a better accuracy, which is actually and all these techniques at
the end of the day is also converting words into vectors, or it is converting sentence into vectors,
right sentence into vectors.
But we have seen lot of advantages and disadvantages.
Maximum number of disadvantages are there with respect to all these techniques.
And this all disadvantages are getting solved by this deep learning trained model.
And the trained model is nothing but which is basically called as word two vec.
It's not like we cannot create it from scratch.
We can definitely create it from scratch.
But again, you require a huge amount of data and what we are going to do in that, first of all, we
are going to understand in the upcoming videos, uh, what is exactly word two vec and how it is basically
converting a word into vectors and how it is solving all the disadvantaged things that were there in
this particular technique.
Everything we are going to discuss, but just understand that what is word two vec?
It is a word embedding technique which will efficiently convert a word into a vectors, which will be
making sure that both this property.
It is expected to have similar in meaning whenever it is converting into a vector space.
Along with that, it will also give you a very good representation of the words right.
Sparsity will not be there and all.
There are many points which I'm going to discuss in the upcoming videos with respect to word two vec.
Now word two vec are of two types.
One is because the entire deep learning architecture is built on two different types.
One is we basically say see bo, see bo see bo is nothing but continuous bag of words.
Super important, Continuous bag of words.
And we are also going to see that how the models gets trained.
But for this you really need to have a prerequisite knowledge of how an Ann works.
What is loss function, what is optimizers.
And all right.
The second technique is something called as skip gram.
Skip gram.
Again this is a different technique.
Uh and again it is a part of word two vec itself.
It is a different type of word two vec.
At the end of the day, we can either use cbow or skip gram to get an efficient conversion of word to
vectors.
Right.
And this is what we are going to see.
We are also going to see some pre-trained models of word two vec, you know, probably created by Google
and it is somewhere around 1.5 GB.
Big model size.
Will try to download it will try to see, we will try to execute it.
But in the upcoming videos what we are going to see is that how word two vec word embedding works,
right?
And how it is making sure that all these disadvantages that are present in this techniques is getting
removed.
Right?
So this is what we are going to discuss, but I hope you got an idea.
At the end of the day, whatever techniques we have discussed till now in converting word into vectors,
it falls under word embeddings, right?
So yes, this was it from my side.
I will see you all in the next video.
And in the next video I'm going to discuss about word two VEC.
Thank you.