Hello guys.
So we are going to continue a discussion with respect to natural language processing.
In this video we are going to cover word two vec.
We have already seen what exactly word two vec is.
It is a deep learning train model.
And again it is a kind of a word embedding techniques, wherein the focus is to convert word into vectors,
making sure that the meaning of different different words are actually maintained.
Like if there is a similar words, we will be getting vectors that are very near to each other if it
probably find out the difference, and will also be able to see that which all words are completely
opposite based on these vectors.
Okay, so let's discuss about word two vec and I will give you an idea about like what exactly is word
two vec and how the words is getting converted into a vectors.
In the upcoming videos, I will try to show you that how word two vec models are basically prepared
with respect to architecture.
Uh, in, in the case of deep learning models.
And for that you really need to have knowledge about an models, right?
If you really need to understand that, how we can train word two vec from scratch.
Okay.
And uh, yeah, let's go ahead and with the uh definition and let's see that what all problems that
actually fixes.
So over here word two vec is a technique for natural language processing published in 2013.
And it was published by Google.
An amazing company already, you know that and they have done some tons of work with respect to NLP.
You know they're doing lot of research.
The word two vec algorithm uses a neural network model.
We'll be discussing about this, how we'll be using it, uh, how it uses a neural network to learn
word association.
Please make make sure that you understand these words to learn word association from a large corpus
of text.
Once trained, such a model can detect synonym words or suggest additional words for partial sentence,
so it will be able to detect synonyms.
It will be able to detect, uh, opposite words and many more things.
As the name implies, word two vec represents each distinct word with a particular list of number called
as vectors.
So at the end of the day, we are converting a word into vectors.
But this vector will have many things.
It will be able to detect synonym words, or it will also be able to suggest additional words for partial
sentence.
Okay, now let's understand what exactly this is.
Nasty guys in bag of words TF-IDF we have already seen right?
Based on the vocabulary size will either get one, zeros, ones or zeros and in short we are getting
a sparse matrix in TF-IDF.
Also, we may get decimals like 0.2 5.6.
Then again zero zeros are there in word two vec.
It will be little bit different.
Now let me talk about how it will be different.
Let's consider that I have a vocabulary okay.
And I have a vocabulary.
Let's say I have my vocabulary.
So these are basically my this many number of unique words I have in my corpus Okay, unique words I
have in my corpus.
Corpus basically means paragraph.
Okay.
Now let's say the vocabulary words that I specifically have like something like boy, girl okay.
And then we have something like King, Queen.
And if I probably talk about some more words like apple and mango, let's say in my vocabulary, I have
this many words.
Okay.
Now, one very important word I'm going to put up over here, which is called as feature representation.
Feature representation.
Please listen to me very, very carefully.
A very important topic.
Now each and every word that are present in the vocabulary will be converted into a feature representation.
Now what this exactly means.
This basically means that we are going to convert this all words into a vectors based on some features.
Now what all features can it be?
Right?
Let me give you a very good example about it.
But understand that when we are training a very big word two vec model at that time, you will not be
getting a clear idea about all the features.
But here, just to make this intuition, to make you understand the intuition, lets discuss about this
feature representation.
So in the left hand side what will happen is that let's say that I will be having lot of features like
this.
So I may be having a feature called as gender.
I may be having a feature called as Royal.
I may be having a feature called as age.
I may be having a feature called As Food.
Like this.
I will be having a lot of features, and let's say that the total number of features, you know, are
basically 300 dimension.
That basically means I will be having one more nth feature over here and that size of this entire features,
like if I probably count all of this, this will be 300 dimensions.
Let's consider 300 dimension basically means I will be having this 300 features.
Now with respect to all this vocabulary, we are going to represent this word in the form of vector
considering this feature representation.
So this is my entire feature okay.
What I'm actually going to do I'm going to take up all these words that are present in the vocabulary.
Based on this particular features we are going to assign one numerical value okay.
And understand one thing.
This numerical value will be assigned based on the relation of this two words.
That is the vocabulary and this feature representation that we have here I have given you as an example.
But when we are training a very large word two vec model, we will not be able to see this features
entirely.
Right?
If I take an example of Google, you know they have come up with this amazing word two vec model, which
is basically trained in 3 billion words, I guess 3 billion words, which is coming from the news feed.
Right.
And that practical example also try to show you and at the end of the day, you'll be able to see that
each and every word is basically represented by 300 dimension of feature representation.
That basically is every word will be having a 300 dimension.
Uh, vectors.
Okay.
Now this is super important.
What kind of values that I can have over here.
Right.
What kind of values I can have over here.
Now, let's say with respect to Y which is present in the vocabulary, okay.
And it's and its relationship with respect to gender.
Let's say that I'm having one minus one over here.
Let's, let's let's just consider okay.
Minus one over here.
Now with respect to girl and gender the value can be plus one because it is the opposite of boy right
opposite with respect to gender.
If I say boy, is there opposite of boy is nothing but girl.
So minus one and one, this kind of vectors can come.
Now here you can see with respect to the next word.
Over here we have boy, we have royal.
We obviously know we cannot say right.
We don't have a sentence that oh he's a royal boy.
He can be a royal prince or he can be a royal king.
Right.
So there is no no proper relationship.
So in this particular case, you know, there will be a value like 0.01.
I'm just giving you as an example.
Right.
Similarly, we will also be having with respect to boy and age, let's say that there is not much relation.
So I'm just going to put it as 0.03 okay.
Very near to zero.
Now similarly I can have all these values over here.
And these all values comes through proper trained models like word two vec word two vec which is trained
by deep learning techniques like an okay.
And I'll show you in the next video how those models are basically trained.
But just understand over here, each and every vocabulary that we are seeing is represented based on
this feature representation.
So that basically means for boy I will be having this vector.
So for boy for boy you can see over here I will be having this specific vector.
Okay.
So this all vectors will be here okay.
Now similarly with respect to girl.
Now you can see with respect to gender.
If boy is having minus one then this will be plus one right.
Because it is completely opposite with respect to royal.
Again no relationship.
So it will be .02.
Let's say here I'm going to put .02 because age with respect to this also no specific relationship right.
So similarly I will be having other values like this.
Now see with respect to King, you know I may have gender like uh, there is a relationship with respect
to gender and King.
So -0.92 will be there with respect to Queen.
It can be plus 0.93 you know.
So here you can see opposite right opposite opposite over here.
Right.
And similarly I can have other vectors like over here Royal and king are related.
Right.
So I can have 0.95 okay.
And over here you can see also royal and Queen can also be a royal right.
So over here probably the value can be 0.96 .97.
Very much nearby each other right now understand because of this vectors you know similar words will
be will be very very close to each other.
Because if I try to subtract it, I'll just give you an idea about it.
So right now with respect to age also obviously there will be some relationship between age and King
because we say that old king right with respect to age.
So suppose let's say here I'm having 0.75.
Here I'm having .68.
And like this I will be having multiple vectors like this.
Right.
And we can clearly see right over here.
It is.
Now see with respect to Apple obviously it will have no relationship with respect to gender uh gender.
So I can probably write point uh point five or .01 something.
Right.
With respect to mango also it cannot be.
So I'll write just .23.
I'm just saying that okay.
We basically, uh, can have some values over here.
If it is not having a relation, it will be very much near to zero, like 0.05.
Okay.
I'm just putting some values.
But once we train all these models right, we will be getting this value here.
I'm giving you a crux idea about like how each and every vectors may look like okay, now over here
with respect to Apple.
And also I will not be having much relationship.
So let's say I'm putting minus 0.20 to.
This will be point plus .02.
With respect to age and Apple, this may have a very good relationship, right.
Because, uh, let's say if the apple is kept for ten days outside, you know, it may, it may, it
may rotten up.
Right?
It may not have that nutritional value.
So this, uh, with respect to age, it may have a direct relationship.
Mango also may it may have a direct relationship.
So we are going to have this vectors pretty much similar with respect to food.
Yes apple belongs to a food item.
Right.
So this will probably have a good value.
It may be 0.91.
This is also 0.92 right.
And similarly I'll be having different different vectors.
Right.
So here what we have done is that each and every vocabulary word is represented based on this feature
representation.
Right.
And here the feature representation is not only be 300 dimensions, it can be 100 dimension.
It can be different different dimensions.
Right.
But here I'm just showing you an example with respect to Google and what will be this feature that will
also not be exactly known, but just consider that I'm just giving you an intuitive example that yes,
based on some relationship with respect to the word, you are able to get this specific vectors.
Now, there are a lot of advantages with respect to this.
You know why I'm saying you?
Because suppose let's say if I do a calculation which is like king minus man plus queen.
If I do this calculation and this is a famous calculation which is also written in Google research paper,
if I probably do this calculation, the output that I am going to get is something called as women.
Okay.
I will definitely get it as woman.
Why or why woman is not there.
It's okay.
So I'll let me just remove it.
Suppose if I say king minus boy plus queen, then the output that I'm going to get C king is this vector
right.
I'm subtracting with boy and then I'm adding it up.
Queen.
At the end of the day you will be seeing that the girl will be much more related to boy.
So I'm going to get the output as girl here.
I'm just doing the vector calculation.
And this is what kind of relations will be able to get it.
You know this is what this is an amazing thing.
We are getting this kind of relation just by seeing these vectors, which has been provided by word
two vec.
Again, I'm not going to do the calculation here.
I've just randomly stuffed some values, but in the real word two vec use case.
You'll be seeing that if you do this kind of calculation you're going to get girl.
And this I will show you practically also as we go ahead once we use this Google or Google Word two
vec which is basically trained on 3 billion words, which is quite amazing.
Right?
So let me give you another example.
Let's say here what I'm doing instead of using 300 dimension, I'll represent every word by two dimension.
Let's say I have 0.950.96 man is represented by something like uh, .95.98.
Let's let's represent like this, okay.
I'm just giving some values, okay?
Some meaningful values so that you'll get an idea about it.
Let's say Queen is given as 0.96, let's say because this is the opposite of this.
Right.
And again this can be a similar keyword, uh, similar vocabulary.
And with respect to human, let's say that I'm having something like 0.94 or 0.96, let's say then what
I do is that if we do the calculation of king minus man plus queen, I'm going to get the human right
as a output.
Now what this vectors represent that also you really need to understand it is super, super important.
And that is where I'm going to discuss about something called as cosine similarity.
Okay.
Cosine similarity.
Super important topic with respect to understanding these things.
Because if you understand these things and all.
Now see over here King is given by two, two, two vectors right .95.96.
So obviously I can I can basically construct this into a two dimension.
Let's say that I'm getting king over here.
Okay.
And in order to make you understand, let's say Queen is over here, okay.
Or Queen can come over anywhere.
Let's say man is over here, right?
Man is over here.
If I probably say King with respect to gender, right.
This two is going to be the most nearest word when compared to queen.
Right?
So if how do I calculate the distance between this vector and this vector which is provided in this
form?
All I do is that I try to probably calculate the distance like this.
Okay, I try to find out the angle, and for this to find out the distance between two vectors, we
use a distance formula which says it is nothing but one minus cosine similarity.
Now what is cosine similarity?
This is super important.
To understand cosine similarity, I can basically say that cosine similarity is nothing, but cosine
similarity is nothing but the angle between these two vectors.
Let's say the angle between these two vectors.
Let's say I'm just taking as an example is 45 degree.
Let's, let's consider then this is nothing but cos 45 cos 45 is nothing but one by root two.
And probably I think this is approximately equal to .7071.
I've done the calculation.
If it is wrong just let me know.
Okay.
But over here then the distance between these two vectors will be nothing but one -0.7071.
Now if I'm calculating the distance it is 0.29.
Let's say so if I'm getting this 0.29 distance right I will say that okay, almost this particular word
are similar.
Let's say if I have two more vectors, if I have two different vectors, one of the vector is basically
over here, one of the vector is over here.
Then the angle between these two is nothing but 90 degree.
In the case of 90 degree my distance will be nothing but one minus cos theta cos theta is nothing but
cos 90 cos 90 is nothing but zero one -0 is one.
So I can definitely say this vector and this vector are completely different because the distance between
them is one.
If the distance is nearer to zero, I will say that they are almost similar vectors.
Now in this case it is 0.29.
I can say that okay, somewhat similar, right?
Let's say if I have one more vector which is in this point only then I can say that this two vectors
are almost same because the angle between them is uh cos zero, cos zero is nothing but one one minus
one, which is nothing but zero.
Right.
Because over here the angle between these two point is nothing but zero, right.
There is no angle at all.
Right.
So if there is no angle then I will be able to find out the distance.
In this case my distance will be nothing but one minus cos zero one minus cos zero is nothing but one
minus one, which is nothing but zero.
Now if the distance is coming as zero, that basically means these two are same word right now.
So this is super important because recommendation also happens in this way itself.
Now in recommendation let's say I have a movie which is like Avengers.
Let's say Avengers is over here.
Where do you think Iron Man will come?
Iron man will come again, near this or near this?
At this particular point, only Right.
Iron man will be coming and it will be based on different, different feature representation.
Right?
Whether it is a comic movie, whether it is an action movie.
So action, comic, right, or comedy, these all are feature representation.
Try to understand this and movie name is basically my vector, my vocabulary.
It can be Avengers, it can be this one.
Right?
So I hope you are getting an idea about how word two vec is basically working.
At the end of the day, we are basically creating a feature representation of every word, okay?
And we are able to find it out.
So yes, this was about word two vec.
Now what we are going to do is that you need to understand that how this feature representation is created
and how this vector is basically created.
How these vectors.
Because here I have randomly written boy to gender is minus one, boy to girl, girl to gender is plus
one.
Because I said that okay, this may be the opposite one.
Now you'll get an idea that how in an deep neural deep learning neural network, basically a simple
neural network, how this entire word two vec is trained.
And that is what I'm going to discuss in my next video.
Then I'm going to also show you the practical implementation.
So I hope you are able to understand this with respect to this, uh, a very good, amazing model developed
by Google, a very good architecture for with respect to this.
And we'll try to solve that in the upcoming video and I'll, I'll give you a clear idea about it.
Okay.
Just the prerequisite is that you really need to know about a and loss function, optimizers and all.
So yes, this was it.
I will see you all in the next video.
Thank you.