So guys, now we are going to continue the discussion with respect to word embedding.
And now in this video we are going to see a practical implementation that how we can use word embedding
and convert your word into vectors.
Again before I go ahead please remember this vocabulary size and feature dimensions because we are also
going to use the same thing in our code okay.
So let's quickly go ahead and open my, uh, open my file.
So here is my entire, uh, you can see in an classification I've created a folder over here.
Similarly, in simple RNN, I will go ahead and create one of my file, which is called as embedding
dot I pi and b.
Okay.
Now this in this embedding I will first of all go ahead and select our kernel quickly and start our
code over here.
Right.
So here uh you will be able to see that.
First let me some make some code cells.
And this is where I'm going to write the my code.
Okay.
Now here also we are going to use this current TensorFlow.
So that is the reason in the same v env environment we are going to work okay.
So here I will go ahead and write from TensorFlow dot Keras dot preprocessing dot text.
First of all what I'm actually going to do is that I'm going to perform one hot representation for a
specific word.
Okay.
So that is the reason I will go ahead and import one.
Hot okay.
So this one hot representation I've already shown you in my previous video.
How do we specifically do it.
Now to try some some of the sentences, here are my list of sentences that I'm actually going to use.
Okay.
So I'll go ahead and execute this particular code with respect to one hot.
And these are my sentences.
The glass of milk, the glass of juice the cup of tea I'm a good boy.
I'm a good developer.
Understand the meaning of words.
Your videos are good.
Okay, so these are my set of questions that I am going to specifically use.
So if you go ahead and execute this, uh, now if I go ahead and probably see my sentences, here are
all my sentences.
Let's say, uh, as I said you, that we need to define the vocabulary size and we are going to use
this word and probably convert this into vectors each and every word.
You'll be able to see that I'll be able to convert into vectors.
Okay.
So here my vocabulary size that I'm actually going to consider.
It's 10,000 okay.
Let's consider it 10,000 that I'm actually going to do.
Now the first thing that we are going to discuss about is one hot representation for every word.
Okay.
So I will just go ahead and write for word in sentence okay.
So first of all I will just go ahead and write.
I'll say for words in sentences because when I'm running through each and every sentence, I'll get
each and every sentence itself.
So I'm writing words.
Okay.
And for every words that I or every sentence that I get.
Okay.
Uh, in, in this specific sentence itself, I will pass it to one underscore hot.
And here I'm just going to give two parameters.
Okay.
Please focus on this I'm going to give two parameters.
The first parameter will be my words what sentence I'm specifically giving.
and the second parameter.
Here we are going to focus on the vocabulary size.
And this I will try to create as a list representation or list comprehension.
Sorry.
Right.
And this will basically give my one hot representation.
Now once I write this code let's go ahead and execute it okay I will just go ahead and execute it.
So here you can see vocabulary size is not defined because okay I did not execute this code okay.
Now it should get executed.
Now if I go ahead and execute it now see this is very amazing okay.
If you remember what were my sentences, if I go ahead and write my sentence here, you can see my first
sentence is the glass of milk.
The second sentence is the glass of juice.
Third sentence is the cup of tea.
Here you have.
I am a good boy.
Now this glass of milk has got converted into this vectors in my 10,000 vocabulary size.
Okay, 10,000 vocabulary size.
That basically means this.
The is given by the index 6186.
Right glass is given by the index 6775 Off is basically given by the index 637.
Right.
And milk is basically given by 4895.
What does this basically mean?
If I expand this off into a vector of dimension in the 638 index, I will get one remaining.
Everything will be zero.
Similarly for this particular word.
Similarly for this particular word, this is what it is basically saying.
It is just giving an idea right for every word which index it is just going to be one okay.
Now here also you can actually see this is one hot representation I am specifically using.
But we need to use this in our embedding layer.
So we are not going to use that entire vectors where only one is present and remaining all are zeros.
We will try to convert all our words based on this index.
So every word that you will be seeing in the sentence is representing one index okay.
Now once we do this okay.
Now see see observe.
One thing I have written the glass of milk.
And also I have written the glass of juice okay.
Now when we write the glass of milk and the glass of juice okay, it is very much important to see you
will be able to see all these words are same.
All these vectors are same.
Only this particular vector is basically changed, right?
Because over here I have milk and then I have juice.
So that is the reason and let's say vector.
But the index has got changed right now.
This is a very important thing that I really want you all to explain okay.
So please focus over here now in word embedding if you probably see based on this vectors okay.
Based on this vectors, if I take all these vectors okay.
And if I start plotting this, let's say I apply a PCA.
Uh, PCA basically means principal component analysis, and I convert this 300 dimensions into two dimension.
Let's say that I go ahead and do a dimensionality reduction okay.
And if I then plot all these vectors, you'll be seeing, let's say if I have man over here Okay.
Human will be somewhere here again.
Okay.
King will be somewhere here.
Queen will be somewhere here.
So here you can actually see the vectors like man and king, because they are almost similar kind of
vectors.
It will be near to each other.
Woman and queen, they are similar kind of vectors.
It will be very close to each other.
Similarly, if I go ahead and see with respect to fruits like apple and mango, these all are fruits,
right?
If you go ahead and see both these vectors will be very much nearby.
Right.
So this is very much important.
That is what it actually mentions.
Whenever you have this particular vector, you will be able to find out which vector is near to which
one.
Right.
And here only your very important algorithm gets applied, which is called as cosine similarity.
One of the amazing use cases that we will be seeing is something called as a recommendation system,
right?
Recommendation system uses this based on this particular vector.
If I'm watching an action movie, I should be recommended another action movie.
Okay.
And that is what we are specifically doing now.
I told you there are some problems with respect to one hot representation, so I will not directly use
that.
Instead, I will convert all my words into this particular indexes.
Okay.
And now I will take all these words sentence by sentence and I'll pass it to my embedding layer.
And then we will try to convert this into a word embedding representation.
So let's go ahead and write this word embedding representation.
Very much important okay.
So quickly let's go ahead and do this now in word embedding representation.
As I said right now we use dense layer.
Similarly there will be one more addition layer which is called as embedding.
So for that we will go ahead and import for TensorFlow dot Keras dot layers.
I'm just going to import embedding okay.
Then we have from TensorFlow dot Keras dot preprocessing dot.
Oops okay let me just go ahead and write from TensorFlow dot Keras.
I'm also going to import one very important library which is called as dot sequence.
And we are going to import pad underscore sequence I'll talk about it why this is really important.
Okay.
Pad underscore sequence.
Okay I will talk about it.
Just wait.
But I have imported the embedding layer over here.
Along with this, we know that we are going to create a sequential model.
So for this I will go ahead and write from TensorFlow Keras dot models import sequential.
Okay now I'm going to use this three important libraries.
So I'm actually getting one error saying that hey from TensorFlow Keras dot processing.
So now here you can see guys we are getting an error which is called as no module name TensorFlow Keras
dot pre-processing.
Now previously uh this pre-processing dot sequence was present inside this TensorFlow Keras dot processing.
But now uh after seeing the documentation it has changed.
Okay.
So I will go ahead and write from TensorFlow dot Keras okay.
From TensorFlow dot Keras dot utils Okay.
And here we are just going to go ahead and import my padsequences okay.
So uh we'll just comment this out.
It was before in this.
But again whenever you see this kind of error always refer to the documentation page okay.
So now I will go ahead and execute it.
This has got executed perfectly.
Uh, now the next step that I am actually going to probably do is that we will also go ahead and import
one important library called as numpy as NP.
Okay.
Now, first of all, we'll understand what this pad sequences basically mean.
Now see observe each and every sentence.
Every sentence is of different different number of words.
Let's say okay first sentence is having four words.
Second sentence is having four words.
This is also four words.
But here you have five words.
Here you have five words.
And here also you have five words.
And some of the sentences may also have six words.
Seven words.
Right.
One very important thing is that we need to make all these sentences of equal size okay.
Otherwise we will not be able to train it in our RNN because at the end of the day, all the words that
will be probably going, it will be going for a fixed number of time stamp based on the sentence size.
Okay.
So that is the reason.
What we do is that we import this padsequences and what we are going to do with this patch sequence.
We will go ahead and set up one maximum sentence length.
So here let me just go ahead and write uh right now based on our data set, I will just go ahead and
set this sentence length to eight okay.
Because I know maximum number of words are five.
So considering this, all the words, all the sentences in this, uh, should have a maximum of eight
words.
Okay.
So I'm just keeping it.
You can also keep it to ten.
Okay.
Now I will go ahead and say use this embedded docs or create my new docs where I apply all this pad
sequence.
Because if I want to make this sentence of eight words, I need to add three zeros more, right?
So Padsequences does that.
Only you can add three zeros before or you can add three zeros after.
So in order to make all the sentences equal.
So for this sentence also I'll say if four words are missing.
So I'll go ahead and add either four zeros in the foreword or four zeros in the backward.
So that will actually become eight sentence.
Similarly here also I'll try to make it as eight sorry, not eight sentence eight words.
Similarly, I'll also try to make this as eight words.
Similarly, I'll try to see over here what is the length?
Five words.
So three more zeros.
If I add it you'll be able to see that okay.
It will become eight words.
So that is what Padsequences basically do okay.
So I will go ahead and use this.
And I'll use the one hot representation that I've used.
I'll say hey go ahead and apply padding.
And inside this padding I will say hey go ahead and do pre padding.
Pre padding basically means if I want to make the sentence of the same length I will.
How many words are basically remaining out there?
That many number of zeros will get added forward, right?
And the third parameter that we really need to give is something called as max length.
So inside this max length I will just go ahead and give my sentence length okay.
And now let me just go ahead and print my embedded docs.
Now this will be lovely because now you'll be able to see that all my sentences are of equal words.
There are two ways.
One is post, one is pre.
So if I do post I will be getting the zeros at last.
If I go ahead and write pre I will be getting zeros at the first.
Okay.
So right now I will just go ahead and use this pre okay.
So I've used my vocabulary size.
I have converted all my words into a representation wherein I'm using Padsequences along with that pre
write, and I've converted all the things based on my maximum sentence length.
Okay.
Now is very much important.
Now we will talk about feature representation.
Okay.
Feature representation.
Now in this feature representation I'll say hey use the dimension ten.
That basically means I want ten features with respect to the dimensions right in the word embedding.
Okay.
Over there I showed you in my theoretical intuition, I showed you the maximum example of 300 dimension.
But here we are going to use this ten dimension okay.
Now I'll go ahead and execute this.
Now let's go ahead and create this model that is embedding layer.
So I will go ahead and write model is equal to sequential first of all.
And then we will go ahead and add model dot add one embedding layer.
How do I add my embedding layer I'll just say embedding Now inside this embedding layer there are some
parameters that we need to give.
One is my vocabulary size.
Second is my dimension how many dimension I want.
And third is nothing but my input underscore.
Length which is equal to your sentence underscore length.
This is just to give you the maximum length okay.
And this is my model with respect to the embedding layer.
Now along with this I will go ahead and say hey I want to probably train with this entire we see I'm
I'm showing how you can also train your entire embedding layer.
Right.
So I'll say hey, model dot compile, I'll use the optimizer which is called as Adam.
Along with this I will go ahead and use my loss function which is nothing but MSE.
Okay, done.
Now if I go ahead and execute this okay, so this is what it has got executed.
Now I'll go ahead and see my model summary.
So this is my embedding model right.
I have this many number of parameters.
This many number of params are there because my vocabulary size is 10,000 okay.
And then you have this eight comma ten okay.
So this is perfect.
Now this is my model.
Now I will just go ahead and use this model.
I'm not trained it.
See I've used this embedding layer.
So embedding already is a uh is a is a class over here.
Now if I pass any words right, you'll be able to see that how it will be converting into a vector.
So now what I will do I will take this embedding layer.
I will go ahead and write model dot predict.
And let's go ahead and give my embedded docs all the embedded docs.
And this is what is very amazing.
Every word will be represented by how many number of dimension ten
123456789
ten C type ten dimensions.
Right.
And if I just give get the first sentence okay.
So my first sentence.
What was my first sentence if I probably see over here.
Right.
So, uh, let's go ahead and write in this way, then you'll be able to understand.
I'll go ahead and write embedded docs of zero.
So this is my first sentence okay.
Now I will just go ahead and predict for the first sentence what my embedding layer will basically be
giving me as a vector representation.
Obviously it needs to be ten dimension or so embedded docs of 0000 okay.
Zero of zero.
Perfect.
Okay.
Zero of zero is not working because this I'm getting is an array.
Um, then, uh, this is my first sentence.
So what you can see over here, this entire sentence has got converted to this.
Okay then my second sentence, where only the last word is changing, has got converted to this.
So that basically means this zero is got converted to this.
This zero has got converted to this zero zero is converted to this particular vector.
This zero is converted into this vector, this where uh index is converted into this vector.
So here you can actually see all these vectors right.
And that is how your entire embedding layer works Okay.
Please see this?
We don't even have to train anything.
I've used the embedding layer internally I think.
So it it is using word two vec here.
It takes vocabulary size input.
Uh then it takes the dimension.
Then it takes the input length.
Now this is something really important because in my next video I'm going to start our end to end project.
And I'm going to take a very much bigger data set.
And this embedding layer will be a part of that simple RNN.
right?
But, uh, I hope you got an idea with respect to working with embedding layer.
So yes, uh, this was it.
Uh, and uh, you can proceed further and you can go ahead and practice with different, different text.
You can go ahead and write more text over here.
Uh, and because this all steps will follow over there, you can add any number of sentences you can
verify.
Right.
So yes, this was it.
I will see you all in the next video.
Thank you.