All right, everyone.
So how are tax?
Almost got clean.
The next step is we are going to apply this feature engineering on this tax data.
Why does feature engineering?
Because you just cannot supply all those tax data on machine learning algorithm for that you to convert
this data into some sort of numbers.
And for that, in this particular project, we are going to use this bag of more than.
Now, before diving, we do this bag of word implementation.
Let me explain you what this bag of our model is.
And for illustration purposes, I'll just search for one of the very good, nice representation of this
bag, a Ford model on.
That is available on a quarter.
That quote is bag of four algorithm.
So this explain in a very valid time what this bag for Martin is.
So let's say you have two documents that are valuable.
One is the quick brown fox jumped over the lazy dog's back and now is the time for all good men to come
to the aid of their party.
Now, we are two documents.
Simply, we have a thousand documents here.
You can.
I'm just trying to, quote, liquidate.
So this is like an official document or a first record.
Same way.
This is one record.
This on the record.
All you can consider this is document one document.
This week we have a thousand different documents.
Right now from this one.
We have to create some sort of representation, something like a vector representation for document
one vector representation for document two and those that we can consider it as a some sort of backup
for representation of the input data.
All right.
So now from both of these documents, the first task in a backup vault, Martin, is to come up with
all those dictionary that what other possible words did after removing the stopwatch.
Now, we do not need to worry about all those stopwork them all because we have already remolding and
whatever the implementation of this bag of word model is available in this cyclone library.
They have also inbuilt mechanism to remove this stockhorse.
All right.
So let's see.
These are the terms that are available across all those documents.
Now, we do not need to worry about all those things because pestilent will take it.
But internally, these are the things that are going on.
So words are like eight.
All black.
Brown come.
And one way to collapse.
This what I would see in a horizontal axis.
These are the vertical axis, horizontal axis.
We have a number of documents, all the number of records.
So very well the intersection occurs.
There will be a one.
And this one here indicates that this eight occurs in a document, too.
So you can see eight.
Eight, eight.
Yes.
So this is a part of document to read is also part of documents to all that is back is a part of document.
So this may very well somewhat correct terms of in individual that goes or document that one will appear.
Otherwise, it will be zero.
Now you can see on a vertical axis document, one has been represented with such a kind of geology,
those know one zero zero zero and simply document high.
So has the same representation, that variable, some walkability appears.
It needs to be represented by one.
Otherwise, it will be represented by zero.
So that is nothing but a conversion of your original text data into kind of numbers.
What I would say vector representation of your input data, and that's what beg of or model does in
tandem.
All right.
So now let's apply the same technique inside this.
All of our review.
Make I say.
So for that, we are going to use from skin line like feature extraction tax.
We are going to import.
Contract to raise a class.
And first, let's create the object of this contract crisis.
Now, there are a number of options are possible.
So the very important one where we are going to focus upon that will be a.
Max Sanders, good features.
You can see Max Sanders score feature.
All right, now, what is this Max Sanders cool feature?
If you can see here, individual terms, you can consider it as like an A feature, that document.
When has this feature or document one do not have this feature.
So it is just giving the existence of some particular terms thena document only to kind of you can see
binary model, the same things apply here that while reading this model, how many features you want
to take into consideration.
So let's say we have our thousand records are available in each of this record has such a kind of tokens.
Now, tokenized is also part of we do not need to take this contract preserve part or so we'll take
it.
Once we supply this thing to the contractor asset class.
So let us get some thousand or let's say find.
So that indicates that by creating this bag of Portmore that only 1500 words will be a total walkability
or 1000 find it worse outcomes that will be a maximum features that allow and meaning all feature will
be counted at Leiker extra token.
So eventually what they will do, they will just take into consideration first fourteen hundred ninety
nine features only in all those vocabulary count, which is apart from those fourteen and ninety nine,
that will be considered as an extra token.
So that is how bad of Vermeulen Modern it will create.
So this total number of terms.
Here we have given like fifteen hundred.
Let me assign it to some CV.
And now we are going to use this and underscore transform on this Karpas.
And let me apply it as it is going to become a very sparse metrics, because many of the times you will
be able to see if you are, let's say, fifteen hundred thumbs.
Not necessarily.
I mean, just one document has a two or three times only or maybe maximum five and 20 times.
So all three, meaning values will be zero.
So the metrics which are going to produce at this particular stage, that is a very sparse metrics.
And let me add only.
Now, let us pause this video for a second.
And let us think about what is the state of this tax.
All right, so how we are going to define this to say so here, the total number of Woodrow's will be
defined by the total number of records.
So obviously, the first value will be total hundred only.
And each and every records, what are documents or each and every review will be presented by fifteen
hundred different numbers.
So obviously the total number of columns will be fifteen hundred.
So the save of this X variable, which we are going to get after this Battleford Martin will be thousand
multiplied by one thousand five hundred.
And many of the places the value will remain zero.
So that is our X next steps, we need to find a way.
Because in the next video, we are going to apply this Navy's algorithm so far that we can just simply
use VI is equal to from what originally delphinium object.
I can just simply grab high lock loops.
Let's say, all right, cause I want to take it, but only for this column.
Let me make it very loose and record of it.
Why?
Let me display my dog ship.
So that will be a housing values and each of those.
Well, you let me.
This place was 10.
I know it will be a zero one one zero.
So one indicates the red light.
And this is negative.
All right.
So now we hope, like this bag of word moralizer.
Next thing is, we are going to apply this name based algorithm.
The real machine learning algorithm to predict and create a model out of it.
So see you in the next video.