Here we went.
So next project on which we are going to work upon EESA.
I am DBI and Amazon has a view classification with this spacy library.
So all those people are siccing step.
We are going to do with this inbuilt functions that are really well in this specially.
And for that.
There are many who will get upset.
I am going to use it.
So let me just upload all those NICUs.
So from file upload.
And I have Amazon sells level, high MVP level and yellow.
Wait a sec.
Let me upload all of them.
OK.
And you can see all the assets that are available now for this particular project.
I know a lady created this notebook.
So let's just walk through very fast, because majority steps are very common with respect to other
projects.
So and every single line of code has been heavily commended.
All right.
So first, as is like Liquide libraries, we are going to it next.
This yellow classification dataset, we are going to load it.
It's a separated at fine.
So let just that only.
And let's display first few records.
So you can see either it will be a zero or it will be a one.
Next is.
It has a blue collar.
So you can see it doesn't have any kind of head.
So let us assign some had a name.
It will be a review and a sentiment.
Let me executed.
And if you would just try to observe first a course again.
You would be able to see.
There will be reviews and assenting.
And how many?
Total number of rows and columns are available.
So we have a total thousand.
Those two items are available.
Let's go with another one.
That will be Amazon sells label, not testify.
Let me add, Randy.
And here also I'm just going to assign it to my column name.
What will define earlier, that will be an ending with the reviews and a sentiment.
So let me sign it and let just misplay first food quotes.
Ship.
So, rehabber, another thousand roads are available in this Amazon, like I say.
And VLA, I am Libbey.
Liberal datasets are also available.
So same like the dataset we have, I am Beebee.
And as usual, like I did it, I said, I'm just going to assign it to two different column names.
So that will be a sentiment.
And that is if you try to find the shape of desirably V v o, just the 748 that goes to column.
So now we have to treat it as it's available and a key difference.
The Dufferin object, what we are going to do, we are just going to combine all those three does it
and create one mega dataset, something like a we'll just paint all those Amazon dataset.
And I am the media, like I said, inside this dataset.
So for that, we are just going to use it for brain function and we'll just make it ignoring next group
because indexing as string.
So they don't create any kind of confusion.
So now come mainly all we have.
Two thousand seven hundred and forty records because one thousand is a part of our first one will be
1000 in the Amazon and sound 48.
I am legit.
Scratch first few records.
Review and assemblyman.
Let's try to find the distribution of individual positive and negative records.
So heroes I cyo heavily coming.
So we are total thirteen hundred eighty six records, which are positive reviews and turning under 60
to records.
That is a negative reviews.
Is there any missing records?
Are there or not?
So we can just simply check it with ease, malfunction and apply some money.
So that will be zero zero.
That means there is no missing record.
Let just segregate input and output.
Let a solid review column will become our input dataset and sentiment column will become of it.
I'll put it aside.
So that is what the segregation we have made.
All right.
So now the full stop of this importing data set has completed.
Let me just call upset and let's go to the next step.
That will be data cleaning staff now here for data, meaning we are going to use this spacy library
and we are going to remove this stop was punctuation mark.
And we are going to apply this limitation.
All right.
So what are what we have learned in know very first initial section of this course and will be basics
from there?
We are going to use those function from the spacy lightly.
Now, as a part of Biton distribution, you know, stream class itself, that is the one contribution.
Attributes that are valuable, which you can apply on this string object.
And if you just try to display this punctuation, we'll be able to see all those punctuation is a part
of one string.
Something like explanation, mark, double quotes, hash nahlah, placenta's ampersand and backslaps,
forest lesson and many more.
All right.
So that is about the punctuation mark.
Next, let's see about the stoplights and afterwards, we are just going to play all those things,
hit a one shot.
So Stoplights is a part of specially not Lange, not even stoplights.
And let just law all those stop or as a list, you know, stop was variable, if you will, let's say
right, to display stoppers.
You can see all those are topless.
All right.
Now for the cleaning.
Bye.
We are going to create one function that will be nothing but a tax, Nataša, cleaning.
And it is going to accept one sentence in the sense that one single review went to pass on here.
So first, we are going to load of a spicy model and we'll be applying those model on sentences.
Let's create one empty tokens.
Now, in the first four Lluvia just going to lowercase every single token.
And in the next four, look, we are just going to remove all those punctuation, token and stopwork
Koven.
So in a first case, there is a little twist due to increased grammar that we have to make the thing
lowercase only all those token.
But that is a little things quite change due to this English grammar.
As it says that the word is proper noun.
If the word is proper noun, there is no Lamai exist.
So in that case you could just take the lower case was an object.
But if the word is the proper noun.
So in that case, you have to take a Alema and then lowercase offic.
And in the next, if the token is not a part of this dopplers, part of contrition, then only we are
wound up to the clean dawkins'.
All right.
So that is what the text near the cleaning function.
Let me execute it.
And let's apply this thing on some simple sentences.
So if they executed.
Hopes so, they say that and helping out defined because we have to import this spacy also.
And all those moralise.
So.
So what we can do.
We haven't applied it here.
So before that, let me load this specially lively import specially and spacy the Lord and let let's
keep here our small size English vap core model.
And let me assign need to some NLB.
And let's define this function again.
And then we are going to apply this sentence on our An Martin.
Let's just test it with this very simple sentence.
So you will be able to observe that all stop words and a punctuation mark will get removed after applying
dysfunction.
So that will be a text later cleaning.
And you can see everything is lowercase.
Hello, beautiful day.
And outside.
So they're quite remote.
Remove explanation, Mark Godinho, because that is a punctuation mark and all Gallimore comma got removed.
It beats slightly more remote because that is a stop loss.
All right.
So our data cleaning part.
Oh.
Next is we are going to apply this D.F. idea like down frequency and in most document frequency.
So those things we would see the next.
Would you?