All right, everyone.
So let's continue our discussion on a spam message classification project.
So, you know, last video we have seen, that is a very much imbalanced dataset.
So how we can make it balanced.
So if you observe the.
We have let me execute this one again.
So we'll get absolute numbers like four.
Have we have four thousand eight hundred twenty five course and four spam?
That is a very limited number of records that are relevant.
So one way we can go, like, oh, we can try to find more data.
We can try to collect more data related to spam category and make it legal.
Not exactly for it to fight what equivalently kind of very same proposal of data so that it will be
I mean, quite good in a stratification.
But no, we are not going to go with that.
We are collecting data.
But that is the one we are really working on some industry level problem.
No other way we can go ahead, Leiker.
We can discard randomly some data from this category.
And equivalently, what there were numbers of messages that are available in a spam category, the same
amount of message.
We can put it into ham category so that we we have a quite balanced dataset, like four percentage of
data belongs to him category and 50 percent belongs to spam category.
So first, let us find out just a ham message into some other variables.
So what do you label?
Can we just compare with the ham and let's just take all those records, which is having the heavens
what a label is, ham.
All right.
So you can see only hand message appears.
Let's just assign it to ham radio and.
And seemed like a ham.
Let just make it hopes another jobs market, like a spam message bucket.
So it will be a spam and they will be spam.
All right.
Legislative fight.
Disabled both.
This video was so ham, not ship.
Spam, dark shape.
So in a word, the video will be Alvo for features like in case of Ham, as we so only earlier, that
we have a quite good number of datasets set available.
So what we can do out of this for eight to five records.
We can play to grab randomly some 747 sample and put it into markets like a ham, so ham, let's say
ham.
We are going to use this sample function and how many we need to take.
So that will be a 747.
So I can give you the hard coded stuff.
Well, or instead of that, I can put away to get Sape zero.
And let me executed.
And afterwards, if you'll just then this shape on this board of this video went home safe and expansive.
So it will be a quiet balance.
And we can just up in all those spam messaging to have a message or a ham message into a spam message
and 48.
We are going to use like a hand or a pen.
Let me append spam to the ham.
And now there is some issue like up ignore index.
So let's just make it ignore.
Index is equal to two because both of these are having the same index.
So it should not create a problem while upending because there should not be a to court, which is having
exactly the same index.
And let me, as I need to know.
New variable like a data.
Now, if you observe the data like shape.
You'll be able to see we have a very limited number of records that are available like fourteen hundred
ninety four records only among them.
That is a 747.
Hmm.
And same number of records resides in a spam category.
So if you apply let's say this label value counts.
Now, not under culpably if magic will be under Karpoff data later.
So we have exactly the same number of records available in both this category while most of this label.
All right.
So that is how we made our database balance.
Now let's do some quick resolution that based on whether latest spam on a database hem.
How different feature affects to this to category.
Let's say for him only, let's try to visualize VLA other columns on, so let me display first few records.
So we have a land also vla punctuation mark.
So let's see for just a ham category, let's try to visualize the data in our form of, let's say,
a histogram.
So we get the idea that what Ham Jindalee, the land of the MACIT recites in some particular category.
We may get some useful information or we may not get any useful information.
So that is like an initial analysis.
As a machine learning engineer, you need to do it.
So let's use VLT, not here.
So that we create a histogram or I would say of one name instead Historia.
So the first records, which we are going to posit at length.
Now, Lanta, we want to display just for the home message.
So what we will do is data from data.
Let's just compare it to.
Hemm.
And we're going to display this the length column.
All right.
And let me just make it the Alpha Dog Show.
So dysfunction we are using it from.
Matplotlib.
All right.
So you can see majority of our ham message, I would say recites in a category of zero to somewhere
around one 50.
I was 150 in terms of length.
We can increase this mean also.
Let's say Lindesay will make it like, let's say hundred.
And let's just make it a little lighter.
Any feared security?
All right, so you can see this is the histogram.
Now, the same information, if you read Rothhaar Spam is it category also.
Then you will be able to get the idea that whether any differentiating factors or not.
So let me just selected another histogram.
Same figure.
We are going to draw that his father span category and legislate only now.
Both of this stone histogram will be displayed in two different fellows.
So now you will be able to see there is little differentiating factor that whenever the message of hiring
terms of land, that is a very high probability.
You can see higher land masses has a very high probability that it belongs to a spam category.
Although there are lendee messages which are harmless, so by the probability of putting those message
into a spam category will be higher.
And let's take to those same thing with contrition, MARCOSSON.
So let me just selected here.
We need analysis for a length.
So let me do for this particular column on.
And let this play.
All right.
So by looking at this crop, there is no much differentiating factor that punctuation mark is not much
affecting the classification of messaging to ham or a spam.
All right, so that is about our little initial analysis.
We tried to understand about the other data and we understood about what we are trying to do with this
data.
We made our database even balanced also.
So next task is we can go ahead with a building model and for building model.
The first ever Cuscuna Machine learning is to separate your data into two different buckets, training
and testing.
So those things we will sener next model.