So, guys, uh, we are going to continue the discussion with respect to this particular project.
Uh, you can see over here all the libraries has been installed.
Okay.
And I even did not get any kind of error.
Okay.
Now, I have also activated this specific environment in my terminal.
So let me just, uh, hide my terminal right now, because I will not be requiring what I will do.
Uh, along with this one last thing, I also need to do the installation.
That is nothing but ipykernel.
So since the first experiment that I'm actually going to do, I'm going to do in my Jupyter notebook.
And for that I require this ipykernel library so that I'll be able to attach a kernel and execute my
Jupyter notebook file.
So here I'm actually going to write pip install Ipykernel.
Okay.
So once I execute this you will be able to see that the installation will take place.
Okay.
So once this ipykernel has been executed, then, uh, I'm going to probably start my coding.
And the first thing that I will do is that I will read this entire CSV file and let me do also one more
thing.
Let me just go ahead and create my experiments dot Ipynb file.
Okay.
So I'm going to use this IPO Nvl file.
And in this my entire experiment will happen okay.
So right now here you can see my Ipy kernel library is basically getting installed.
That is going to take some amount of time.
Now it is done.
Now I will quickly, uh, hide this terminal.
Okay.
Um, let me just go ahead and select my kernel.
First of all, like here I have selected my kernel with, uh, Python 3.11.
Okay, now I will just go ahead and make some more code cells.
Now let's start this entire coding okay.
So first of all, uh, what I'm actually going to do is that, uh, I'm going to import pandas.
So let me just go ahead and import pandas as PD.
Uh, along with this I will go ahead and import from sklearn dot model selection because I also have
to do the train test split.
Right.
So for that I will be using this model selection import train test split.
I hope you know this from the machine learning which we have actually used.
And then along with this I will also go ahead and import from sklearn dot pre-processing.
So here I will just go ahead and write pre processing.
And then I'm going to import two important libraries.
One is standard scalar and the other one that I'm actually going to use is something called as label
encoder.
Okay.
So this two libraries I'm actually going to use along with this I'm going to go ahead and import pickle
file okay.
Pickle is specifically for pickling because when we are using the standard scalar and label encoder
it is important that we go ahead and pickle this file so that we can reuse it when we are doing the
deployment.
So here are some of the libraries that we have already done the installation.
And we are just going to import it.
Okay.
So these are some of the basic libraries.
Now let us go ahead and load the data set okay.
So for this I will say data is equal to PD dot read underscore CSV.
And here I'm going to basically call churn underscore modeling dot csv file.
Now over here you'll be able to see I'll just go ahead and display data dot head.
Okay, so these are all my values or all my features that are available over here.
I've already told you in my previous problem statement what we are going to do.
We are going to take this credit score, geography, gender, age, tenure, balance, number of product,
has credit card, is active member and estimated salary as my independent feature.
And I'm going to predict whether the person is going to exit the bank or not.
Okay.
And this is the past data from the bank which shows which person has exited or not.
So this is just the top five records.
Now the first important thing is that this first three features are not that important, right?
You have row number, you have customer ID, you have surname.
This definitely will not play a very important role.
So first of all what we will do we will go ahead and pre-process the data.
Okay.
What we are going to do here, we are just going to drop irrelevant features okay.
Which we can directly see it okay.
So here uh I will use this data variable.
And from this I will say hey go ahead and drop which all features we need to drop okay.
So drop is a function over here.
We we really need to drop this row number.
So I'll copy and paste it over here.
Then we have something called as customer ID.
Okay.
Then we have something called as surname.
Okay.
And we are going to drop all these things with axis equal to one.
Axis is equal to one basically means column wise.
Okay.
So once we do this particular drop we will be able to get the data value.
So here is my data value.
Now here you can see my features are starting from credit score geography gender age balance and all
these values are there okay perfect.
So this is good enough.
Uh we have done this.
Now here you can actually see we have a column called as, uh, geography over here.
Right.
So geography is there.
Okay.
And then, um, we also have this gender.
We have this, uh, this two features.
We specifically have, uh, geography and gender.
Now geography and gender, uh, is specifically in this particular case, you can see it is a categorical
variable.
Okay.
Now in the case of categorical variable, what I can actually do is that I can apply some kind of encoding
for this categorical variable.
Okay.
And that is what I am actually going to do.
Now let's go ahead and encode one categorical variable.
Now in this I will say hey um let's take first of all gender I'll say label underscore encoder underscore
gender is equal to label encoder okay I'm going to just use this and I'm going to say hey DF of gender
or I'll say data of gender because we have used this particular feature, data of gender.
And I'll just use this label encoder.
Encoder underscore gender dot fit.
And along with fit we will also be using transform.
And we have already discussed this why we use fit underscore transform.
And here I'm going to take this particular data.
I'm going to basically use apply this on my gender itself.
Right.
So now let's go ahead and see the data.
And let's see with respect to this particular gender.
So here you can see okay I'm getting an error the spelling mistake I'll go ahead and write this one
okay.
So once I do this so my gender will have two main features right.
One is male and female.
So that male and female with the help of label encoder has got converted to zeros and ones.
Okay, we have also seen this in machine learning okay.
So this is one of the encoding techniques with respect to the categorical variable okay I hope you are
able to understand this.
So guys uh over here in the gender we saw that our values are male and female.
And so that is the reason we use label encoder.
And we changed it to zeros and ones.
But what about the geography column?
In this geography column you basically have 2 to 3 features right.
2 to 3 categories like France, Spain and Japan.
Germany.
Right.
So all this country names are specifically there.
So here if we apply, just imagine that if we go ahead and uh, apply, uh, you know the label encoder.
So let me just go ahead and open my epi pen so that I will explain you over here.
Why, uh, why I'm specifically focus on, uh, on this particular geographic column.
Right.
So let's say if I just go ahead and convert this into a categorical column.
So here you'll be able to see I will be able to get France as zero, Spain as one, and Germany as two.
And when these types of numbers are specifically coming up, then there is a problem because, uh,
when we assign a value of Germany to two, it the at the end of the day, an is all about, you know,
numerical calculations.
Since the number or since the label of Germany is two, it will just consider that Germany is greater
than Spain or Spain is greater than France.
Right.
So this should not happen.
So for this particular case, what we will do is that we will not directly convert this with the help
of label encoder, but instead what we will do.
We'll go ahead and write some code and we will use one hot encoding okay.
Now one hot encoding will just actually give us some values with respect to zeros and ones.
Okay.
So now uh, I will just go ahead and write one hot encode geographic column.
Okay.
So here I'm just going to write geographic column over here.
Now quickly let's first of all go ahead and import the one hot encoder.
So I will say from sklearn dot pre-processing.
From sklearn dot pre-processing I'm going to just import one hot one hot encoder.
Let me just see this particular I'm seeing the documentation inside wise.
So it will be helpful for me to also check it out directly instead of wasting time.
Okay.
And, uh, writing each and every thing.
Okay.
So here what I'm actually going to do, I'm going to specifically use one hot encoder.
Now quickly I will just go ahead and use this one hot encoder.
I will initialize this.
And this time there is a parameter which is called as sparse.
Right.
So I will just make this feature sparse is equal to false.
I will let you know why we are basically using this.
And once you see the encoded values then you'll be able to understand.
I'll create a variable called as one hot underscore encoder underscore geo okay.
So this will basically be my values over here okay.
Now uh quickly uh, the next step that uh we are going to specifically use is nothing but geo underscore
encoded value.
And I'll say, hey, let's apply this one hot encoder one hot encoder underscore geo dot fit dot fit
under okay one port underscore geo dot fit underscore transform on which column basically on my geographic
column.
Right.
So here I'm going to give my geography column right now.
Once this is done you'll be able to see that.
Let's go ahead and see how I'm actually going to get this geo underscore sorry geo underscore encoder.
Um and let's execute this.
So here it says uh got an unexpected argument.
Sparse is equal to false okay.
Let me just remove it for now.
Onwards.
Okay.
Let's execute this.
So here I'm actually able to get a sparse row matrix okay.
Uh I think in one hot encoder I have something called as sparse.
Sparse is equal to false.
Let's see.
Um, got an unexpected argument called as sparse.
I think in the recent version the sparse is not present.
Let me just go ahead and use directly this one.
Okay.
Now once I get this Jio encoder, uh, over here.
Right.
What I will do is that, uh, if I use this Jio encoder, okay.
Or if I also go ahead and see what all features specifically we have.
If I try to convert this into a numpy array, I will be able to see all the values.
So first of all what I will do I'll just go ahead and write geo or sorry one hot one hot underscore
encoder underscore geo dot get underscore features.
Underscore names underscore out.
Right.
So if I go ahead and see this parameter okay.
Sorry out.
And I here I'm just going to give my function.
And inside this I'm just going to give my V column geography.
So if I go ahead and execute it it says get underscore features okay.
It should be feature sorry get underscore feature underscore names underscore out okay.
So here you'll be able to see with respect to the geography I have this three features that is getting
converted into a one hot encoder.
Right.
So, uh, with respect to this, if I try to convert, I'll use this particular features and I will
be using this, uh, sparse matrix that I've actually got in the form of data frame.
Now, in order to get that in the form of data frame, what I will do, I will just go ahead and write
geo underscore encoder or geo underscore encoder.
Uh, here I will use this particular value and I will convert this entire into PD dot data frame.
So I'll just go ahead and write PD dot data frame.
And here what I'll say this is my data G0 underscore encoder.
Along with that uh if I really want to assign the columns to this, it will be nothing but this entire
values.
Because from this I'm actually able to get the columns okay.
so this is my entire, uh, data frame.
And this is what I'm actually getting as my encoded underscore DF.
Okay.
So let me just go ahead and execute this.
So once I execute this and if I go ahead and see my g underscore encoded underscore df I'm getting an
error.
Uh, the reason is uh shape of value uh is 1000 comma one.
So guys here you can see that uh I'm actually getting this particular error.
You know, it.
Uh, shape of value that is passed is 10,000 comma one.
And it is specifically implying that it needs 10,000 comma three because here, uh, we are giving three
columns.
Right.
Now why this specific problem is happening.
See if I go ahead and print this geo encoder.
It shows compressed sparse row matrix of dtype right.
Now if I probably convert this geo encoder to two underscore array okay.
And if I execute it uh two underscore array.
Sorry.
It should be two array.
So here you will be able to see that I am able to convert this into a sparse value.
Right.
And uh that is the reason, you know before I was just giving this instead of giving this, I will just
go ahead and give this.
Okay.
So here you will be able to see that if I go ahead and give it now And if I go ahead and see this encoded
underscore f uh df so here you can see that I'm actually able to get this geography France, geography,
Germany and Spain, wherever France is that value would be one and remaining.
All will be zero wherever Germany is.
Um, that will be one like over here and wherever Spain is, that will be one.
Okay.
So this is how you fix this particular problem.
Again, I'm not going to probably, uh, delete any of this or, uh, remove any or part of this particular
video because I want everyone to see the errors that we specifically face, you know, because once
you see the error, you probably become much more confident in, uh, solving that particular error.
Right.
So let's go ahead and combine all the columns now.
So here I will just go ahead and combine all the columns.
Call the one hot encoded columns with the original data.
Okay, with the original data.
So here what I'm actually now going to do is that quickly, uh, and this is really important, uh,
because without this trust me, because the new part of the code.
Right.
What we did with respect to transformation, we really need to update that.
So I will just go ahead and update it.
And I will write data is equal to PD dot concatenation.
And here first of all I have to concatenate the new the new geo encoded de underscore DF with our this
particular data.
Right.
So first of all I will just go ahead and do the data dot drop.
So I will go ahead and write data dot drop.
And here I'm going to use my column which is called as geography because I don't require it any time
now.
And here I'm going to just use it as access is equal to one okay.
So once this is dropped the next thing is that I will just go ahead and concatenate with my geo underscore
encoded underscore df.
And this time I'm just going to give my access equal to one.
Right.
So now if I go ahead and see my data dot head you'll be seeing that okay gender has become zero one.
And along with this there'll be three more features like geography, France, Germany and Spain that
has got concatenated.
Okay, so this is how we have actually converted our categorical features into numerical features.
And uh, we have basically used two important type of uh encoding techniques.
One is label encoder and the other one is uh one hot encoder.
So or so I hope.
Uh, you have, uh, you are able to understand this entire thing, uh, and, uh, see, everything
I'm trying to do step by step will do it very much in a slower way so that you get all the points,
you understand things, and, uh, you go in, you see, uh, all this, ah, like, all machine learning
kind of feature engineering.
Only we did nothing.
Something new would not apply.
Over here.
Everything is same, like how we have actually performed right now.
Uh, let's quickly go ahead and, uh, see, uh, one is what I have used, uh, label encoding one,
I have used one hot encoder.
Right.
So I have to probably go ahead and save all this particular files right in the form of pickle file,
because this same one hot encoding and same label encoding I'm actually going to use in the later stages.
Right.
So label encoder gender I will try to go ahead and save it.
And then one hot encoder with respect to this G0 I'm going to save it okay.
So let's go ahead and save this particular file in the pickle.
So that in our end to end project we have to probably use this.
So I'll just go ahead and write save the encoders and scalar Okay.
So I'll say hey with open.
I'll open a pickle file.
I'll say label underscore encoder.
Okay.
Label underscore encoder.
Underscore geo okay.
Label underscore encoder underscore gender.
First of all we'll go ahead and save this pickle file.
And for this I have already imported my pickle file I'll go ahead and open this in my write byte mode,
because this is a deserialized or serialized file itself.
So we will be writing in the form of bytes and this as file I have created a temporary variable.
I'll just go ahead and write pickle dot dump, and I will dump this entire label encoder gender inside
this particular file okay.
Similarly you'll also be seeing that I will go ahead and open with open, uh, one more pickle file.
I'll just go ahead and create it.
That is nothing but one hot encoder underscore geo dot pickle file.
And this will also get opened in my write byte mode okay WB mode.
And here I can see as file okay.
And this also we will go ahead and dump it.
So I'll say pickle dot dump.
And here also I'm going to use my one hot.
Encoder underscore Geo.
And finally I'm just going to convert this into my file okay.
So let's go ahead and open it.
So I'll say with open or okay this two file right now I'll save it.
So once I've save it uh you can see this is my pickle file that has got created.
So till here uh the basic transformation that I have actually done right, that all has been completed.
And, uh, we have almost done this kind of transformation, and we have saved it.
Okay, now, uh, let's do one thing.
Okay, so you have done the transformation.
Now let's quickly divide our data.
Divide the data set into independent and dependent features.
Now you know that my my data dot head.
If I just go ahead and execute it.
Right.
So this is data dot head here.
Exit is my exit is my column which I will be keeping as my dependent feature and remaining all this
particular column.
I will keep it in my independent features.
Right.
So quickly let's go ahead and do that.
So I will just try to go ahead and divide my data set into independent and dependent features.
So I'll say data dot drop.
And here I will go ahead and write exited comma access is equal to one okay y is equal to data.
And here I will just go ahead and say exited right.
So this will basically be my dependent Y will be my dependent X will be my independent feature.
Then we will split the data in training when training and testing sets Okay.
Now let me just go ahead and write x train comma x test comma y train comma y test quickly.
And then this is very much common because we are using it from machine learning x comma y.
Uh let's go ahead and use some test size.
So test size will be 0.2.
And some random underscore state will be 42.
So here we have also what we need to do.
One more thing guys.
We'll scale down this features.
Okay.
Scale this features.
And the scaling up of this all the features.
Also we will be specifically doing okay.
Now scaling we will use the scaler scaler standard scalar okay.
And uh once we'll go ahead and initialize it.
Uh, here I'm just going to write X train is equal to scalar dot fit transform fit transform on my uh
the entire X train data.
And for my X test underscore data X underscore test data I'm just going to use scalar dot transform.
And here I'm just going to go ahead and write x underscore test.
So this transformation is specifically required I will quickly go ahead and execute it.
So.
Okay.
Um x comma y okay.
Perfect.
So let's go ahead and execute this.
So here you'll be able to see this will be my X train data which will be the uh it will be basically
scaling is getting uh, getting applied with the help of standard scalar.
And similarly I have my X test.
So once this is done, we'll also go ahead and save this scalar file with the help of uh in the form
of pickle format.
Okay.
We will be requiring all this pickle format.
So with open I will say hey go ahead and use my scalar dot pickle.
And uh it will be in the right byte mode Okay.
Uh, right byte mode.
Now, once we are using this right byte mode, I will just go ahead and create my context over here
as file.
And we'll go ahead and dump this dump this scalar dot scalar comma file okay.
Whatever scalar value is there.
So let's go ahead and execute it.
So here you have each and every thing.
Here you have label encoder pickle.
You have one hot encoder pickle.
We have scalar pickle.
This is perfect.
So in short in this video what I have done is that I've just cleaned some data set.
I've applied all type of uh feature engineering that I want to do.
Okay.
I have divided this into dependent and independent features.
What all, uh, main feature engineering methods that we have applied is nothing, but we have applied
both our standard scalar.
We have applied, uh, we have converted categorical features into numerical features using label encoder,
one hot encoder, everything we have specifically done.
So, uh, now my data is specifically ready.
Okay.
This is my data.
Now on this particular data, I'm going to just go ahead and train my artificial neural network.
And that is what I'm actually going to show you in the next video.
So I hope you like this particular video.
Uh, I will see you in the next video.
Thank you.