Now that you've learned how to do the basics of data analysis with pandas,
it's time to put your knowledge to the test.
And we're going to be working with a really interesting data set. Back in 2018/
2019, a bunch of volunteers went into New York central park
and they basically combed the entire park to find all of the squirrels.
So this resulted in a huge dataset of squirrels, squirrel numbers,
squirrel fur color and a whole bunch of other things.
And all of this data can be found on the New York city open data website
which we'll link to in the course resources. When you head over here,
you can take a look at all of the data that they collected.
If you take a look at all of the columns in this dataset and click show
all, you can see they've logged the location of each of the squirrels,
they gave each squirrel a unique ID,
they evaluated whether if the squirrel was an adult or a juvenile,
and they looked at what the primary fur color was. So it could be gray, cinnamon
which is red, or black. Now, if you scroll down
there's actually a really interesting visualization of this data,
where they've plotted all of this squirrels
onto a map of central park.
So you can take a look at the distribution of the squirrel population in central
park.
So you can see where the red squirrels are, where the gray ones are and where the
black ones are. So there's only three colors, really.
So what you're going to do is you're going to go to this website,
click on export and download the CSV data.
Now, once you do that,
you'll end up with a 2018 central park squirrel data like this.
Now you're going to pull that data into your day-25 project folder and click
re-factor to move it in.
And you can see that this is a lot bigger than what ever data we had before.
There's thousands of entries,
and it's actually real dedication that these volunteers went around
logging all of this data, observing all the squirrels.
It must've been a very tedious task.
But now that somebody else has carried out the hard part,
we can do some data analysis on that data.
So I want you to comment out everything that you've got so far in your day-25
project
and the goal is for you to use that data and use what you've learned about
pandas
to be able to create a CSV that's called squirrel_count that has a small table
which just contains the fur color, so there's only three fur colors,
and they are logged under the primary fur color column.
And it can either basically be gray, cinnamon
which is red, or black.
There's only three possible values in that column. Now,
what you're going to do is you are going to figure out how many gray squirrels
there are in total,
how many cinnamon ones and how many black ones based on that primary fur color
column.
And then you're gonna take that data and build a new data frame from it,
and using that,
create this final CSV using pandas. Now that you know what you need to do,
have a think about the problem and see if you can complete this challenge.
Pause the video now. All right.
If you've commented out the line where you've imported pandas,
then you'll obviously have to do that again.
So let's think about what we want to do. We want to isolate the column
which is the primary fur color. And if it helps you,
you can actually better visualize the data on this website where they've got a
table preview. So you can see that here is the primary fur color,
and you can see the colors that have been logged.
So our goal is to somehow get hold of this data series
which contains this entire column, figure out how many of them are gray,
how many of them are black and how many are cinnamon. How do we do this?
Well, firstly, let's get hold of our data.
So we're going to use our pandas.read_csv method.
And then we're going to direct it towards this 2018 central parks squirrel census
data. Now, if you want,
you can actually right-click refactor and rename it to something a little bit
shorter. But because I know that PyCharm will actually fill this in for me
as long as I start out with a string
and that that file is in the same folder,
it'll actually type it all out if I just hit enter, then it doesn't really matter.
But of course,
make sure that you don't have any typos in here if you're typing it out,
because otherwise when you hit run, you're going to get a whole bunch of error
text inside your console.
So once I've successfully read that CSV,
I've now got a dataframe. From that data frame,
I can get hold of the column that I'm interested in
which is called primary fur color. Now,
because it's got spaces, it's easier to access that data by using a square
bracket and then putting in the name of that column like this.
This is one of the methods that we showed you. Now,
once I've gotten hold of that column, well
the next thing I need to do is to find all of the rows in that column
where the data is equal to each of the colors.
So there was the color which is gray,
so gray not 'ey'.
And then once we've got hold of all the gray squirrels,
then we're going to pull that out from our data.
So now we should have a bunch of gray squirrels,
and it's probably a good idea to print them out
just to see if that actually worked.
And because I expect there'll be lots of rows with gray squirrels,
it makes sense to make it a plural, grey_squirrels. So now if I hit run,
you can see listed here are all of the rows where it contains a gray squirrel.
Now it redacted this table
so that it can actually display it in the console because we know that there's many,
many columns and there's many, many rows.
It's just showing you the first few rows and then the last few rows and also the
first few columns and the last few columns.
So we can be pretty sure that we've managed to get hold of all of the rows that
contain gray squirrels as their primary fur color. Now,
what if we wanted to know the gray squirrels count? Well,
we could use our length method because remember, once we get hold of the rows
it kind of gets treated a bit like a iterable,
like a list, and can use methods like length on this data.
So now if we print grey_squirrels_count,
you can see that we've got a total of 2,473 grey squirrels.
So now all I need to do is to repeat this process for the other colored
squirrels. So I'm going to call it a red squirrel, even though theoretically,
their fur color is cinnamon.
So I'm just going to copy and paste that in there in case I make any typos and
the final squirrel is the black squirrel.
So those are all three squirrel types,
and if I go ahead and print out all of these,
so the reds squirrel count, the black squirrel count and the gray squirrel count,
you can see that you got mostly grey squirrels, a few red ones and very
rarely do you actually see a black squirrel.
I certainly haven't seen one recently. Now that we've got our three values,
it's time to construct our data frame. So to construct our data frame,
the easiest way is to actually just create a dictionary.
So I'm going to create a data dictionary and this dictionary is going to have
two key-value pairs.
So the first key is going to be the fur color
and this is going to contain the three fur colors, which is, um,
gray, cinnamon or red, and black.
And then, um,
the next key value pair is going to be the count.
So now we can create a list where the first value is going to be the gray squirrel
count, next is going to be the red squirrel count and finally,
it's going to be the black squirrel count. So now that we've got our dictionary
and this is what it looks like,
then we can go ahead and actually turn this into a data frame.
So to do that,
we need to get hold of the pandas and then get hold of the data frame class,
and then initialize it using this data dictionary.
So I'm going to save that as df, df for data frame.
And then the final thing I need to do is to get my df
to convert to a CSV.
So now I get to specify the name of the file,
which I will call squirrel_count.csv
and once I hit run, you'll see that new file show up right here
and you can see that it's constructed my new CSV file with all of the data that
I've extracted from my central park squirrel census
and I've now got a new table with the data that I'm interested in.
So did you manage to complete this challenge? If you found it tricky
working with the data frames and figuring out how to get hold of the columns or
how to get hold of the rows depending on the conditions we're interested
in, then I strongly recommend to just head back the last lesson,
just try to write out the code that we're doing in each step of the video
yourself,
just to make sure that you're a hundred percent sure with what's going on.
Once you are ready, head of the next lesson
we're going to finally build our US states game.
For all of that and more, I'll see you on the next lesson.