Have you everyone what's going on this is Caleb with slopes dot com and we're now into the good good
stuff.
We've set the groundwork we've laid the foundation for actually processing the data that comes in in
this video we're going to write the function that will actually perform the request and update our labels
with all of the information that we need to show our prediction for a particular image I'm super excited.
So let's go ahead and write that function.
Now we're going to go down beneath process classifications and write a function called update classifications
for image.
OK and that's going to be of type UI image.
Then we're going to basically start off by updating the label to say classifying giving the user a hint
that something is happening.
So classification label dot text is classifying dot dot dot.
And then after that we're basically going to perform the classification request by passing in this variable
k which when we pass in this variable what it's going to do is it's going to then instantiate the model
perform the request process the classifications and then return that request for us to actually process.
So let's go ahead and create what's called a handler type flat handler and that's going to be of type
V an image request handler and as you can see it's an object that processes one or more image analysis
requests pertaining to a single image which is perfect for our use case.
Now as you can see you can either pass in a CD image a CD image a CV pixel buffer or data and we're
going to basically pass in the CIA image here with orientation K options.
I'm not going to matter in this instance so we can get rid of options now for the CIA image we need
to basically create an instance of CIA image and pass in our UI image.
That's just the format that this vision framework requires.
But we also need an orientation parameter and basically we're just going to capture the orientation
of the original image and just say this is the way that this image should be oriented so that it knows
how to look at the image because if we pass in a bunch of sideways pictures of a cat and then we try
to take a normal upright picture of a cat.
It's not going to know what to look for which is a really cool way to make sure that we're passing in
good data.
So go ahead and type guard let orientation.
We're going to set that to be equal to CEG image property orientation.
And we need to actually use the raw value here which is of Una 32 and we can get that by going into
the UI image we pass in Taiping image orientation and then getting the raw value from that.
Okay.
And that's how we can get that value.
But it's still not in.
You went 32.
So we're going to actually have to cast it as you went 32 and then put another parentheses there at
the end so that it's the proper type.
And now we're going to go ahead and do one more property inside this guard let so type Let see I image
and that's going to be equal to see-I image.
And as you can see there is an option to pass in a UI image here.
Pass that in.
And now that we have both of those We're going to go ahead and type else.
And if that does not work we are going to basically need to display an error of some kind and we'll
come back to that in just a second.
At the end of that if that doesn't work basically we're just going to say return and just get out of
this guard let's so that we don't continue to move on and make mistakes.
So we now have an orientation property and a CIA image property so we can pass both of those in of course
ensure that you're passing in the right CIA image and the right orientation.
So at this point we now can try to perform our classification request and return back those pieces of
information that we get from our core a male model.
So to do that we're going to use a do catch block.
OK.
So type do.
Then make sure to put cash down below that's where we're going to catch errors.
And basically in this do block we're going to try handler not perform.
OK and you can see here that we can pass in some requests and it expects an array so we're going to
pass a single item into that array.
And what that's going to actually be is our classification request.
We're going to pass that in right here.
OK.
And what the perform function is going to do is it's going to schedule our vision requests to be performed
and of course when we pass it in we'll create all of this.
It will process all of the requests create these properties and then actually present the label the
way it should be.
Once we actually perform that request which is pretty cool and we also have some good air handling.
So in the instance that this doesn't work we're just going to print an error and basically just say
fails to form a form classification.
OK.
And then we can actually do the line breaks and tax and after that we can pass in the error the errors
localize description just like that.
So at this point we are now ready to try it out but I'm just going to tell you that there are several
things that we need to actually perform on some different threads.
And the reason for that is because this classification request eventually we'll process the classifications
which eventually actually updates the UI.
Anything that updates the UI needs to happen on the main thread.
So we're going to make sure that that happens by typing dispatch.
Q global.
And we're going to use this little handy dandy feature here called Q OS and that means quality of service.
And in this particular instance we're going to use user interactive.
OK.
And essentially what this is is for when things like the UI are being updated it's going to handle these
requests so that the user interactive elements are as usable as possible and they move nice and smoothly.
So we don't have any type of UI bugs or hiccups.
We're going to run this code asynchronously and we're actually going to move this entire block into
the dispatch queue block there.
So at this point we are now ready to actually try our classification request however now that it is
inside of a closure we need to make a reference to self just like so.
And I think we are nearly there.
However we need to write our function to display errors OK because right now we're not really doing
a great job at displaying particular errors.
We are here we are in other places but I'm just going to paste a function in here called display error
and I'm just going actually call that and this will be more for like.
If you don't have a specific error to present but you need to just say hey something went wrong please
try again.
That would be a really good way to do it.
So that is the display function.
Super simple.
Now we need to oh you know what.
Sorry.
That needs to be a classification label all spelled out.
Very cool.
So now the only thing we need to do is to call our update classifications function.
So go ahead and call that in our function once we have picked an image and we can just pass in the image
right here.
Pretty cool stuff.
And at this point we are now ready to build and run send over our new code and test out our classifier
to see how it works.
So press the camera button to choose an image.
Boom here we go.
Camera Roll.
Choose that classifying.
Good good.
And we get a crash interesting.
Check this out.
I label dot text must be used from main thread only.
Very interesting isn't it.
So that means we need to also pull this code.
This entire classification code we need to pull this out onto the main thread so to do that we're going
to type dispatch.
Q Dot main dot async and we're going to pull all this good stuff and run it on a main thread.
OK.
So let's go ahead and close that let's build and run it one more time and we should be able to see that
it works and it updates the label.
Thanks to putting it on the main thread so select it.
Boom boom and let's pick that image again classifying and for some reason what it sees here it thinks
about birds and tigers.
Now we could do some work to update this to where if it's not totally sure.
It would say hey you know I don't know what that is but we only showed 10 images in our training.
So it's not super super accurate.
However it is accurate enough to note key differences between different animals.
Now I don't know about you but I think we should go get some actual images of you know maybe a bird
maybe a cat.
So let's go ahead let's head to Google.
Or better you know what let's go to pectorals dot com.
They have free royalty stock images and we can just say how about cats.
Let's download an image of a cat.
I know that it did not see this image so let's go ahead and let's download this image and I'll hold
down on this and we'll save it.
So now we should be able to go into our classifier here click on our image and let's see how it does
100 percent cat.
That's awesome.
So it knows for sure that it's a cat.
Let's go find one.
How about a lizard.
Lizard let's find a cool image of a lizard that the model hasn't seen.
I'm going to scroll down a bit because I downloaded quite a bit of these.
How about this guy.
Let's see if it understands that this is in fact a lizard.
And here we go so let's save this to our phone boom.
Let's go ahead and click the photo and choose our lizard classifying 100 percent lizard.
That's awesome.
So it looks like it's really accurate for the animals that we've trained it to be accurate on.
Let's go find.
Let's do a bird and let's find a bird.
Here is a bird that I did not show the model.
And let's save it.
And let's see how we do animal classifier.
Let's choose our bird here 76 percent bird.
So it's not 100 percent accurate but it knows beyond the shadow of a doubt that this is in fact a bird
which is super super cool.
So amazing so it looks like our app works.
I'm going to go ahead and build this to my iPhone really quickly to verify that the camera also works
and I will.
We'll be back in just a second.
Here we go.
All right.
So I have the app here running on my iPhone and if I tap on the camera button you can see since I do
have a camera I have the option to choose take photo or choose photo.
Let's tap take photo and we get a crash.
So why would that be.
Let's read the error we.
Oh we can't see the error but I'll tell you what it is.
We did not yet tell it.
We didn't yet give it the proper usage description so we're going to go ahead and quickly add the privacy
camera usage description and we're going to just say we need your camera to take cute pictures of animals.
Make sure you spell it right.
OK.
Make sure you spell it right.
There we go.
And now if we build it and run it to my iPhone we can see that we have the app running right here on
our phone.
It's given me an error thinks X code beta but if I tap on the photo image tap take photo it will ask
for permission to use the camera and it will bring up this usage description that we already had here
but it appears that this is working very very cool stuff.
We can take a photo and we can use that photo and it will do its best to classify the image based on
what it thinks it is.
Or alternatively you can choose a photo like I did and it will make its own classification.
So fantastic work.
Congratulations.
We have just successfully built our very own Corum lap using a model that we trained.
Imagine if you owned maybe like a machine shop for cars and there were tons of different parts you could
take a bunch of photos of your various car parts identify them and label them and then a new employee
you could take your app and scan someone's part that they brought into the shop to get fixed they would
know what it is immediately they could pull up information on it.
There are so many use cases for this fantastic work.
Very cool stuff.
Thank you so much for watching this section of the Iowa 12 course.