1
00:00:06,380 --> 00:00:10,940
Everybody what's going on this is Caleb with Deb slopes and in this video we're going to go ahead and

2
00:00:11,030 --> 00:00:15,630
add in our core a male model that we created when to pull that over here.

3
00:00:15,920 --> 00:00:21,590
And then what we're going to do is we're going to create some functions to engage with it pass in data

4
00:00:21,800 --> 00:00:26,810
and then make a prediction and return that information we're going to actually process the requests

5
00:00:26,810 --> 00:00:27,560
that we make.

6
00:00:27,560 --> 00:00:33,640
So to begin let's take our M-L model file and we're going to just drop it into the animal classifier

7
00:00:33,680 --> 00:00:34,880
App project.

8
00:00:34,970 --> 00:00:41,480
And when you're done make sure that copy items if needed is ticked and click Finish.

9
00:00:41,480 --> 00:00:45,980
And as you can see when we import it we get all that information that we created.

10
00:00:45,980 --> 00:00:49,280
We see the input data and the output data.

11
00:00:49,610 --> 00:00:52,750
We have a label for what the category of the image is.

12
00:00:52,760 --> 00:00:57,440
And we also have a number of the probability of that category.

13
00:00:57,440 --> 00:01:04,550
Now as you can see the model class here isn't automatically generated swift model class for our animal

14
00:01:04,550 --> 00:01:05,330
classifier.

15
00:01:05,330 --> 00:01:12,810
And as you can see X code does all of this work for us and basically converts it into a model.

16
00:01:12,890 --> 00:01:15,020
It has all of the prediction functions.

17
00:01:15,200 --> 00:01:17,250
Everything we need which is very cool.

18
00:01:17,270 --> 00:01:20,130
So it does that for us right out of the box.

19
00:01:20,210 --> 00:01:26,070
And so let's go ahead and let's create what is called a V.N. core M-L request.

20
00:01:26,240 --> 00:01:32,690
And in order to actually do that we need to first import Corum L and we need to import vision.

21
00:01:32,990 --> 00:01:38,480
Vision is the image processing framework from Apple and it's a part of core MLM we're going to need

22
00:01:38,480 --> 00:01:41,460
it to actually make our classifications.

23
00:01:41,470 --> 00:01:47,510
Now when I was thinking I realized we're not doing any custom stuff in view ad loads we can actually

24
00:01:47,540 --> 00:01:48,790
get rid of that.

25
00:01:49,040 --> 00:01:54,450
And with that in mind we can create our V.N. core M-L request right now.

26
00:01:54,710 --> 00:02:00,170
So we're going to use what's called a lazy variable and a lazy variable is just what it sounds like

27
00:02:00,170 --> 00:02:07,950
it's a variable that is only created or the memory for it is only allocated when it's actually needed.

28
00:02:08,180 --> 00:02:17,410
So we can type lazy var and we'll just call this classification request and that's going to be of the

29
00:02:17,450 --> 00:02:19,830
and core M-L request.

30
00:02:19,840 --> 00:02:20,670
All righty.

31
00:02:20,990 --> 00:02:27,980
Now the cool thing is with a lazy variable we can initialize it like so and then using curly brackets

32
00:02:28,250 --> 00:02:35,810
we can set it up exactly how we want but it won't actually take up any memory or do anything computationally

33
00:02:36,200 --> 00:02:39,560
until we actually use this variable somewhere else.

34
00:02:39,560 --> 00:02:46,820
So what we are going to do is we're going to use a do try and catch block in order to perform these

35
00:02:46,820 --> 00:02:53,290
tasks for core M-L you know classification and then use a catch block to catch our error.

36
00:02:53,450 --> 00:02:56,140
So we're going to go ahead and type do.

37
00:02:56,540 --> 00:03:01,880
And then down below we're going to add catch and that's where we're going to catch any errors that we

38
00:03:01,880 --> 00:03:06,500
do have and let's just go ahead and use fatal error now to do that.

39
00:03:06,560 --> 00:03:15,830
We're going to say fails to load vision animal model and for some more specific information we can actually

40
00:03:15,830 --> 00:03:20,640
pass in the error here that is thrown.

41
00:03:21,110 --> 00:03:26,690
So that is what we're going to do in order to talk about what actually happened.

42
00:03:27,110 --> 00:03:31,010
And in order to actually throw an error we need to actually try something.

43
00:03:31,100 --> 00:03:36,860
So to do that we're going to go ahead and essentially initialize our model and to do that we're going

44
00:03:36,860 --> 00:03:39,460
to type let modeled equals.

45
00:03:39,680 --> 00:03:46,070
And this is where the tri part comes in and it's going to basically say we're going to make a V and

46
00:03:46,350 --> 00:03:48,540
core a male model instance.

47
00:03:48,620 --> 00:03:49,290
OK.

48
00:03:49,610 --> 00:03:54,080
And when we actually try to initialize this you can see that it throws and that's why we need to use

49
00:03:54,080 --> 00:03:58,430
try and that's why we need to catch the error in case something goes wrong.

50
00:03:58,790 --> 00:04:04,480
So the M-L model you might be wondering Well we're creating a model why are we passing in a model.

51
00:04:04,610 --> 00:04:08,740
Well we need to pass in the model we generated and to do that.

52
00:04:08,870 --> 00:04:17,150
Remember it auto generates an animal classifier swift class so we can type animal classifier and check

53
00:04:17,150 --> 00:04:17,790
this out.

54
00:04:17,810 --> 00:04:22,700
We can initialize it and then pull out the model property.

55
00:04:22,700 --> 00:04:31,170
So we now have a swift instance of our Cornell model that we created which is so so cool.

56
00:04:31,250 --> 00:04:36,410
So now assuming that worked properly what we're going to do is we're going to create what's called a

57
00:04:36,410 --> 00:04:38,480
V.N. core a mail request.

58
00:04:38,480 --> 00:04:46,220
So let's type let request and that's going to be equal to a v n core M-L request.

59
00:04:47,450 --> 00:04:53,180
And if we try to initialize this you'll see there is an option with only a model and there's an option

60
00:04:53,240 --> 00:04:55,670
with a completion handler and that's that's the one that we want.

61
00:04:55,670 --> 00:05:02,820
Because that's how we're actually going to process all of the classification requests that we pass in.

62
00:05:02,870 --> 00:05:06,900
So we need to pass in a model and that's the one we just made.

63
00:05:06,900 --> 00:05:09,950
So go ahead and type model past that guy in there.

64
00:05:10,230 --> 00:05:17,970
And then we're going to go ahead and push enter and we're going to essentially get a request and an

65
00:05:18,060 --> 00:05:19,300
optional error.

66
00:05:19,470 --> 00:05:20,080
OK.

67
00:05:20,310 --> 00:05:24,480
And so we're going to go ahead and type request and error.

68
00:05:24,480 --> 00:05:30,150
Those are the names of the parameters for the data that gets passed to the completion handler.

69
00:05:30,170 --> 00:05:38,190
Now inside the actual closure here this is where we are going to process all of the classifications

70
00:05:38,240 --> 00:05:44,120
and basically we're going to pass the request in and then we're going to essentially run through all

71
00:05:44,140 --> 00:05:50,340
those classifications pull out the labels pull out the probability and handle all of the data that does

72
00:05:50,340 --> 00:05:51,360
come back.

73
00:05:51,540 --> 00:05:53,780
And that's the next function that we're going to create.

74
00:05:53,790 --> 00:06:01,140
But for now assuming that the request is successful we need to actually tell it something very important

75
00:06:01,140 --> 00:06:02,530
about our image.

76
00:06:02,550 --> 00:06:08,820
Now if you pull in an image from your phone it is likely not a square but if you look our core a male

77
00:06:08,820 --> 00:06:13,940
model accepts an input of 299 by 299 That's a square image.

78
00:06:13,950 --> 00:06:21,690
So what we're going to do is we're going to go ahead and call request dot and we can choose image crop

79
00:06:21,720 --> 00:06:23,050
and scale option.

80
00:06:23,400 --> 00:06:27,830
And if we go ahead and look at our options here there are quite a few that will pop up.

81
00:06:27,960 --> 00:06:31,740
But the one that's most important here is center Kropp.

82
00:06:31,800 --> 00:06:36,910
So it's going to essentially crop the image in the middle into a square which is pretty cool.

83
00:06:37,200 --> 00:06:42,410
So now once we have set everything up we have processed all of our results.

84
00:06:42,420 --> 00:06:49,260
We can then return our request and that as soon as we use this it's going to run through all of this

85
00:06:49,320 --> 00:06:53,400
and then return the request to this parameter for us to use.

86
00:06:53,400 --> 00:06:57,660
So at this point we need to process our classification.

87
00:06:57,770 --> 00:06:58,330
Okay.

88
00:06:58,590 --> 00:07:05,130
And so to do that let's go ahead and just write a function down beneath our lazy variable and that's

89
00:07:05,130 --> 00:07:11,590
going to be called process clarification and that's going to be called process classifications so phunk

90
00:07:12,250 --> 00:07:20,570
process classifications for request of type Vienne request.

91
00:07:21,010 --> 00:07:24,830
And we're also going to need to include an error here of type error.

92
00:07:24,830 --> 00:07:29,160
However it is optional because you know there's not always going to be an error.

93
00:07:29,320 --> 00:07:33,550
It is an optional parameter and hopefully we won't need to use it.

94
00:07:33,580 --> 00:07:40,960
So we are going to go ahead and now create a parameter called classifications and it's actually going

95
00:07:40,960 --> 00:07:44,750
to pull the results variable out of our request.

96
00:07:44,770 --> 00:07:54,450
So we're going to go ahead and use guard let classifications equals request dot results.

97
00:07:54,490 --> 00:07:54,730
OK.

98
00:07:54,760 --> 00:07:59,390
That's the results that come back as the type V.N. observation.

99
00:08:00,130 --> 00:08:04,870
Request dot results and we're going to cast these results in an array.

100
00:08:05,020 --> 00:08:05,350
OK.

101
00:08:05,350 --> 00:08:09,330
And the array the type that these need to be are V.N..

102
00:08:09,360 --> 00:08:11,760
Cl. observation.

103
00:08:11,770 --> 00:08:17,620
Now it's classification information produced by image analysis request which is perfect for what we're

104
00:08:17,620 --> 00:08:18,290
doing.

105
00:08:18,580 --> 00:08:24,450
But if for some reason we can't do this we need to fall back on our else block here and we're going

106
00:08:24,450 --> 00:08:32,950
to go ahead and set the classification levels text parameter to say unable to classify image.

107
00:08:33,400 --> 00:08:39,750
And then we're going to go ahead and use our little you know return line syntax here and we're going

108
00:08:39,760 --> 00:08:46,420
to pass in a parameter into this string encapsulation syntax.

109
00:08:46,420 --> 00:08:53,230
We're going to pass in the error already but the localized description of that error and in the instance

110
00:08:53,230 --> 00:08:54,880
that there's not actually an error.

111
00:08:55,060 --> 00:08:58,880
We'll just go ahead and pass in error.

112
00:08:59,160 --> 00:08:59,940
Looks like.

113
00:08:59,950 --> 00:09:02,600
So that looks pretty good.

114
00:09:02,870 --> 00:09:03,210
OK.

115
00:09:03,220 --> 00:09:09,520
So if we can't actually classify the image it'll say unable to classify image then it will return a

116
00:09:09,520 --> 00:09:12,660
line and then it will spit out the error code.

117
00:09:12,730 --> 00:09:18,430
Otherwise it'll just say error which that's you know a good amount of context information for our user.

118
00:09:18,430 --> 00:09:20,370
Then of course we will return.

119
00:09:20,410 --> 00:09:24,760
So that it stops doing what it's doing and cancels out of this function.

120
00:09:25,210 --> 00:09:33,190
But let's say that we are successful and do in fact get some results back for our classifications what

121
00:09:33,190 --> 00:09:38,440
we're going to check first to see is if it is empty if 0 classifications came back.

122
00:09:38,440 --> 00:09:47,500
So if classifications classifications are it is dot is empty.

123
00:09:48,430 --> 00:09:55,670
We're going to say hey self-caused classification level text We're gonna set it to say nothing recognized

124
00:09:57,430 --> 00:09:59,740
that didn't recognize a single thing.

125
00:09:59,740 --> 00:10:02,500
We'll tell it to tell us otherwise.

126
00:10:02,650 --> 00:10:09,040
If it did recognize things and classifications came back we're going to go ahead and essentially pull

127
00:10:09,040 --> 00:10:15,880
out the top two classifications and then we're going to map over them and return their percentage of

128
00:10:15,880 --> 00:10:19,920
confidence as well as what the model thinks that the image is.

129
00:10:19,930 --> 00:10:25,120
So let's go ahead and get the top two parameters by typing let top classifications

130
00:10:27,910 --> 00:10:31,090
equals classifications dot prefix.

131
00:10:31,090 --> 00:10:33,700
This is a really cool way to slice an array.

132
00:10:33,730 --> 00:10:36,220
We're going to do prefix and then two.

133
00:10:36,220 --> 00:10:42,040
And what that's going to do is it's going to cut off the top two items in an array and return them as

134
00:10:42,040 --> 00:10:42,700
a new array.

135
00:10:42,700 --> 00:10:46,540
So we have now an array of the top two classifications.

136
00:10:46,540 --> 00:10:49,030
Now we need to go ahead and set up the descriptions.

137
00:10:49,060 --> 00:10:49,690
OK.

138
00:10:49,900 --> 00:10:53,850
And to do that we're going to basically map over the two items in here.

139
00:10:53,890 --> 00:10:57,670
We're going to pull out their confidence and convert it to a percentage then we're going to pull out

140
00:10:57,670 --> 00:11:00,370
the identifier which is the actual name of the animal.

141
00:11:00,460 --> 00:11:07,360
So let description's equals top classifications dot map.

142
00:11:07,450 --> 00:11:12,410
And we're going to map over each classification like so.

143
00:11:12,940 --> 00:11:19,270
And what we're going to do is we're going to essentially return a property to this classification which

144
00:11:19,270 --> 00:11:22,030
will then get returned back to descriptions.

145
00:11:22,060 --> 00:11:28,000
But we're going to use a string and we're going to use some special string formatting here to convert

146
00:11:28,000 --> 00:11:30,120
it into a percentage.

147
00:11:30,130 --> 00:11:35,140
So for the actual format this string is going to look super weird but basically we're going to return

148
00:11:35,200 --> 00:11:37,760
percentage sign dot to f.

149
00:11:37,780 --> 00:11:43,780
And what this says the percentage indicator basically just indicates how a certain chunk of data should

150
00:11:43,780 --> 00:11:45,670
be used.

151
00:11:45,670 --> 00:11:51,790
Point 2 means after the decimal place we're going to allow two numbers after the decimal and it's going

152
00:11:51,790 --> 00:11:53,930
to be a floating point type number.

153
00:11:54,070 --> 00:11:54,760
OK.

154
00:11:54,880 --> 00:11:56,820
Now we can get rid of this argument's parameter.

155
00:11:56,830 --> 00:12:01,660
We don't actually need it but we're going to go ahead and pass in the pieces of data that are most important

156
00:12:01,690 --> 00:12:02,310
to us.

157
00:12:02,320 --> 00:12:10,340
So go ahead and pass in the classification remember we're mapping through so each classification has

158
00:12:10,380 --> 00:12:12,920
you know its own variable.

159
00:12:13,220 --> 00:12:17,190
And we're going to pull out a parameter called V confidence.

160
00:12:18,770 --> 00:12:23,960
And the confidence it might not load here but it is the level of confidence in the observations accuracy

161
00:12:23,960 --> 00:12:26,040
from zero to 100 percent.

162
00:12:26,210 --> 00:12:28,820
But the way that it currently comes in is a decimal.

163
00:12:28,820 --> 00:12:32,410
So for 75 percent it would be 0.75.

164
00:12:32,510 --> 00:12:38,150
But instead I want it to show seventy five point to 5 percent like an actual how you would actually

165
00:12:38,150 --> 00:12:40,130
see it written out.

166
00:12:40,130 --> 00:12:46,050
So I'm going to basically take the confidence and multiply it by 100 to move the decimal over two spots.

167
00:12:46,190 --> 00:12:52,340
Then I'm going to go ahead and add the percentage sign and a dash and another space.

168
00:12:52,400 --> 00:12:58,570
And what this is going to do is it might say 60 percent dash and now is where we need to actually add

169
00:12:58,580 --> 00:13:01,230
the identifier Maybe cat maybe Bird.

170
00:13:01,310 --> 00:13:01,930
OK.

171
00:13:02,270 --> 00:13:06,480
So now type classification dot identifier.

172
00:13:06,770 --> 00:13:12,440
And as you can see the identifier is the classification label for the type of observation.

173
00:13:12,470 --> 00:13:19,340
And so now we have an array of our descriptions in K and what we're going to do is we're going to essentially

174
00:13:19,370 --> 00:13:23,720
take all of those descriptions we're going to join them with a line break and print them out on our

175
00:13:23,720 --> 00:13:24,470
label.

176
00:13:24,470 --> 00:13:30,920
So let's go ahead and go down below our descriptions parameter here and type self dot classification

177
00:13:30,920 --> 00:13:39,140
label dot text and we're going to set that to be equal to classifications at the top with a colon and

178
00:13:39,170 --> 00:13:41,180
our line break syntax.

179
00:13:41,390 --> 00:13:50,810
Then we're going to go ahead and put an end quote there and we're going to add descriptions joined and

180
00:13:50,840 --> 00:13:57,860
our separator here is important we need to separate each item in the descriptions with that line break

181
00:13:57,860 --> 00:13:59,500
syntax just like so.

182
00:13:59,750 --> 00:14:05,180
And at this point the classification label can successfully print out not only classifications at the

183
00:14:05,180 --> 00:14:11,010
top but also all of the guesses that it has for what the item in the picture is.

184
00:14:11,170 --> 00:14:13,790
OK so that's pretty cool.

185
00:14:13,790 --> 00:14:20,900
That's a good way that we can do this so we're now ready to process our classifications in our model

186
00:14:20,900 --> 00:14:21,200
here.

187
00:14:21,200 --> 00:14:28,160
So let's actually use that we can pass in self datt process

188
00:14:30,020 --> 00:14:39,770
classifications for will pass on our request and then error and we'll pass in our error parameter from

189
00:14:39,770 --> 00:14:40,090
a peer.

190
00:14:40,090 --> 00:14:43,350
So we're passing request and error because they're optional.

191
00:14:43,460 --> 00:14:48,590
Shows up as blue the the auto complete was not working but at this point we should be able to build

192
00:14:48,590 --> 00:14:55,500
this and get some requests back however I don't know about you but I want to try this.

193
00:14:55,490 --> 00:15:02,300
So let's build and run pull open our simulator here and if all goes well we should be able to select

194
00:15:02,300 --> 00:15:05,540
an image and then run our classification on it.

195
00:15:05,810 --> 00:15:06,370
OK.

196
00:15:06,560 --> 00:15:08,790
So let's go ahead and try this out.

197
00:15:08,810 --> 00:15:15,020
We will select an image from our photo library and nothing happens.

198
00:15:15,020 --> 00:15:21,850
And that is because we have not yet set it to update our UI but I want to see if this actually is working.

199
00:15:22,040 --> 00:15:26,390
So we're going to head over to the next video where we're going to write our update classifications

200
00:15:26,390 --> 00:15:31,910
function and actually get the information showing on the screen for the classifications that we're trying

201
00:15:31,910 --> 00:15:32,210
to make.

202
00:15:32,210 --> 00:15:34,400
So let's head over there now and let's do that.

203
00:15:34,460 --> 00:15:34,910
I'll see there.