1 00:00:06,090 --> 00:00:11,950 Hey everyone Caleb here with Deb slopes and in the last video we finally printed out our bounding box 2 00:00:12,310 --> 00:00:17,290 and since we successfully printed it out that means that vision is already doing the work to identify 3 00:00:17,290 --> 00:00:23,500 faces and it's telling us it's giving us a treasure map telling us exactly where that face is on the 4 00:00:23,500 --> 00:00:28,720 image or at least where it thinks it is and we're going to use these coordinates and these width and 5 00:00:28,720 --> 00:00:34,330 height values to actually drop this on the screen so that we can actually see what is going on where 6 00:00:34,330 --> 00:00:39,880 the face is on our image so I'm going to go ahead and remember that bounding box parameter and delete 7 00:00:39,880 --> 00:00:41,220 this for now. 8 00:00:41,440 --> 00:00:46,060 Then what we're going to do is we're going to go ahead and create a quick little function that is going 9 00:00:46,060 --> 00:00:54,040 to allow us to draw up a view on our screen and it's going to be a nice you know outline a yellow outline. 10 00:00:54,100 --> 00:01:01,540 So let's build that function now create a face outline but we're going to need to pass in a rectangle 11 00:01:01,540 --> 00:01:04,970 that's going to need to know for you know what rectangle. 12 00:01:05,020 --> 00:01:09,720 And so we'll pass in a C-g recked just like so. 13 00:01:10,350 --> 00:01:12,410 And now we can go ahead and create the view. 14 00:01:12,490 --> 00:01:14,660 So let yello view. 15 00:01:14,680 --> 00:01:20,830 It's going to be yellow and that's going to be you view of course now we need to set the background 16 00:01:20,830 --> 00:01:21,640 to be clear. 17 00:01:21,700 --> 00:01:22,730 So yellow. 18 00:01:22,930 --> 00:01:29,940 Background color is clear and it'll be transparent at that point yellow view. 19 00:01:29,950 --> 00:01:37,360 We want to set the border color so layer dot border color that's going to be equal to you eye color 20 00:01:37,570 --> 00:01:43,810 dot yellow dot C-g color just a weird requirement of the border color has to be a C.G. color because 21 00:01:43,810 --> 00:01:47,800 I guess the border is drawn using core graphics which is pretty cool. 22 00:01:47,830 --> 00:01:50,390 We're going to set the border with so do yellow. 23 00:01:50,410 --> 00:01:57,370 If you layer that border with and let's set that to something like three and then we want it to actually 24 00:01:57,370 --> 00:02:03,610 be rounded If you think about the the iOS camera when it identifies the face it puts a yellow outline 25 00:02:03,640 --> 00:02:05,900 and there's a bit of a rounded edge on the corners. 26 00:02:05,920 --> 00:02:12,350 So we're going to do the same thing yellow view layer dot corner radius and we'll set that to be five. 27 00:02:12,360 --> 00:02:15,210 Just very small rounding but still nice. 28 00:02:15,520 --> 00:02:20,460 And then we're going to go ahead and set the alpha or the opacity of this layer to be zero. 29 00:02:20,560 --> 00:02:24,360 And then as soon as we're ready to add it we're going to fade it in nicely with an animation. 30 00:02:24,370 --> 00:02:29,710 So yellow of you dot Alpha will be zero point zero at the beginning. 31 00:02:30,100 --> 00:02:35,250 And then we're going to go ahead and set the frame of this to be the rectangle that we pass in that 32 00:02:35,250 --> 00:02:39,750 will actually tell it where it should go and how wide and tall it should be. 33 00:02:39,790 --> 00:02:44,830 So yellow frame we're going to set that to be the rectangle and then we're going to go ahead and add 34 00:02:44,830 --> 00:02:46,060 it to our view. 35 00:02:46,070 --> 00:02:54,790 So self dot view adds subcu and we'll add in the yellow view but at this point it is transparent because 36 00:02:54,790 --> 00:02:56,650 we have set the Alpha to zero. 37 00:02:56,800 --> 00:02:59,540 So let's animate it back in once it's been added. 38 00:02:59,560 --> 00:03:06,230 With you I view it animate with duration and the duration will be zero point three seconds. 39 00:03:06,400 --> 00:03:11,440 And the animations block we can just push enter on and now we can say what things we want to actually 40 00:03:11,440 --> 00:03:19,720 modify at this point if you think back to our app here this label and the you know indicator here they 41 00:03:19,720 --> 00:03:20,560 should they should stop. 42 00:03:20,560 --> 00:03:21,760 They should stop spinning. 43 00:03:21,790 --> 00:03:22,900 It should hide away. 44 00:03:22,900 --> 00:03:29,110 So we're going to go ahead and actually first fade in our yellow view yellow dot Alpha and set it to 45 00:03:29,110 --> 00:03:32,110 be zero point seven five seventy five percent. 46 00:03:32,170 --> 00:03:33,500 Should be good. 47 00:03:33,550 --> 00:03:40,410 We will then go ahead and call self dot spinner dot Alpha and that will be equal to zero point zero. 48 00:03:40,410 --> 00:03:42,100 We want that to be completely hidden. 49 00:03:42,400 --> 00:03:49,940 And same with our message label will set that to be zero point zero so that it'll fade away nicely. 50 00:03:49,990 --> 00:03:58,080 But remember we called that spinner to actually hide when it is when it's stopped. 51 00:03:58,270 --> 00:04:03,970 And so I'm going to go ahead and once that's done once that animation is done I mean we're going to 52 00:04:03,970 --> 00:04:06,990 go ahead and actually stop it from animating. 53 00:04:06,990 --> 00:04:10,260 So we now can create a face outline. 54 00:04:10,440 --> 00:04:11,070 OK. 55 00:04:11,320 --> 00:04:17,410 And we're going to go ahead and actually call that here create face outline for CEG recked And now we 56 00:04:17,410 --> 00:04:20,990 need to actually generate our rectangle using these values. 57 00:04:21,010 --> 00:04:31,660 So let's go ahead and let's create a face rectangle so type let's face rectangle equals C-g rect and 58 00:04:31,660 --> 00:04:34,590 of course we need one with x y width and height. 59 00:04:34,840 --> 00:04:42,280 And for now let's just say 0 0 0 and let's just give it a width of one hundred and a height of 100 and 60 00:04:42,280 --> 00:04:45,940 we'll pass in that that face rectangle into our function. 61 00:04:45,940 --> 00:04:52,630 This is just kind of to test to see how everything works and making sure it's all good. 62 00:04:52,630 --> 00:04:53,850 So you know what. 63 00:04:53,850 --> 00:05:01,430 Actually I think after this I'm going to go ahead and just tell the spinner to stop spinning. 64 00:05:01,960 --> 00:05:02,920 There we go. 65 00:05:03,280 --> 00:05:12,900 OK so now it should hopefully fade in a nice view at 0 0 and then the rectangle should be 100 and 100 66 00:05:13,390 --> 00:05:14,500 0 0 0. 67 00:05:14,610 --> 00:05:16,140 I needa call self here. 68 00:05:16,140 --> 00:05:18,700 There we go because we're in a closure. 69 00:05:18,870 --> 00:05:25,080 And now if I build this we should get a little rectangle that pops up on the screen up in the top right 70 00:05:25,080 --> 00:05:26,140 corner. 71 00:05:27,330 --> 00:05:29,620 So here we go and look at that. 72 00:05:29,640 --> 00:05:30,510 That's exactly right. 73 00:05:30,510 --> 00:05:36,000 One hundred one hundred It's sort of transparent you can kind of see the hair through there. 74 00:05:36,090 --> 00:05:38,320 Very very cool stuff. 75 00:05:38,340 --> 00:05:42,010 So it is successfully adding the image. 76 00:05:42,420 --> 00:05:47,100 But did you notice that when we actually run the app the screen stays white for quite a long time and 77 00:05:47,100 --> 00:05:51,620 we don't actually get to see the interface until the work has already finished. 78 00:05:51,860 --> 00:05:54,630 So the boom done that is a problem. 79 00:05:54,630 --> 00:05:59,690 And the reason that this is happening is because we're doing everything on the same thread. 80 00:05:59,940 --> 00:06:05,370 So we're going to actually do some work here to ensure that this this you know continues to work the 81 00:06:05,370 --> 00:06:11,220 way we want and in order to actually do that we're going to go ahead and put this out on the main thread 82 00:06:11,700 --> 00:06:14,550 the UI related stuff. 83 00:06:14,610 --> 00:06:16,170 So we're going to go ahead and say dispatch. 84 00:06:16,170 --> 00:06:24,900 Q Like so dot main dot async we're going to do this asynchronously and I'm going to go ahead and paste 85 00:06:24,900 --> 00:06:25,990 that in there. 86 00:06:26,010 --> 00:06:31,950 And now we build and run this we should see that the UI appears and that changes 87 00:06:34,520 --> 00:06:35,140 OK. 88 00:06:35,300 --> 00:06:37,130 Let's see. 89 00:06:37,130 --> 00:06:43,530 Interesting so I saw a flash of the label and I saw a flash of the spinner right before they went away. 90 00:06:43,670 --> 00:06:51,780 But I want to make sure that everything is I want to make sure that everything is visible the time it 91 00:06:51,780 --> 00:06:52,290 should be. 92 00:06:52,290 --> 00:07:00,280 So you know what I think that this vision request I think I can pull this back on a background thread 93 00:07:00,790 --> 00:07:04,960 so that it happens in the background and all of the related stuff happens in the foreground. 94 00:07:04,960 --> 00:07:06,330 So go ahead and type dispatch. 95 00:07:06,350 --> 00:07:09,130 Q dots global. 96 00:07:09,130 --> 00:07:15,170 And are we going to use q OS which is quality of service and background is one of those. 97 00:07:15,190 --> 00:07:19,090 Q OS classes and it's something I want to happen in the background. 98 00:07:19,210 --> 00:07:24,640 So this of course will be asynchronous and I'm going to pass in that vision or quest but it's going 99 00:07:24,640 --> 00:07:29,770 to yell at me that since I am now in a closure I need to make a reference to self. 100 00:07:29,830 --> 00:07:33,530 And so now if we build and run this we should see the interface first. 101 00:07:33,610 --> 00:07:38,860 All of the vision or quest stuff happens in the background and the UI is actually happening in the foreground 102 00:07:38,860 --> 00:07:41,280 so as you can see there it is. 103 00:07:41,290 --> 00:07:48,160 And as soon as the rectangle is done we will see that actually pop up right on the screen like so. 104 00:07:48,160 --> 00:07:49,470 Very cool stuff. 105 00:07:49,480 --> 00:07:53,940 So what we couldn't see before we now can thanks to threading. 106 00:07:54,250 --> 00:07:55,260 Nice. 107 00:07:55,270 --> 00:07:57,600 So that is now in place. 108 00:07:57,880 --> 00:08:02,510 And so we need to basically get this rectangle showing up in the right spot. 109 00:08:02,530 --> 00:08:03,880 We have a face observation. 110 00:08:03,880 --> 00:08:05,530 We have a bounding box. 111 00:08:05,620 --> 00:08:10,690 But now we need to actually set up the parameters to get these in their rightful places. 112 00:08:10,690 --> 00:08:14,500 So we're going to need a height a with an X and a Y value. 113 00:08:14,500 --> 00:08:19,920 So let's go ahead let's start with the width that's going to be a pretty simple calculation. 114 00:08:19,960 --> 00:08:25,930 So we're going to go ahead and type it with and that's going to be equal to self-comfort view framed 115 00:08:26,060 --> 00:08:31,780 with we're going to take the the width of the screen and what we're going to do is we're going to actually 116 00:08:31,780 --> 00:08:37,930 multiply it by face observation bounding box width. 117 00:08:37,930 --> 00:08:44,500 Now if you remember it said that the dimensions were normalized or sorry the coordinates or normalized 118 00:08:44,500 --> 00:08:46,500 to the dimensions of the process damage. 119 00:08:46,600 --> 00:08:52,750 And so what that means is we need to basically scale it up so that it matches the width what we're going 120 00:08:52,750 --> 00:08:54,250 to use the width of the view. 121 00:08:55,000 --> 00:08:59,530 And we're going to scale it to match the width of the image. 122 00:08:59,560 --> 00:09:05,350 And so if I pass that in let's pass in with if we build and run this you'll see that the width will 123 00:09:05,350 --> 00:09:05,800 change. 124 00:09:05,800 --> 00:09:11,110 It won't necessarily be 100 but if you think about it and if you were to move it over you should see 125 00:09:11,350 --> 00:09:15,500 that the width of the frame should fit the face perfectly. 126 00:09:15,820 --> 00:09:19,880 Let's see how we do. 127 00:09:19,890 --> 00:09:24,820 OK so it's a little skinnier that should fit right on her face which is awesome. 128 00:09:25,020 --> 00:09:32,440 So now we're going to go ahead and set up the height and height actually it's simpler than you'd think 129 00:09:32,480 --> 00:09:37,930 but we can't use the height of the view because that doesn't reflect the height of the image. 130 00:09:37,980 --> 00:09:44,160 And if you remember we set a property earlier called scaled height and that scaled high value basically 131 00:09:44,160 --> 00:09:47,700 ensured that the UI image view was the same size as the image. 132 00:09:47,910 --> 00:09:54,120 But if I tried to access scaled height right here I don't have access to it because we created it up 133 00:09:54,120 --> 00:09:56,880 here in our set up image view function. 134 00:09:57,090 --> 00:10:03,200 But what I can do is I can add a parameter to perform vision or Quest and I can pass that in right there. 135 00:10:03,200 --> 00:10:10,720 So for image with scaled height and that's going to be a CEG float value. 136 00:10:11,010 --> 00:10:12,620 That's what the height comes in as. 137 00:10:12,810 --> 00:10:18,450 So now we have access to scaled height but I need to actually pass that in here. 138 00:10:18,900 --> 00:10:25,730 And so if I pass in scaled height we should be able to access that down below if I build that it sees 139 00:10:25,760 --> 00:10:30,500 a good where is that function yet looks good. 140 00:10:30,520 --> 00:10:31,780 Very cool. 141 00:10:31,780 --> 00:10:37,360 And I can take my scaled height and then in order to scale it properly I can access face observation 142 00:10:37,970 --> 00:10:43,010 bounding box and pull out the height of the parameters we get from the bounding box. 143 00:10:43,090 --> 00:10:49,630 If I don't build it yet getting ahead of myself if I pass in the height value here and build and run 144 00:10:49,630 --> 00:10:55,930 this we should see that our face in the simulator the box matches the face much better. 145 00:10:55,930 --> 00:10:59,990 So it's going to do its thing its thinking and finding faces. 146 00:11:00,490 --> 00:11:01,900 Here we go. 147 00:11:05,040 --> 00:11:11,960 And it appears to still be 100 by 100 which is a little confusing. 148 00:11:12,000 --> 00:11:13,530 Let's think about what we're doing here. 149 00:11:13,530 --> 00:11:20,070 We are taking the wits we're taking the height we're passing it into our rectangle and we're creating 150 00:11:20,070 --> 00:11:22,700 a face outline for that rectangle. 151 00:11:23,220 --> 00:11:24,850 OK so that's good. 152 00:11:26,270 --> 00:11:30,520 Looks like looks like we're doing everything right which is which is awesome. 153 00:11:30,710 --> 00:11:34,420 Let's go ahead and let's modify the X and the y parameters here. 154 00:11:34,490 --> 00:11:39,260 So let's go ahead and create a variable for X and set it to equal to self. 155 00:11:39,370 --> 00:11:41,760 View of frame width. 156 00:11:42,140 --> 00:11:47,870 And what we're going to do is we're going to multiply that by the the origin point of the bounding box. 157 00:11:47,870 --> 00:11:55,400 So let's go ahead and type face observation bounding box origin dot X and that's basically going to 158 00:11:55,490 --> 00:12:01,010 you know use the width and it's going to take this normalized x variable and it's going to actually 159 00:12:01,010 --> 00:12:04,680 fit it so that it is in reference to this screen size. 160 00:12:04,700 --> 00:12:10,620 So if I pass in X here we should see that the box moves on the screen. 161 00:12:10,870 --> 00:12:16,640 It'll You know think a little bit and then it will put it in a better place should be lined up with 162 00:12:16,640 --> 00:12:19,540 the actual person's face in this image. 163 00:12:19,790 --> 00:12:20,600 Let's see how we did. 164 00:12:20,600 --> 00:12:21,370 OK. 165 00:12:21,500 --> 00:12:22,100 Beautiful. 166 00:12:22,100 --> 00:12:25,850 So as you can see it is perfectly aligned. 167 00:12:25,940 --> 00:12:28,360 Now we need to figure out the Y parameter. 168 00:12:28,370 --> 00:12:37,010 So let's go ahead and let's add the Y parameter let y and for y remember y is going to be going up and 169 00:12:37,010 --> 00:12:39,190 down and that has to do with the height. 170 00:12:39,230 --> 00:12:43,520 So we're going to use that scaled height parameter once more scaled height. 171 00:12:43,730 --> 00:12:49,960 And what do you think we're going to do you might think scale it by face observation bounding box that 172 00:12:50,000 --> 00:12:51,910 origin dot Y. 173 00:12:52,130 --> 00:12:53,130 And that's a good thought. 174 00:12:53,150 --> 00:12:55,630 Let's see what happens when we do that. 175 00:12:55,730 --> 00:12:57,660 Let's go ahead and build and run. 176 00:12:57,930 --> 00:12:58,490 OK. 177 00:12:59,590 --> 00:13:00,610 It is building. 178 00:13:00,610 --> 00:13:01,860 It's running. 179 00:13:02,110 --> 00:13:04,590 And let's see where that box ends up. 180 00:13:08,970 --> 00:13:11,460 OK so it is way too low. 181 00:13:11,490 --> 00:13:13,010 Now you might be wondering well why. 182 00:13:13,020 --> 00:13:16,500 Why wouldn't it be in the right place that doesn't make sense. 183 00:13:16,500 --> 00:13:21,240 Remember the bounding box it says that the origin is the images. 184 00:13:21,240 --> 00:13:23,820 Lower left corner right. 185 00:13:23,850 --> 00:13:25,280 So what that means. 186 00:13:25,290 --> 00:13:26,590 Think about where this box is. 187 00:13:26,610 --> 00:13:34,050 If you were to flip the image upside down or I guess if this clear layer with the yellow box if you 188 00:13:34,050 --> 00:13:36,930 flip that around it would be right over her face. 189 00:13:36,930 --> 00:13:42,120 If you think about the distance from here to here and you were to focus here to here that brings it 190 00:13:42,120 --> 00:13:47,910 to exactly where her face is on the bounding box is actually upside down. 191 00:13:47,910 --> 00:13:54,360 So what we're going to do to fix that is we're going to take one hundred OK or one because these values 192 00:13:54,360 --> 00:14:02,070 are all on a percentage basis we're going to take one and we're going to subtract one from this so one 193 00:14:02,130 --> 00:14:03,990 minus whatever this is. 194 00:14:03,990 --> 00:14:07,110 And that's going to get the remainder of where it should be. 195 00:14:07,110 --> 00:14:10,240 So let's go ahead let's build and run that one more time. 196 00:14:10,260 --> 00:14:12,450 That should help invert it up to the top. 197 00:14:12,450 --> 00:14:13,710 We will get the. 198 00:14:13,970 --> 00:14:16,640 The other side of those values. 199 00:14:16,740 --> 00:14:17,790 It's loading. 200 00:14:17,790 --> 00:14:19,610 Doing its thing. 201 00:14:19,630 --> 00:14:24,110 Let's see. 202 00:14:24,140 --> 00:14:31,570 OK so we are super super close to where we need to be it's still too low though. 203 00:14:31,790 --> 00:14:39,590 And you know it sounded like we were on the right track but what we actually need to do is the bounding 204 00:14:39,590 --> 00:14:43,160 box its origin point is zero zero. 205 00:14:43,250 --> 00:14:43,850 Right. 206 00:14:44,060 --> 00:14:51,170 And zero zero point is here or at least the origin point is the top left corner but our bounding box 207 00:14:51,170 --> 00:14:57,110 the origin point is down at the bottom so we need to actually delete or subtract the height of the box 208 00:14:57,170 --> 00:14:57,710 itself. 209 00:14:57,710 --> 00:15:00,520 So we'll use the height parameter to do so. 210 00:15:01,190 --> 00:15:09,050 And once we subtract the height we should be able to see that our height should be perfect on the y 211 00:15:09,050 --> 00:15:15,860 axis and it should go right over our subjects face finding faces here we go 212 00:15:20,400 --> 00:15:28,180 and boom the labels hide the spinner hides the box is perfectly over our persons face here. 213 00:15:28,260 --> 00:15:31,410 That is seriously so so cool. 214 00:15:31,770 --> 00:15:32,490 I love it. 215 00:15:32,730 --> 00:15:37,440 But I want to make sure that this works with more than just one face we pass in a request. 216 00:15:37,450 --> 00:15:39,630 But how many faces can handle. 217 00:15:39,870 --> 00:15:46,320 If you remember we have a Faces image here and so I'm going to change the character to s that will switch 218 00:15:46,320 --> 00:15:52,130 out the image and we'll see if we get a similar result with multiple faces in the same image. 219 00:15:52,150 --> 00:15:52,980 Let's check it out. 220 00:15:53,350 --> 00:15:53,630 OK. 221 00:15:53,640 --> 00:15:56,410 There's our image it's finding faces. 222 00:15:57,820 --> 00:15:58,950 And what do we get. 223 00:16:05,020 --> 00:16:08,420 It's taking a bit longer I guess because there are multiple faces but look at that. 224 00:16:08,560 --> 00:16:14,540 We get all four faces perfectly identified and they drop in at the exact right coordinate. 225 00:16:14,560 --> 00:16:16,360 Guys this is so so cool. 226 00:16:16,360 --> 00:16:23,370 We have now successfully identified faces and put a bounding box over them in an image in a future section 227 00:16:23,380 --> 00:16:27,970 we're going to actually draw the lines for the facial features that you see in a photo. 228 00:16:28,270 --> 00:16:34,870 And beyond that we're going to use a live view to actually track and identify user's faces on the camera. 229 00:16:34,870 --> 00:16:36,400 So pat yourself on the back. 230 00:16:36,400 --> 00:16:37,810 Fantastic work. 231 00:16:37,810 --> 00:16:43,360 We have now successfully built an app that can analyze and recognize faces in photos using the vision 232 00:16:43,360 --> 00:16:44,440 framework from Apple. 233 00:16:44,500 --> 00:16:46,690 Let's head over to the next section and let's keep learning.