Transcription
Hello guys, this is Paul McQuarter with TopTechBoy.com, and we're here today with episode number 23, the Homework Solution Edition, in our incredible new tutorial series where you are going to learn artificial intelligence, or you're going to die trying.
What I'm going to need you to do is pour yourself a nice tall glass of ice-cold coffee. That would be straight-up black coffee poured over ice, no sugar, no sweeteners, none needed. And as you're pouring your coffee, as always, I want to give a shout-out to you guys who are helping me out over at Patreon. It is your support and your encouragement that keeps this great content coming. You guys that are not helping out yet, take a look down in the description, there is a link over to my Patreon account. Think about hopping on over there and hooking a brother up.
But enough of this shameless self-promotion, let's talk about what I am going to teach you today. And what I'm going to teach you is, I'm going to teach you what you need to know in order for us to sort of wrap up our core media pipe program. Now, in the earlier lessons, you have seen that MediaPipe returned some really odd and some really bizarre data structures, and it's very hard to parse that data, and it's not logical. At least for us normal people, it's not very logical in how they stack that data together. So, what we've been doing in the last lessons is we have been busting open that data structure and creating very simple, elegant arrays of data that have the landmarks in them and have them in a way that's very intuitive to interact with them.
So, what we will do moving forward is after we finish this program, we will have the classes, the core classes that we need in order to parse the data coming from the hands, the hand landmarks, the pose landmarks, the full-body pose landmarks, and now the face landmarks, the face mesh landmarks. And so now, after today's solution, we will have everything that we need to very easily use MediaPipe in our, uh, in our various projects. Okay.
Now, I got to tell you, this lesson, it's not going to be hard, but it's going to be a little more tedious than some of the earlier ones because we're going to start with that last program that we had where we were already parsing face position, hand landmarks, and body pose landmarks, and now we're going to add to that a class that will do the face mesh. So, it's not that hard, but it's just sort of adding on to a program that's already getting kind of big. And then after we add that class, then we'll go down in the main loop, in the main program, and kind of easily do a couple of things that will allow us to sort of see what's going on with that face mesh. I hope this makes sense.
But anyway, I'm going to need you to kind of really get ready to concentrate and learn some cool new stuff. So, what we're going to do is we are going to get me out of the way. We are going to switch over to my desktop, and then you are going to fire up the most excellent Visual Studio Code. We are going to come over here to the file explorer view. We are going to be working in our main Python folder. We are going to add a file. We are going to add a program that is going to be openCV-dash-is it 40? Could we already be on our 40th program? I believe so. And then dot py. And the dot py is kind of important. And then fresh, new Python program, just waiting to be written.
Now, this behemoth, we don't want to start from scratch. So, we're going to start with that last program that we had in lesson number 22, where we had the class, we had the class structure for the hands, for locating the face, and for the body pose. And so, let's just start from that. So, I'm going to need you to hop over to the most excellent www.toptechboy.com, and then you're going to search on something like "parsing MediaPipe data for pose landmarks, hand landmarks, and face bounding box." Now, this is the most recent one, and make sure you might just search on that whole term there to make sure that you get to this lesson, and it has all of those classes. We'll check. Yes, I've got the MP face, I've got the MP pose, and I've got the MP hands, uh, structure.
So, now we're going to come over here, and we are going to come to these two little boxes, and we're gonna click on the two little pages, copy to clipboard. Now, we're gonna come over to our Visual Studio Code, and we are going to paste it in. And so, now what this program should do is this should do something like box the face. It should do something with the hands, and it should do something with the body pose because when we wrote the program, it was just sort of interacting with all three of those classes. And it's not so much important what is happening down here as it is what's happening up here where we define the classes. Once those classes are defined, it's very easy to do this stuff down here. But the nice thing about this, it just kind of gives you a simple example of interacting with those classes that we set up.
I'm going to have some copy. Okay, just to make sure that the universe is in proper working order and that we can copy and paste without breaking something, we're going to run this. Okay, there it is. My face is boxed. Okay, it knows the difference between my left hand and my right hand, and then it looks like that we are getting. Oh, I remember I said to do, uh, wrist and elbows. And so, it's not doing the whole pose, it's just doing wrists and elbows and labeling the left and the right hand. So, that looks pretty good. That actually looks like it is working pretty well.
What we want to do is we want to add to that the face mesh. What is the good news? The good news is we know how to interact with MediaPipe face mesh because we did that when we did that in lesson number 23. But now we're just going to take what we learned in lesson number 23, and we are going to build what we are going to build a class. And then what are we going to call that class? We are going to call that class MP face mesh. Why mesh? To distinguish between that and MP face, which just drew a box around the face. So, we are going to look at class MP face mesh as such.
Now, what are we going to need to do? Well, this class is going to need to import MediaPipe as MP. You should be very familiar with that. And there is some large flying object buzzing around my head here, but I'm going to power on through it and try to ignore it. Okay. Now, anytime you have a class, you have to set up your init method. Your init method is the method that is run whenever you invoke the class, when you create an object based on the class. At the creation of that object, what will it do? It will go in and it will run the init method. So, I'm going to say define underscore underscore init underscore underscore. And now I've got to pass it some parameters. Well, I want to pass itself. So, it knows who it is. So, if I create a box call, if I create an object called dog, self would be dog. If I create an object called cat, self would be cat. So, self is whatever you call the object that you create when you call MP face mesh. This doesn't make sense? Go back to the earlier Python lesson where I explained functions, methods, and classes, and you can get up to speed on that.
Then I'm going to need to pass it. I'm going to need to pass it some parameters that are going to be used later on, some parameters that are going to be used later on. The first I said was self. Then also, we need to tell it. We need to tell it whether we are going to be using a still object, a fixed object, or a live video. Well, you'd set still. Is it still? Well, the default value I'm going to put as false. And so, if I don't pass it a still parameter, it will use false. Okay. And so, these things that I'm defining in here are the defaults in case the numbers do not get passed in. Okay.
Now, also, it wants num_faces, and I'll just set the default for number of faces to three. Okay. And then we have a tolerance one, which is equal to 0.5. And then we have a tolerance two, which is equal to 0.5. And then we need to pass it a parameter whether we want it to draw the mesh. Remember, there's that MP draw method that will actually draw the mesh automatically on the face. Well, if we're going to do that, we ought to actually do it inside of this. So, we need to pass to the class whether we want it to, to draw out or annotate the landmarks or not. And so, I'm going to initially set that to true. So, it will just, as a matter of default, unless told otherwise, it will draw_mesh is equal to true. So, unless told otherwise, it will draw the, draw the face mesh. Okay. So, that is our init.
What are we going to need to do in our init? Well, we are going to need to set up our MediaPipe objects. So, the first one, always, we're going to put self here. So, like, you know, if I call the object my_face_mesh, then itself would be my_face_mesh. So, so itself. And I shouldn't have said my_face_mesh because that's actually the name that I'm using next for the actual MediaPipe object, which is my_face_mesh. And that's going to be equal to what? Self.mp.solutions.face_mesh. Wait for it. Face mesh. And then I need to pass it those parameters that I had passed in when I created the object or invoked the class. And so, what was that? It needs to know still. It needs to know num_face, num_faces, and it needs to know tolerance one and tolerance one. [Music] Tolerance two. Now, I don't pass this. I don't pass this object draw_mesh because that's going to be used somewhere else when it comes time to decide, do we want to draw it or not? I hope that makes sense. I'm looking at that buzzing object and trying to decide whether it's dangerous or not. I think we'll be okay.
All right, so I create the my_face_mesh. Now, I also need to do a self.my_draw. Okay. And then that is going to be equal to self.mp.solutions.drawing_utils. All right. I'm not explaining all this in a lot of detail, and the reason I'm not explaining it, I explained all this last week. So, I'm taking what you learned in the last lesson and I'm just turning it into a class. If I'm going too fast on this stuff, go back to, go back to the last lesson. All right. And now, I also need a self.draw. And that is going to be equal to draw_mesh. Now, let me explain that. All right. So, I passed in draw_mesh, that's whether I want to draw it or not. Now, what I've got to do is I kind of got to make it a class variable so the other methods will know what it is that whether I want to put the mesh on or not. That is going to come in. That was not good. That is going to come in here. But then I've got to take it where everybody, not just init, but everybody knows what I want to do. So, I create the kind of class variable, self.draw, and then that is going to be equal to draw_mesh. Now, that should make that data available to the other methods that are going to be in this class. If I'm going too fast, go back and look at my lesson, my earlier Python lesson on methods, classes, and functions. All right.
So, now I've got all those things set up. So, basically, I think that's what we want to do when we create an object, when we invoke this class. And now, what we got to do is we've got to kind of do the business end where we actually go in and analyze a frame. So, we've sort of got our framework set up. But now, if we pass it a picture, how do we, how do we, uh, analyze that picture? Well, I've been calling that thing. I'm going to kind of keep with my naming conventions. I've been calling that method marks. Okay. And what does marks need? It needs to know who it is, self. It needs to know its own object name. And then it needs to have the frame passed to it. I don't have to pass draw_mesh, okay, because draw_mesh was put into self.draw, and this method here will know about self.draw. All right.
Now, I'm going to need to do this analysis. This method needs to understand what the width and the height of the overall frame is, right? And we sort of in our old program, we always set the width and the height, but that is on the outside. I've got to make sure that inside this class, it has access to that. So, I need to make it a global variable. What do I need to make global? I need to make global width. And then global height. Now, inside this method, if I talk about width and height, it will use this width and height here. This should make sense if you guys have been taking my class. Okay.
Now, I have my, I have my frame because I passed it in when I call, whatever my object name dot marks is, whatever whenever I call that, I'm going to have to pass it the frame. That frame is from OpenCV, so it is going to be BGR. The rest of the world operates in B. And so, I'm going to have to say frame_rgb is equal to cv2.cv2.cvt_color. All right. And what do I want to convert? I want to convert frame. And how do I want to convert it? cv2.cv2.color_bgr_to_rgb. Like that. Okay. Now, I got an RGB version of my frame, which is quite handy.
Now that I've got the frame, we're ready to analyze it. We're going to put the results in a data structure that we call results. It's going to be an array. Results. And results is going to be equal to. All right. Well, what is the object that I'm working with? Well, it's whatever I called it, which is self. Could be kitty litter box, could be something else, but whatever I named it, self. All right. And then what? My face mesh, because that is the name I gave the MediaPipe object. So, I'm going to use that. Self.my_face_mesh. And now, what do I want to do? I want to use that method to process. And what do I want to process with two S's? I want to process frame. Be careful. RGB. Right.
So, now at this point, I should have a frame and I should analyze it, and I should then for every face in that frame, I should have an array of landmarks. And so, we get to the data really fast. Now, we've got to kind of process the data. Well, we're going to need. And remember, this face mesh will find all the faces. It will find more faces than just one. And in fact, I think for our default, we said find up to three faces. So, I need an overall array which will include all the faces, the faces, and so I'm going to call this array faces_mesh_landmarks. And landmarks is plural too, because each of the faces will have lots of landmarks. And that's just going to be an empty array to start with. Does that make sense?
Now, we only want to do the analysis further if we only want to do the further analysis if, in fact, there is a face in there. And so, I'm going to say if results.multi_face_landmarks is not equal to None. Like that. Okay. How did I know results? Because I called it that. How did I know it was multi_face_landmarks was the right method here, or the right designator for the data? Because we looked at that last week, right? We looked at it in the last lesson. So, if you don't understand this, go back and look at the last lesson. Okay.
Now, we only do this if we have data. But if we have data, then we want to step through all the faces. We want to step through all of the faces. So, I'm going to say for face_mesh, singular, for face, singular, mesh, for face_mesh in results.multi_face_landmarks. All right. This results.multi_face_landmarks, it has all the landmarks of all the faces. Now, I'm going to step through all the faces, one face at a time. And the first face is going to be face_mesh. And then the next time through, it will be face_mesh. And the next time through, it will be face_mesh. Face_mesh is the variable where I hold one face at a time as I step through all of the faces. Okay.
Now, I need to get the location of the, I need to get the location of those landmarks. And so, what I'm going to do is I'm going to create a base_mesh_landmarks is equal to this. All right. So, this first array will be a group of arrays. An array. The this, this faces_mesh_landmarks, faces plural, will have an array for the first face, an array for the second face, an array for the third face. So, it is an array of arrays of landmarks. Now, this face, singular, mesh_landmarks, will be the set of landmarks associated with one face at a time. We've done this before. Hopefully, this is making sense. Okay.
So, now we have that. Now, we want to step through. Now, I've got one face. I want to, now for that face, step through the landmarks. So, I'm going to say for lm in landmarks in face_mesh.landmark. And we know it's dot landmark because we looked at the data structure in the last lesson. Again, you could figure that out if you looked at the last lesson. Now, where is that landmark? What is its location? Well, it is going to be equal to, in a sort of simple term, the XY position would be lm.x, lm.y. Like that. That would be the location. We have a tuple. We have an X, Y coordinate of the first landmark. lm.x, lm.y. But what do we know from last week? The lm.x and lm.y are a number between 0 and 1, where like full screen is normalized to 1. Well, we don't want that. We want the actual pixel position. So, lm of x, I need to turn back to actual pixels by multiplying by what? Width. And it knows what width is because I made it a global variable above. And then landmark.y times what? Height.
Now, what is the problem? The problem is we're taking a floating-point number and multiplying it by an integer. That will give us a floating-point number. Later on, OpenCV always wants integers on those pixel locations. And so, we very well could cause a crash. So, what we want to do is we want to force these to be integer values so that our program won't crash. So, that says, what? That would say is, if you ended up with the 29.23, 20, the 29.23 pixel, it would just turn it into the 29th pixel because there is no 29.23 pixel. You're either the 29th pixel or the 30th pixel. So, it's rounding everything to the nearest pixel. And similarly for the Y value, we'll need to do that. Make sure you have your parentheses right. So, this parentheses closes the int, and then I need to close the tuple. All right.
Now, I've got the location. What do I need to do with that location? Well, I need to say face_mesh_landmarks. So, this is the singular face. I'm going to start building this array with all of those data points. Face_mesh_landmarks.append. Append. And what do I want to append? I want to append that location tuple that I just created. Now, after you get done with this face, uh, after you get done with this for loop, what you're going to have is one array that has all of the XY tuples of all of the landmarks for one face. All right. For one face. But now, what I have to do after I leave that for loop, I'm going to have to do faces.mesh_landmarks. Faces, not dot, but faces_mesh_landmarks.append. What am I going to need to append? Well, that array that I just created, face_mesh_landmarks. And that should be singular. So, I'm adding the face to the array called faces. So, faces becomes this face, and then I append this face, and I append this face. Okay. And I've got some sort of crazy face mesh. I really messed that up, didn't I? It's face_mesh_landmarks, man. I messed that up, didn't I? All right. So, that looks good. So, now I should have an array, faces, faces_mesh_landmarks, which are all of the faces put together. All right.
So, now I've just got to decide whether I've got to decide whether I actually want to do the automated mapping of the landmarks on the face. And so, I'll say if self.draw is equal equal true. Now, what's that? Self.draw. Well, that's what we set up up here, which brings draw_mesh into the overall class, right? We passed it draw_mesh, and then we put draw_mesh and self.draw, and then here I can use self.draw. I'm saying, if self.draw equal true, well, what do I want to do? Self.my_draw.draw. [Music] Landmarks. Where do I want to draw them? On the frame. And then what do I want to draw? Face_mesh. Okay. Now, why face_mesh? Because face_mesh is the overall face that is still in the MediaPipe data format. I don't pass it my simplified data format because it's expecting the MediaPipe complicated data format. So, I pass it that. Okay. Like that. That looks pretty good to me. That looks pretty good to me.
All right, let's just run this and see if we get any errors. It's not going to do anything, and it probably isn't going to get an error. Don't get too excited because we didn't ever actually invoke this class, and because we didn't invoke it, it might not even find errors in there. I'm not for sure. So, what we need to do is go ahead and create an object based on that. And so, we come down here. Where did we create our objects? We created our objects down here. So, I created the find_hands, I created the find_face, I created the find_pose. And now, what we should do is we should create a find_mesh, which will find the, the landmarks of the face mesh. And we will call that MP face mesh. And then I'm going to go ahead and say draw_mesh equal true. So, I'll say, yeah, draw the, you know, draw it after you find it. Now, why do I call this MP face mesh? Because that was what I named the class up here. Okay. This one might actually get us an error. I'm just trying to go one little step at a time here. Okay. So, we were able to, you know, invoke that class without it, it crashing. Now, we'll actually need to come down here in the while loop and do something.
So, I say face_location is equal to find_face.marks(frame). Pose_landmarks is equal to find_pose.marks(frame). And now, what I can do is I can come down here and just say find, let's see, what would be a good thing to call that? Uh, we'll, we'll call it, we'll call it faces_mesh_landmarks. Like that. And that is going to be equal to find_mesh, which was the object that we created. Dot marks. And then what we need to pass it, we need to pass it frame. Now, this one might actually do something because since I set it to true on plot, it will probably actually put all those points on my face. But we will see. Hey, boom! Look at that.
Okay, so let's see what we can do now. I boxed the face. I put all the landmarks on the face. And I've got a left hand and I've got a right hand. And then I've got elbows and wrists. Okay. So, I'm using all of those different methods. And when I look at it, we still have a really nice smooth frames per second there. So, that is really looking good. So, we have that class setup. Let's just make sure that we kind of have our data right. So, I'm going to come in here. Instead of saying draw_mesh is true, I'm going to say false. And now it'll still be doing the analysis, but it just shouldn't, it probably won't draw. It probably won't draw those dots this time. Okay. So, I still got, I've got a right hand, I've got a left hand, I've got elbows and wrist, got a box around my face, but I'm not drawing, I'm not annotating that face mesh on my face. Okay.
So, we do need to probably do something kind of interesting. And so, what we can do down here after we've done, so we're going to do the pose. Okay. And then we are going to do the face. And then we are going to do the hands. And so, now what we can do is we can do something with our face mesh. And so, what I think I will do is I will say for. And remember that when we call, when we called the find_mesh.marks, we put the data in an array called faces_mesh_lm. So, then what we're going to do here is we're going to step through that. So, I'm going to say step through face_mesh_landmark. So, this is going to be the first face in faces_mesh_landmark, which is what I got back from the method. Faces_mesh_landmark. So, face singular is stepping through faces plural. [Music]
Now, what we want to do is we want to go in and I just want to put a number on each one of those, like print the index at the face location. So, instead of a dot, I want to put the index. Okay. So, if I'm going to do that, I've got a count. I've got to say cnt is equal to 0. So, I've set up an index counter. So, cnt is equal to 0. And then I'm going to step through the landmarks for lm singular in all the landmarks on the face. Okay. I'm going to step through each landmark at a time. And then I'm going to do a cv2.put_text. And where do I want to put it? I want to put it on the frame. And then what text do I want? I want to put count. But count is an int. So, for me to print it, I've got to change it to a string because OpenCV wants to print strings. Make sure you close that string. Okay. And now, where do I want to put it? Well, lm, that's the tuple of the position of that landmark. So, I'll put it at lm. And then we want a font, we want a font size, we want a font color, and we want a font thickness. Now, what is the problem? I haven't set any of these things up yet. I haven't set. How did it recognize font color? It shouldn't know that. Maybe there was another place I did it, but font, font size, font color, and font thickness. Let's see. Maybe I did a few of those. I think I'll say font_thick, font_size, and font_thick somewhere I set up a couple of fonts. So, I need to add font_size and font_thick. Okay. There's font, font_color. Okay. So, here I'll say font_size is equal to, I want it small. I'm going to say like 0.2. And then font_thick, we'll just make equal to one. Like that. No. Okay. It should know. It should know all that. And then I closed it. So, let's see. What should this do? This should put numbers. What color numbers? It should put blue, green, red. It should put red numbers all over my face. Ah, 116. It's not iterable. What do you mean it's not iterable?
Okay, so this is sort of perplexing because if I just run the function, if I run the class, the object, it does the analysis, it puts, puts the annotations on. But if I come down and try to do it, it doesn't do it. And what it's saying is, it's saying that this faces_mesh_landmark is not iterable, which means it's like there's no data in it. But I actually got it. I got faces_mesh_landmark by calling find_mesh.marks method. And so, I come up here, and so I'm calling this. And if I set the draw to true, it does do it. So, it has the data. But what did I never do? You guys should have been screaming at me. What did I not do? I did not return. I did not return the data that I had put together. And so, what we need to do here is it would be lined up with this. If we need to, when all this is done, we need to return what? Faces_mesh_landmarks. Like that. We need to return faces_mesh_landmarks. What did I do wrong? Oh, okay. I think I didn't, I didn't do several things. Okay. Yes, I did faces, faces_mesh_landmarks. I need to append this. And somehow I have it not spelled right. So, sometimes you just want to go like that. And I didn't spell that right. Okay. Maybe it's because I didn't spell return right. So, that might be it.
Now, let's actually try it here. Okay. Boom! Look at that. And now, what is the problem? It is putting the numbers there. It is putting the numbers on my face, but it's putting all zeros. Now, why is it putting all zeros? Let's come back down here. Okay. So, ah, what did we forget to do? We forgot to increment count. We forgot to increment counts. We need to do that. Uh, I don't know if you can hear that ruckus outside. The locals are getting a little restless. Hopefully, we don't have any trouble out there. Had trouble yesterday. There was like a six-foot lizard in the, uh, in the path. The locals became quite animated over that. And I think they're still maybe a little excited today about that lizard that was out there yesterday. But I digress. So, what we forgot to do is we forgot to increment count. So, I need to say count is equal to count plus 1. Now, when I do this, I should have proper, I should have proper numbers on my face. And look at that. We do have proper numbers on my face.
Now, I'm not going to worry right now about looking at those numbers. But let's just see if we can do two faces. Okay. Let's see if we can do two faces. Okay. Yeah, you see we're completely analyzing two faces. So, that is good. So, we do have the data structures working right. Everything is working right there. Now, what is the problem? The problem is we have so many different numbers for us to do something, it would be kind of hard. And especially over the important areas like around the mouth and around the eyes and around the nose, there are so many numbers, they're hard to read. You can see that I really wouldn't want to make it very, I wouldn't want to make it very much smaller than that, the numbers. But also, like if I look, one of the things I see is it seems like it's finding the outline of the eye, but it's not actually finding the iris. And so, the, the maximum detail that it has is like, here's your eye, but it's not going in and giving you the location of the iris. Now, the iris could be interesting because then if you had the iris and the outline of the eye, you could start trying to kind of detect where you were looking. But now, the problem is there are so many numbers there. Like, if I wanted to go in and just highlight the eyes, or study the eyes, or study the mouth, the problem that I've got is there are so many numbers there, they're not readable. And making them smaller would make them even more not readable.
So, what we need to do is we need to kind of have a way that we could go in and we could look at a smaller number of those landmarks at a given time. Now, one way you could do it is you could just say that, you know, only for landmark and face landmarks, you could go from like an index of 1 to 10, and then 10 to 20, and 20 to 30, and you could keep monkeying with the program, monkeying with the program, and seeing where those landmarks actually are. But I think that it would indeed be a little easier if we had something that on the fly, we could adjust which landmarks that we were looking at. And so, that sounds like something that we learned some time ago, which was called what? Who was our friend? Who was our friend? Our friend was track bars. Do you remember the track bars? I want to have like a track bar that's that says like the starting, the lower index, and then a track bar for the higher index, and then only plot, only show the landmarks between the lower and the upper value. And that way, I can get some of those numbers off my face and see a little bit more of what's going on.
So, how would we do that? Let's create some track bars now. Okay, let's create some track bars. And so, what we would do is I'm just going to come in here above the while loop, and I'm just going to set, first of all, I'm going to set some variables that I will then be able to adjust in the track bar. But I'm just going to say lower_limit is equal to 0. And I'm going to say upper_limit is equal to 468 because that is the number of, uh, that's the index count that we would go to. There's 468 landmarks. And now I'm going to create a window to put the track bar in. So, I'll say cv2.named_window. And what do I want to call that window? I want to call it track_bars. All right. Like that. That's pretty good. Then I'm going to move it. cv2.move_window. And then which window do I want to move? I want to move track_bars. And then where do I want to move it? I think that I want to move it like, let's say, width, the width of the frame, you know, the width of my picture. I want it to the right of that. So, I'm going to say width, and just give it a little room, say width plus, let's say 50. See what that looks like. Okay. And then width plus 50, that's to the right. And then how far down? Zero down. Okay.
Now, I'm going to resize that window. cv2.resize. Because it sort of makes it a random size. I want to resize what? My track_bars window. And then what do I want to resize it to? Oh, let's say 400 wide by 150 tall. Something like that. So, now I should have a track bar window. And now I need to create my track bar. So, I'm going to do a cv2.create_trackbar. Right. And then what do I want to call this track bar? I want to call it lower. Lower limit. Okay. And then I want it to be in the window track_bars. And then I want it to start at 0. And I want it to go to 468. And then I want to call the function. I have to call a function when you monkey with it. It will be the function set_lower. Like that. All right. Now, it gives me an unhappy little squiggly because I haven't defined set_lower yet. But I'm going to go ahead and make my other track bar, which is going to be the upper limit. Upper limit. And it's going to be in track_bars. And it's going to have an initial value of 468. And a maximum value of 468. And this is going to be the function set_higher. Okay.
Now, I've got to define those functions. I will do that here before I start doing the track bars. And so, what I'm going to do is I'm going to define set_lower. Okay. And then I've got a, it's going to pass it a value, right? It'll pass it a value whenever you openCV gives it a value when you mess with the track bar, it sends into value what the track bar is now set at. And so, now I've got to make global the lower, lower_limit. I've got to make that global. And then I'm going to say lower_limit is equal to what? It's equal to value. All right. Now, I've got to do the same thing with set_higher, or set_upper. You've got to be real careful with variable names. If I'm saying upper, I've got to make sure I use upper and not change it to higher or something like that. So, set_lower. And then this is going to be set_upper. And let's make sure I did that. Ah, let's call it set_higher. I'm going to call it upper. Upper. Set_upper. That likes it. Set_upper. That likes it. And then the global is going to be the upper_limit. And then I set on this one, upper_limit to value. Get rid of a little bit of this white space here.
Now, this isn't going to do anything, but it should just set up the track bars. And let's just make sure nothing crashes before we do anything else. And it's not happy. Resize. It didn't like my resize. Track_bars, 400, 150. What did it not like there? Why is it being so picky on me? cv2.resize. Oh, it's resize_window. cv2.resize_window. Because this isn't a picture, it's just a window. So, that should, that should fix that, hopefully. Okay. So, there we are. And we have a track bar. And we can invoke the track bar. We can invoke the track bar, and it doesn't crash. Right.
Now, we've got to do something with these numbers that we're getting from the track bar, upper_limit and lower_limit. And that is going to be down here around this put_text. Okay. This put_text here on the face. I'm going to say if, if, if cnt is greater than or equal to lower_limit, and cnt is less than or equal to upper_limit, then what do you want to do? Then you want to label it. Okay. Otherwise, you don't want to label it. So, now, as I move those slider bars, it should show smaller amounts. And I should be able to, like, zoom in on my eye, or zoom in on my eye, or zoom in on my mouth. And then we could actually go in and read those numbers off. So, we could see what are those index values around facial features of interest. And again, what we're trying to do is we're trying to create something like this, which is the index value of the whole pose of the body. And this is the index value of the landmarks of the hand. Well, we want that same thing on the face, but they didn't provide us that image. So, we're having to kind of generate it ourselves, if that makes sense.
And so, let's come back over here. And now we have set that up. So, now let's just run it and see if that does what we want to do. Okay. We're going to run it. Okay. That's a good sign. We've got all of those values on there. We've got all those values on there. Now, if I start pulling in the upper values, it is cutting off like half of my face. Okay. So, now I'm just showing what is the left side of my face. If I bring it back and I cut these down, I can cut out the right side of my face. Okay. Like that. So, I can start cutting it down. But now, let's see if I wanted to just get like, let's say, my left eye. Let's see if I can just get my left eye. Okay. Do you see what is kind of annoying? It seems like these, it seems like these, [Music] besides it periodically thinking that my face is a hand, probably, kind of what the problem that I'm seeing is, is that these landmarks seem to be a little bit in random order. So, you see, if I'm trying to just zoom in more on my, on my eye, you see I start losing my eye before I start losing those other points. You see what I'm saying? The, the points around my eye are just not sequential.
But what you could do is you could still come in and like, was like 137 to 162 was pretty good. And let's just make that font bigger. Okay. So, font_size, instead of 0.2, let's make it one. And maybe that would be more readable. And I am going to make my life a little easier by using a different picture here so I can. Okay. So, I can look and see at the same time. So, now, whoa, that is a crazy lot of things that it is showing there. Okay. So, now let's see if we can zoom in on around the eye. And you can see what you would want to do is turn off that other nonsense so that it's not trying to put hands and other things in there. [Music] Okay. So, you see with that, I would still want to make the font a little bit smaller. I would want to make the font a little bit smaller. But I could start then reading off those indices around the eye. And therefore, if I wanted to just focus on the eye, or focus on the mouth, maybe I would have better luck if I tried to get the math. Okay, let's see if we can kind of just focus in on the mouth. It's kind of the same type of thing. It's the same type of thing that we're having happen. And what we're having happen is is that it's not like they went around the eye with one, two, three, four, five, six, seven, eight, nine, ten. It's like this is landmark one, two, three, four. They're kind of in random order. And so, what we would have to do is if we wanted to study the eye, the eyebrow, the mouth, we would have to kind of enlarge this and go over it a little bit more step by step. Maybe what would be good would be to have a very large face, put all the numbers on there, and then do a screen capture and print it out. Because you can see like this, it is just too many numbers, and it is not allowing us to just go in and do them one at a time. But we have the class that we need in order to put the mesh on there, to put the face mesh on there, if we want to draw it, or to give us access to the individual parts of the face if we want those. So, we have the things that we need in order to move forward.
So, let's take a second here and think about where we are. We now have a core Media program, a core Media program where we can kind of forget now those strange data structures that MediaPipe uses. And we're going to be just doing simple arrays. We'll have an array of the hand landmarks. We'll have an array of the face positions. We'll have an array of the body pose landmarks. And then we'll have an array of the facial mesh. And so, we've got all the data points in arrays that are very easy for us to work with.
Now, what I think would be interesting for us to go next with this would be to start trying to do things. We know Python can do lots of things. Python is going to be able to control Arduino. Python is going to be able to interact with the internet. Python is going to be able to do all these things interacting with a much broader scope of the world. But what we're going to do is we're going to see if we can make hand gestures and then have, have OpenCV understand what those hand gestures are as it analyzes the data, the landmarks that are coming back from MediaPipe. And then we're going to do things based on the hand gestures that we are able to analyze, are able to detect. Okay, that's kind of where we're going with this.
I hope you guys are having as much fun taking these lessons as I am making them. This was a little bit of a tedious lesson today, a little bit harder, but it was bringing together a lot of the stuff that we've, uh, that we've, uh, learned up until this point, bringing it all together into one program. And then I'll post this program on TopTechBoy.com. So, in case we need to use them in the future, we'll have kind of a, a common starting point to go with. If you guys like this video, make sure to give us a thumbs up. If you haven't already subscribed to the channel, when you subscribe, make sure you ring that bell so you'll get notifications when future lessons come out. And then share this with our other people. Try to introduce coding to other people because the world needs more people doing engineering and coding, and fewer people sitting around watching cat, silly cat videos. Paul McQuarter with TopTechBoy.com. I will talk to you guys later. [Music]