Transcription
Hello guys, this is Paul McQuarter with TopTechBoy.com, coming to you today from the banks of the lovely River Nile in order to bring you episode number 26 in our incredible new tutorial series where you're going to learn artificial intelligence, or you're going to die trying.
What I'm going to need you to do is pour yourself a nice tall glass of ice-cold coffee. That would be straight up black coffee, poured over ice. No sugar, no sweeteners, none needed. And as you're pouring your coffee, as always, I want to give a shout-out to you guys who are helping me out over at Patreon. It is your support and your encouragement that keeps this great content coming. You guys that are not helping out yet, take a look down in the description. There is a link over to my Patreon account. Think about hopping on over there and hooking a brother up.
But enough of this shameless self-promotion, let's talk about what I am going to teach you today. And what we're going to do is we're going to look at the homework assignment that I gave you in episode number 25. And what that was was that we have developed a very good gesture recognition system, but the one limitation is is that if your hand is real close, or and your hand is far away, it doesn't work so well. It works when your hand is the same distance from the camera where you trained it. And if you trained it here, then you've got to keep it here. If you've trained it here, you've got to keep it here. And I asked you guys to go in and try to come up with something that would be a little more generic.
And so, let's hop over to our trusty sketchpad and let's kind of chat through it a little bit, and then I can maybe describe to you what my approach is going to be to solve this. I also need to kind of get out of your way here. Okay. Now, remember what we did was we identified, we identified certain things that we call key points. And the key points were the wrist, and then the tips of the fingers. The wrist and the tips of the fingers, as such. And then we also called key points the base of the fingers where they came out of the palms. And so those were our key points. And then what we did was we created a distance matrix that measured the distance between each of these key points. And not just between the wrist and each one, between each one and each other one. And then what we did was we took that difference-based matrix, or that distance matrix, for a trained, uh, for a trained gesture versus an unknown gesture. We create, we then calculated an error. And if the error was below a certain threshold, we said we had a match.
But now, this is the problem. Imagine that we trained very, let's say we were going to do the number one, and we trained very far away from the camera. Well, here we would have something like the wrist, and then we would have all these other digits, and then the folded digits, and then we would have the one sticking up here like this. But now imagine now that we tried to recognize it while it was closer. Well, while it was closer, all of those things are going to be relatively the same. Okay, they're going to be relatively the same. You see the, the shape is the same, but the problem is the distance between here and here is very different than the distance between here and here. And therefore, the algorithm kind of breaks down as your hand is getting bigger and smaller.
So, this is the way that I am going to approach this. I'm going to normalize all of the hands, whether they're close or far away. And what am I going to normalize it to? I'm going to normalize it to what I am going to call the palm size. Okay. And the palm size is going to be the distance between 0 and 9. So, the palm size, the palm size is going to be equal to the distance between point 0 and point 9. Okay. Then every other distance, every other distance between A and B, is going to be divided by the distance between A, between 0 and 9. And so that means you're not reporting a distance between, a distance between here and here is so many pixels, but what fraction of a palm size do you have? Does that make sense? And I think this might just work.
So, let's go ahead and let's give it a try. I'll need to do a little rearrangement here. You guys are going to see my, uh, my, uh, studio here, my workspace is getting a little bit more busy. I'm actually kind of trying to set up a Raspberry Pi over here to get it going. And so things are getting a little bit crowded. I'm trying to be really, really neat about it, but I am starting to get a little bit more crowded. But I digress.
What I'm going to need you guys to do is come over and fire up your most excellent Visual Studio Code. We're going to come over here and remember that we are working in the Python folder. I'm going to create a new Python program, which I am going to call opencv-43.py. And the .py is kind of important. And boom, a fresh new Python program, just waiting for us to write. But now, we don't want to write this program from scratch. We want to go in and kind of start where we left off last week. We want to start where we left off last week. And if you guys didn't save your program or have lost track of it, you can come to the most excellent www.toptechboy.com, search on AI for Everyone, lesson 26, and then you'll come up to this page. You can copy the code there, and then you can come back over here and we can paste the code. Okay, paste the code.
And now, just to make sure that the universe is in proper working order, let's go ahead and try to run this. Let's just do a quick one to make sure that it works. And so, a very quick one would be, uh, let's do six. I can't. I just got to do one. Okay, we're going to do gesture 1, gesture 2, gesture 3, gesture 4, gesture 5, and resist the metaverse. That's going to be a theme here. Resist the metaverse. I'll put an exclamation point on that one. Okay.
Now, what it wants to do is it is ready to train. And so what we need to do is we need to come up and train it. It wants gesture 1. So I'm going to say that's a 1. Okay, that's a 1. And then that's a 2. And then that's a 3. That's a 4. And then that's a 5. Okay, that looks good. Oh, once we resist the metaverse. Resist the metaverse. Okay, resist the metaverse. One, two, three, four, five. Resist the metaverse. One, two, two, one, five, or five, one, two, three, four, five. Resist the metaverse. Okay, that works good.
But what happens if I come back here? Everything is unknown until I get back to about that right distance. Everything works. And then as we get closer and a bigger hand, it doesn't work anymore. All right. So you can kind of see what the problem is. So what do we want to do? Well, what I want to do is when we're finding distances here, when we're finding distances, we pass this function, we pass this function the hand landmarks. Okay? So it's got the hand landmarks. And I want to find the distance between the, the wrist and this digit here. And we said that was between point 0 and point 9. Well, we can just grab this because this was that distance formula. This was that distance formula. I need to go slower so you guys can see what I, what I'm doing. I'm in the find distances function. And where I'm calculating the distance matrix, I want to k, I want to copy this equation that calculates distance because I don't want to type that whole thing in again. I'd probably make a mistake if I did that.
And then what I'm going to do here is right after I, right after I get into this function, I want to calculate palm size. And palm size is just going to be equal to this. But instead of rows and columns, instead of rows and columns, the row would be the point I'm coming from, which was the point 0, and the column is the point that I'm going to, which is 9. You know, I'm just going to hardwire those in. And then row was what? Row was 0. And then column was what? Column was 9. All right. Now that should calculate, that should calculate my palm size. Okay, that should calculate my palm size. Now, every time I calculate a distance, I don't want to save that distance. I want to divide that whole thing in because I want to divide that whole thing. I'm going to put another set of parentheses. And then here, we're calculating that distance. But I'm going to take that whole thing, that whole thing, I'm going to close the parentheses and I'm going to divide by what? Palm size. Okay.
Could it really be that easy? Could it really be that easy? Well, not quite. Because what's one thing that we're going to have to do? We're going to have to come in and remember when we put in our tolerance, like what we considered, what we would consider, uh, how can I say this, what we would consider a match or not? We set up that, uh, tolerance. And so we said, like, a, a total error of 1500 would be sort of the breakout between a match or not a match. But now all my numbers are divided by this distance. So this needs to be a lot less. Now, how much less does it need to be? Well, I think the way we would, we would do that is now we would go up to, we could just sit and guess. That's probably what most of you do is you just sit and guess, right? But I think what we could do is we could, uh, we're creating this error matrix here, find gesture. And then when we actually calculate an error, when we find the error, when we're going to return that error, let's just pause and print the error.
[Music]
Okay. And what I'm going to do is I'm going to do one gesture and I'm just going to kind of look at what the errors do as I give it that gesture or not give it that gesture. Does that make sense? I hope it does. Let's, let's give it the number one. Okay, how many? I'm gonna do one. All right. And then the name of gesture one is just one, like that. Okay. Now it wants gesture one. Okay. So I'm going to come up here and I'm going to give it the number one. Okay. And now I'm going to tell it T. And now the error looks like it's less than 1. And the error goes to 20. The error is like 2 and 20. 2 and 20. If I come up here, look at that. Now the error is still 20. It's about 5, 20. Okay. Three. If I come way back here, it looks like the error. I think a good cutoff would be 10 because 10 is way less. 10 is way less than 20. Okay. 10 is way less than 20, and it's way more than five. And so I think that if I, if I made this tolerance, the error that I can tolerate, if I made that 10, did I really do that right? We'll see. Am I thinking about that right? We'll see. Okay, making us wait.
Okay, I'm going to do 11 gestures. Okay. One, two, three, four, five, six. Now you can't do six. Okay, we'll do a high five. We will go forth and prosper. We will resist the metaverse. [Music] Resist the metaverse. Okay. And then we'll do guns up. We'll do an A-okay. And then I always get to the end and I always can't figure out which one I should do last. Maybe I should just do 10 instead. Oh, I remember. Hi. No, I did high five already. Five. High five. Go forth and prosper. Resist. Ah, forgot my almond modder. Hook 'em horns. Okay, there. Now it is going to want us to do those things. It's going to want us to give them those. Okay. And so I'm going to give it a real clear shot. It wants one, so that's one. Two, three, four, five. High five. Go forth and prosper. Resist the metaverse. Guns up. A-okay. Hook 'em horns. Okay, I've got a hook 'em horn. So let's just go. Resist the metaverse. One, two, three, four, five. Okay. Resist the metaverse. Forth and prosper. High five. Five. Uh, what was it? Uh, hook 'em horns. See, I forget all those things that I did. Ah, guns up. A-okay. Did I forget any? I think I did most of them.
Okay, now, but what's the real question? Look at that way back, huh? Resist the metaverse. One, two, three, four, five. High five. Go forth and prosper. Five. A-okay. Hook 'em horns. Guns up. Look at that. Now let's see if we can do it close. Resist the metaverse. One, two, three, four, five. Let's see. Five. One, two, three, four, five. Up close. Hook 'em horns. A-okay. Guns up. Okay. All right, Kazman, do you see that? Look at that. That is a huge step forward. Because now, if we're way back here, it'll work. Or if we're way up here, it'll work. And so that is just a big old, huge, enormous step forward. You guys let me know, were you able to do this on your own? Or maybe you came up with a different way of doing it? Maybe you had a better way of doing it? But, but let me know. Give me feedback if you guys were able to do this or not. I think this is just a really, really good solution. And at this point, we kind of have it.
Now, what you can be thinking about is if you have a camera sitting looking at you, knowing what your gestures are, what could you do with those gestures? So that's kind of the next thing for you guys to be thinking about is what is the application that you could use now that your computer knows what your hand signal is? What can you have your computer do based on those hand signals?
Okay guys, man, I hope you guys are having as much fun taking these lessons as I am making them. I am just really, really, really enjoying this series of lessons. And I think this was a really fun lesson because I think it really, really made a huge difference. And it's going to make this routine that we've developed much, much, much more, more useful. Okay? So you guys work and start thinking. Leave some comments down below about what some applications would be. If you enjoyed this video, make sure you give us a thumbs up because that will help us with the old Google juice, the YouTube juice. And YouTube will show this video to more people if you guys give me a thumbs up and leave a comment down below. Also, subscribe to the channel if you haven't already. When you subscribe, make sure you ring that bell so you'll get notifications when I release new classes. And then share this with other people because the world needs more people doing engineering and writing code and fewer people sitting around watching silly cat videos. Paul McQuarter, sitting on the banks of the Nile River. I will talk to you guys later.
[Music]