Transcription
Convolutional neural networks are used for image classification and other stuff. StatQuest. Hello, I'm Josh Starmer, and welcome to StatQuest. Today, we're going to talk about neural networks, part 8: Image classification with convolutional neural networks.
Note: This StatQuest assumes that you are already familiar with the main ideas behind neural networks, the main ideas behind back propagation, the main ideas behind the ReLU activation function, and you should know about neural networks with multiple inputs and outputs. If not, check out the quests. The links are in the description below.
Now, imagine we have a friend named Statsquatch, and Statsquatch wants to play tic-tac-toe against their computer. Unfortunately, Statsquatch can't remember if you start with the letter X or the letter O. So, each time Statsquatch plays tic-tac-toe against their computer, the computer has to figure out if Statsquatch drew the letter X or the letter O. The good news is that the computer can figure out if Statsquatch drew the letter X or O using a convolutional neural network. Bam!
Because the letter O and the letter X are drawn on a computer screen, we can zoom in on the letter O and the letter X and see that each image is just a bunch of pixels. And in this case, each pixel is represented by either a zero for a white pixel or a one for a black pixel. So, let's walk through step by step how a computer can classify this image as the letter O and this image as the letter X. We will start with the image of the letter O.
Now, because this image is so small, just 6 pixels by 6 pixels, it is possible to make a normal, everyday neural network that can correctly classify it. We simply convert this 6x6 grid of pixels into a single column of 36 input nodes and connect the input nodes to a hidden layer. So, here we have 36 connections from the 36 input nodes to this node in the hidden layer. Now, remember, each connection has a weight that we have to estimate with back propagation. So, that means we need to estimate 36 weights in order to connect to this node. However, usually, the first hidden layer has more than one node, and each additional node adds an additional 36 weights that we need to estimate. Like I said, because the original image is small, 6x6, and black and white, something like this could work.
However, if we had a larger image, like 100 pixels by 100 pixels, which is still pretty small compared to real-world pictures, then we would end up having to estimate 10,000 weights per node in the hidden layer. So, this method doesn't scale very well. Another problem is that it's not clear that this neural network will still perform well if the image is shifted by one pixel. For example, if this is the image we used for training, then it is not clear that the neural network will still recognize this letter O correctly if each pixel is shifted to the right by one. Lastly, even complicated images like this teddy bear tend to have correlated pixels. For example, any brownish pixel in this image tends to be close to other brown pixels, and any white pixel tends to be near other white pixels. And it might be helpful if we can take advantage of the correlation that exists among each pixel. Thus, classification of large and complicated images is usually done using something called a convolutional neural network.
Convolutional neural networks do three things to make image classification practical:
1. They reduce the number of input nodes.
2. They tolerate small shifts in where the pixels are in the image.
3. They take advantage of the correlations that we observe in complex images.
So, let's go back to our tic-tac-toe game and see how a convolutional neural network can recognize this letter O. The first thing a convolutional neural network does is apply a filter to the input image. In convolutional neural networks, a filter is just a smaller square that is commonly 3 pixels by 3 pixels, and the intensity of each pixel in the filter is determined by back propagation. In other words, before training a convolutional neural network, we start with random pixel values, and after training with back propagation, we end up with something more useful.
To apply the filter to the input image, we overlay the filter onto the image and then we multiply together each overlapping pixel and then we add each product together to get a final value, which in this case is three. Oh no, it's the dreaded terminology alert! In fancy math lingo, we call this sum of products a dot product. By computing the dot product between the input and the filter, we can say that the filter is convolved with the input, and that's what gives convolutional neural networks their name. Small. Bam!
Now, we add a bias term to the output of the filter and put the final value into something called a feature map. Now, in this example, we slide the filter over 1 pixel. However, other convolutional neural networks might move over two or more pixels, but in this example, we just move over one pixel and calculate the dot product of the filter and overlapping pixels in the image, add the bias term, and put the final value into the feature map. Then, we shift the filter over again and repeat until we have filled up the feature map. Bam! We filled up the whole feature map.
To summarize, we started with an input image of the letter O, and then applied a filter to it. In other words, we convolved the filter with the input and added a bias term to the values, and that gave us a feature map. Because each cell in the feature map corresponds to a group of neighboring pixels, the feature map helps take advantage of any correlations there might be in the image. Bam!
Now, typically, we run the feature map through a ReLU activation function, and that means that all of the negative values are set to zero, and the positive values are the same as before. In this case, that means everything gets set to zero except for these two points. So, let's move this stuff over to make some room for the next step.
Now, we apply another filter to the new feature map. However, unlike before, we simply select the maximum value, and this filter usually moves in such a way that it does not overlap itself. Oh no, it's another dreaded terminology alert! When we select the maximum value in each region, we are applying max pooling. To see the effect that max pooling has, in this example, let's go back to the input image. Here, we see that the upper left-hand corner in the input image has an exact match with the filter, and this results in the highest possible value in the feature map. And then the max pooling step reduces this region to the one spot where the filter did the best job matching the input image. In other words, this region in the input image corresponds to this part of the feature map, and the max pooling step selected the spot where the filter did the best job matching the input image. Likewise, this region in the input image corresponds to this part of the feature map, and the max pooling step selected the spot where the filter did the best job matching the input image. So, we see that max pooling selects the spots where the filter did the best job matching the input image. Oh no, it's another terminology alert! Alternatively, we could calculate the average value for each region, and that would be called average or mean pooling.
Now, going back to the max pooled layer, let's move and shrink things to give us more room. Now, let's convert the pooled layer into a column of input nodes. Lastly, let's plug the input nodes into a normal, everyday neural network. This neural network has four input nodes, a single hidden layer with a single node using the ReLU activation function, and two output nodes: one for the letter O and one for the letter X.
So, given this image, we run it through the filter to create the feature map. Then, we run the feature map through a ReLU activation function. Then, we select the maximum value in each area and end up with these values in the input nodes. Now, we multiply the values in the input nodes by their associated weights and add each term together, and then add the bias, and we end up with 0.34. And thus, the x-axis coordinate for the activation function is 0.34. Now, we plug 0.34 into the ReLU activation function, and the output is 0.34 because 0.34 is greater than 0. Now, this connection from the hidden layer to the output for the letter O gives us one for the letter O, and the connection from the hidden layer to the output for the letter X gives us zero for the letter X. So, when the input is a picture of the letter O, this convolutional neural network classifies it as a picture of the letter O. Bam!
Now, let's see what happens when the input is a picture of the letter X. Note, even though we have changed the input, the filter is the same as before, and we're doing max pooling just like before. In the neural network with its weights and biases is the same as before. So, just like before, we run the filter over the input to create the feature map. Now, we run the feature map through the ReLU and do max pooling, and the results of max pooling become the values for the input nodes. Now, we run the values in the input nodes through the neural network, and the result is zero for the letter O and one for the letter X. So, when the input is a picture of the letter X, this convolutional neural network classifies it as a picture of the letter X. Double bam!
Now, remember that we said convolutional neural networks help reduce the number of inputs in the neural network. In this case, we started with a 6x6 image, or 36 potential inputs, and compressed those down to just four inputs into the neural network. We also said that convolutional neural networks take correlations into account, and this is accomplished by the filter, which looks at a region of pixels instead of just one at a time. Lastly, we said that convolutional neural networks can tolerate small shifts in where the pixels are in the image. So, let's see what happens when we shift the picture of the letter X one pixel to the right. In other words, will this convolutional neural network still decide if the input is a picture of the letter X? And we see that the output value for the letter X, 1.23, is much closer to 1 than the output value for the letter O, negative 0.2. So, this convolutional neural network decided that the input image is of the letter X. Triple bam!
Note: If we wanted to, we can make the output easier to interpret by running it through the softmax function or the argmax function. Also, note this is about as simple a convolutional neural network as you can get. However, no matter how fancy the convolutional neural network is, it's still based on filters, aka convolution, applying an activation function to the filter output, and pooling the output of the activation function.
Now, the next time Statsquatch decides to play tic-tac-toe with their computer, Statsquatch can start with the letter X or the letter O, and a convolutional neural network will figure it out. Bam!
Now, it's time for some shameless self-promotion. If you want to review statistics and machine learning offline, check out the StatQuest study guides at statquest.org. There's something for everyone. Hooray!
We've made it to the end of another exciting StatQuest. If you like this StatQuest and want to see more, please subscribe. And if you want to support StatQuest, consider contributing to my Patreon campaign, becoming a channel member, buying one or two of my original songs or a t-shirt or a hoodie, or just donate. The links are in the description below. Alright, until next time, Quest on!