Transcription
The normal distribution is arguably the most important concept in statistics. Everything we do, or almost everything we do, in inferential statistics—which is essentially making inferences based on data points—is to some degree based on the normal distribution.
So, what I want to do in this video, in this spreadsheet, is to essentially give you as deep an understanding of the normal distribution as possible. For the rest of your life, if someone says, “We’re assuming a normal distribution,” you can say, “Oh, I know what that is. This is a formula, and I understand how to use it,” et cetera.
So this spreadsheet, just so you know, is downloadable at www.khanacademy.org/downlads/, and if you just type that part in, you’ll see everything that’s downloadable. Then download/normalintro.xls, and you’ll get this spreadsheet right here. I think I did this in the right standard. Anyway, if you go onto Wikipedia and if you were to type in “normal distribution” or were to do a search for a normal distribution—let me actually get my pen tool going—this is what you would see. I literally copied and pasted this right here from Wikipedia. I know it looks daunting; you have all these Greek letters there, but this is just—this sigma right here—that is just the standard deviation of the distribution. We’ll play with that a little bit in this chart and see what that means. You know what the standard deviation means in general, but this is the standard deviation of this distribution, which is a probability density function.
And I encourage you to rewatch the video on probability density functions because it’s a little bit of a transition going from the binomial distribution, which is discrete. The binomial distribution will say, “What is the probability of getting a 5?” and you just kind of look at that histogram or that bar chart and say, “Oh, that’s the probability.” But in a continuous probability distribution or a continuous probability density function, you can’t just say, “What is the probability of me getting a 5?” You have to say, “What is the probability of me getting between, let’s say, a 4.5 and a 5.5?” You have to give it some range. And then your probability isn’t given by just reading this graph; the probability is given by the area under that curve. It’s given by this area.
For those of you who know calculus, if p(x) is our probability density function—doesn’t have to be a normal distribution, although it often is a normal distribution—the way you actually figure out the probability, let’s say between 4 and a half and 5 and a half: “What is the probability, the odds of me getting between 4 and a half and 5 and a half inches of rain tomorrow?” It’ll actually be the integral from 4 and a half to 5 and a half of this probability density function, or of this probability density function, dx. So that’s just the area of the curve. For those of you who don’t know calculus yet, I encourage you to watch that playlist. But all this is saying is the area of the curve from here to here.
It turns out for the normal distribution, this isn’t an easy thing to evaluate analytically, and so you do it numerically. You don’t have to feel bad about doing it numerically because, “Oh, how do I take the integral of this?” There’s actually functions for it, and you can even approximate it. One way you can approximate it is you could use the way you approximate integrals in general. You could say, “Well, what is the area of this? Well, it’s roughly the area of this trapezoid.” So you could figure out the area of that trapezoid, taking the average of that point and that point and multiplying it by the base. Or you can just take the level of—let me change colors because I think I’m overdoing it with the green—or you could just take the height of this line right here and multiply it by the base. You’ll get the area of this rectangle, which might be a pretty good approximation for the area under the curve, right? Because you’ll have a little bit extra over here, but you’re going to miss a little bit over there, so it might be a pretty good approximation. That’s actually what I do in the other video, just to approximate the area under the curve and give you a good sense that the normal distribution is what the binomial distribution becomes, essentially, if you have many trials.
What’s interesting about the normal distribution—I don’t know if I mentioned this already—this right here, this is the graph. This is just another word; people might talk about the central limit theorem. But this is really kind of one of the most important or interesting things about our universe, the central limit theorem. I won’t prove it here, but it essentially tells us—and you could kind of understand it by looking at the other video where we talk about flipping coins—if we were to do many flips of coins—those are independent trials of each other—and if you take the sum of all of your flips, if you were to give yourself one point if you got ahead every time, and if you take the sum of them as you approach an infinite number of flips, you approach the normal distribution. What’s interesting about that is each of those trials—in the case of flipping the coin, each trial is a flip of the coin—each of those trials doesn’t have to have a normal distribution. So we could be talking about molecular interactions, and every time compound x interacts with compound y, what might it result in doesn’t have to be normally distributed. But what happens is, if you take a sum of a ton of those interactions, then all of a sudden the end result will be normally distributed. This is why this is such an important distribution. It shows up in nature all of the time, and if you do take data points from something that is very complex and it is the sum of arguably many, almost infinite, individual independent trials, it’s a pretty good assumption to assume the normal distribution. We’ll do other videos where we talk about when it is a good assumption, when it isn’t a good assumption.
But anyway, just to digest this a little bit, and let me actually rewrite it. This is what you’ll see on Wikipedia, but this could be rewritten as 1 over sigma times the square root of 2π times—x is just e to that power—so it’s just e to the, this whole thing over here, minus (x − the mean)² over 2σ². This is the standard deviation. Standard deviation squared is just the variance. Just so you know how to use this, you’re like, “Oh wow, there’s so many Greek letters here, what do I do?” This tells you the height of the normal distribution function. Let’s say that this is the distribution of people’s heights above 5’9”. Let’s say that this was 5’9” and not 0. What this tells you is, if you wanted to figure out what is the probability of finding someone who was roughly 5 inches taller than the average right here, what you would do is you would put in this number here, this 5 into x. And then you know the standard deviation because you’ve taken a bunch of samples. You know the variance, which is the standard deviation squared. You know the mean, and you just put your x in there, and it’ll tell you the height of the function. And then you have to give it a range. You can’t just say how many people are exactly 5 inches taller than average. You would actually say how many people are between 5.1 inches and 4.9 inches taller than the average. You have to give it a little bit of range because no one is exactly—it’s almost infinitely impossible to the atom to be exactly 5’9”. Even the definition of an inch isn’t defined that particularly. So that’s how you’d use this function. This is so heavily used in—one, it shows up in nature—but in all of inferential statistics, I think it behooves you to become as familiar with this formula as possible. I guess to make that happen…
Let me play around a little bit with this formula just to kind of give you an intuition of how everything works out, et cetera. If I were to take this—I’d like to maybe help you memorize this—this could be rewritten as, if we take the sigma into the square root sign, if we take the standard deviation in there, it becomes 1 over the square root of 2πσ². I’ve never seen it written this way, but it gives me a little intuition that σ², it’s always written as σ², but it’s really just the variance, and the variance is what you calculate before you calculate the standard deviation, so that’s interesting. And then this top right here, this could be written as e to the −1/2 times—and both of these things here are squared—so we could just say (x − the mean)/σ². And this kind of clarifies what’s going on here a little bit better. Because what’s this? x − σ is the distance between whatever point we want to find. Let’s say we’re here. x − μ is the mean, so that’s here, so that’s this distance, and this is the standard deviation, which is this distance. So this in here tells me how many standard deviations I am away from the mean. That’s actually called the standard z-score; I talked about it in the other video. And then we square that, and then we take this to the −1/2. Well, let me rewrite that. If I were to write e to the −1/2 times a, that’s the same thing as eᵃ to the −1/2 power, right? If you take something to an exponent and take that to an exponent, you can just multiply these exponents. So likewise, this could be rewritten as—this is equal to 1 over the square root of 2πσ², which is just the variance. And I’m just playing around with the formula because I really want you to see all the ways—maybe you’ll get a little intuition. I encourage you to email me if you see some insight on why this exists. Once again, I think it is cool then all of a sudden we have this other formula that has π and e in it. So many phenomena are described by this, and once again π and e show up together, just like e^(iπ) = −1. It tells you something about our universe. Anyway, I could rewrite this as e^[(x − μ)/σ]² and all of that to the −1/2. Something in the −1/2 power, that’s just 1 over the square root, which is already going on here. So we could just rewrite this over here as 1 over the square root of 2π times the variance times e to essentially our z-score squared. If we say z is this thing in here, z is how many standard deviations we are from the mean, z-score squared. And all of a sudden this becomes very clean. We just say 2π times our variance times e to the number of standard deviations we are away from the mean, and you square that. You take the square root of that thing and invert it, and that’s the normal distribution.
Anyway, I wanted to do that. I thought it was neat, and it’s interesting to play around with it. Then that way, if you see it in any of these other forms in the rest of your life, you won’t say, “What’s that? I thought the normal distribution was this, or it was this,” and now you know. With that said, let’s play around a little bit with this normal distribution.
So this spreadsheet, I’ve plotted the normal distribution. You can change the assumptions that are in this kind of green, blue color. So right now it’s plotting it with a mean of zero and a standard deviation of 4. I just write the variance here just for your information; the variance is just the standard deviation squared. And so what happens when you change the mean? So if the mean goes from 0 to let’s say it goes to 5. Notice, this graph just shifted to the right by 5. Right? It was centered here; now it’s centered over here. If we make it −5, what happens? The whole bell curve just shifts 5 to the left from the center. Now what happens when you change the standard deviation? The variance is the average squared distance from the mean; the standard deviation is the square root of that. So it’s kind of—not exactly—but kind of the average distance from the mean. So the smaller the standard deviation, the closer a lot of the points are going to be to the mean. We should get kind of a narrower graph, and let’s see if that happens. When the standard deviation is 2, we see that. The graph—you’re more likely to be really close to the mean than further away. If you make the standard deviation—if you make it 10—all of a sudden you got a really flat graph, and this thing keeps going on forever. And that’s a key difference: the binomial distribution is always finite. You can only have a finite number of values, while the normal distribution is defined over the entire real number line. So the probability, if you have a mean of −5 and a standard deviation of 10, the probability of getting a thousand here is very low, but there’s some probability. There’s some probability that all of the atoms in my body just arrange perfectly that I fall through the seat I’m sitting on. It’s very unlikely, and it probably won’t happen in the life of the universe, but it can happen. And that can be described by the normal distribution because it says anything can happen, although it could be very improbable.
The thing I talked about at the beginning of the video is when you figure out a normal distribution, you can’t just look at this point on the graph—let me get the drop pen tool back—you have to figure out the area under the curve between two points. So if I wanted to say—let’s say this is our distribution—and I said, “What is the probability that I get 0?” I don’t know what phenomena this is describing, but at 0 happened. If I say exactly 0, the probability is 0—I shouldn’t use 0 too much—because the area under the curve, just under 0, there’s no area; it’s just a line. You have to say between a range. So you have to say the probability between—and actually I can type it in here—between −0.005 and +0.05 is—well, it rounded—it says they’re close to 0. Let me do it, between −1 and between 1. It calculated at 7 percent, and I’ll show you how I calculated this in a second. So let me get the screen draw tool. So what did I just do? Between −1 and 1—and I’ll show you the behind the scenes, what Excel is doing—we’re going from −1, which is roughly right here, to 1. And we’re calculating the area under the curve. We’re calculating this area, or for those of you who know calculus, we’re calculating the integral from −1 to 1 of this function where the standard deviation is right here, is 10, and the mean is −5. Actually, let me put that in. So we’re calculating, for this example, the way it’s drawn right here, the normal distribution function, our standard deviation is 10 times the square root of 2π times e to the −1/2 times (x − our mean). Our mean is negative right now; our mean is −5, so it’s (x + 5) over the standard deviation squared, which is the variance, so that’s 100, dx. This is what this number is right here, this 7 percent, or actually 0.07, is the area right around there.
Now, unfortunately for us in the world, this isn’t an easy integral to evaluate analytically, even for those of us who know our calculus. So this tends to be done numerically. And kind of an easy way to do this is—well, not as easy a way—a function has been defined called the cumulative distribution function that is a useful tool for figuring out this area. So what the cumulative distribution function is essentially—let me call it the cumulative distribution function—it’s a function of x. It gives us the area under the curve, under this curve. So let’s say that this is x right here, that’s our x. It tells you the area under the curve up to x. So another way to think about it, it tells you what is the probability that you land at some value less than your x value. So it’s the area from −∞ to x of our probability density function. When you actually use the Excel normal distribution function, let me say NORM.DIST, you have to give it your x value, you give it the mean, you give it the standard deviation. And then you say whether you want the cumulative distribution, in which case you say TRUE, or you want just this normal distribution, which you say FALSE. So if you wanted to graph this right here, you would say FALSE in caps. If you wanted to graph the cumulative distribution function, which I do down here—let me move this down a little bit, let me get out of the pen tool—then you say TRUE when you make that Excel call. So this is a cumulative distribution function for this. This is a normal distribution; here’s a cumulative distribution. Just so you get the intuition. If you want to know, “What is the probability that I get a value less than 20?” So I can get any value less than 20 given this distribution. The cumulative distribution right here—let me make it so you can see—if you go to 20, you just go right to that point there and you say, “Wow, the probability of getting 20 or less is pretty high. It’s approaching 100 percent.” That makes sense because most of the area under this curve is less than 20. Or if you said, “What’s the probability of getting less than −5?” Well, −5 is the mean, so half of your results should be above that, and half should be below. And if you go to this point right here, you could see that this right here is 50 percent. So the probability of getting less than −5 is exactly 50 percent. So what you do is, if I wanted to know the probability of getting between −1 and 1, what I do is—let me get back to my pen tool—I figure out what is the probability of getting −1 or lower. So I figure out this whole area. And then I figure out the probability of getting 1 or lower, which is this whole area—well, I’m going to give it a different color—1 or lower is everything there. And I subtract the yellow area from the magenta area, and I’ll just get what’s ever left over here. That’s exactly what I did in the spreadsheet. Let me scroll down. This might be taxing my computer by taking the screen capture with it.
So what I did is I evaluated the cumulative distribution function at 1 to be right there. And I evaluate the cumulative distribution function at −1, which is right there. And the difference between these two—I subtract this number from this number—and that tells me essentially the probability that I’m between those two numbers. Or another way to think about it, the area right here. I really encourage you to play with this and explore the Excel formulas and everything. This area right here would be between −1 and 1. Just so you know this graph, the central line right here, this is the mean. And then these two lines I drew right here, these are 1 standard deviation below and 1 standard deviation above the mean. Some people think, “What’s the probability that I land within one standard deviation of the mean?” Well, that’s easy to do. What I can do is I’ll just click on this. And I’ll just call this—what’s the probability that I land between 1 standard deviation—the mean is −5—1 standard deviation below the mean is −15, and 1 standard deviation above the mean is 10 + (−5) is 5. So that’s between 5 and −15. So 68.3 percent. That’s actually always the case that you have a 68.3 percent probability of landing within one standard deviation of the mean, assuming you have a normal distribution. So once again, that number represents the area under the curve here, this area under the curve. And the way you get it is with the cumulative distribution function. I’ll go down here. Every time I move this, I have to get rid of the pen tool. You evaluate it at +5, which is right here. This is 1 standard deviation above the mean; that’s a number right around there. It looks like it’s like 80-something percent, maybe 90 percent roughly. And then you evaluate it at 1 standard deviation below the mean, which is −15. And this one looks like roughly 15 percent or so, 15, 16, maybe 17 percent, I’ll say 18 percent. But the big picture is when you subtract this value from this value, you get the probability that you land between those two. And that’s because this value tells the probability that you’re less. So when you go to the cumulative distribution function, you get that right there. It keeps crawling back and forth. So when you go to 5—and you just go right over here—this is essentially tells you this area under the curve, the probability that you’re less than or equal to 5, everything up there. And then when you evaluate it at −15 down here, it tells you the probability that you’re back here. So when you subtract this from the larger thing, you’re just left with what’s under the curve right there.
Just to understand this spreadsheet a little bit better because I really want you to play with it and see what happens if I make this distribution—the mean was −5; now let me make it 5. It just shifted to the right. It just moved over to the right by 5. Whoops. Let me use the pen tool. It just moved over to the right by 5. If I were to try to make the standard deviation smaller, we’ll see that the whole thing just gets a little bit tighter. Let’s make it 6. And all of a sudden this looks like a tighter curve; we make it 2, it becomes even tighter. I really want you to play with this and play with the formula and get an intuitive feeling for this, the cumulative distribution function, and think a lot about how it relates to the binomial distribution, and I cover that in the last video.
To plot this, I just took each of these points. I went to plot the points between −20 and 20, and I just incremented by 1. I just decided to increment by 1. So this isn’t a continuous curve; it’s actually just plotting a point at each point and connecting it with a line. Then I did the distance between each of those points and the mean. So I just took 0 − 5; this is this distance. So this just tells you the point −20 is 25 less than the mean. That’s all I did there. Then I divided that by the standard deviation, and this is the z-score, the standard z-score. This tells me how many standard deviations is −20 away from the mean. It’s 12 and a half standard deviations below the mean. Then I use that, and I just plugged it into essentially this formula to figure out the height of the function. Let’s say at −20 the height is very low. Let’s say at −2 the height’s a little bit better; the height’s going to be some place, it’s going to be right there. And so that gives me that value. But then to actually figure out the probability of that, what I do is I calculate the cumulative distribution function. Well, this is the probability that you’re less than that. So the area under the curve below that, which is very small. It’s not zero; I know it looks like 0 here, but that’s only because I rounded it. It’s going to be 0.0001; it’s going to be a really small number. There’s some probability that we even get −1000.
Another intuitive thing that you really should have a sense for is the integral over this, or the entire area of the curve, has to be 1 because that takes into account all possible circumstances. And that should happen if we put a suitably smaller number here and a suitably large number here. There you go; we get 100 percent. Although, this isn’t 100 percent. We’d have to go from −∞ to +∞ to really get 100 percent. It’s just rounding to 100 percent. It’s probably 99.99999 percent or something like that. And so to actually calculate this, what I do is I take the cumulative distribution function of this point and I subtract from that the cumulative distribution function of that point. And
That's where I got this 100 percent from.
Anyway, hopefully that will give you a good feel for the normal distribution. I really encourage you to play with the spreadsheet and to even make a spreadsheet like this yourself.
In a future exercise, we'll actually use this type of spreadsheet as an input into other models. If we're doing a financial model and if we say our revenue has a normal distribution around some expected value, what is the distribution of our net income? Or we could think of a hundred other different types of examples.
Anyway, see you in the next video.