Transcription
Welcome back. So, in the last lecture, we showed that for relatively large n and moderate probabilities, the binomial distribution starts to converge to the normal distribution. Um, remember that the binomial distribution is, you know, like how you would count successes in a bunch of independent trials. If I flip, you know, 100 coins, what's the probability, um, you know, of getting a certain number of heads? The number of heads in 100 coin tosses is a binomially distributed variable, and when n is large, that starts to converge to the normal distribution.
And just to remind you, the normal distribution is the standard kind of uh Gaussian that we're used to seeing. So this is going to converge to a Gaussian with mean μ. Okay, so this is uh kind of just a normal Gaussian, normal distribution with mean μ and this standard deviation, this uh σ, is essentially going to measure the spread of this Gaussian. So if I have um a large σ, I'm going to have a more spread-out PDF. So this is, you know, large σ, and this one is a small σ. Okay, so that tells you kind of how broad the distribution is.
Okay, so, um, you know, if you look at the distribution of uh SAT scores going into Harvard, well, that's first off, it's not normal, but it also probably has a pretty narrow standard deviation. Okay, actually, I'm not sure they take SAT scores anymore. Um, but a good example of a normal distribution would be something like the height um of Americans, um, or you know, the number of heads given 100 coin flips, or you know, something like that. And you can have a large variance or a small variance. The standard deviation squared, σ², is often called uh the variance, or Var(X). Okay, and big variance means a lot of spread; small variance, small standard deviation, means you have a very narrow, peak distribution, which means you're very likely to be pretty close to that mean value. So if everything is super close to the mean, you have a small standard deviation.
Okay, so I mentioned last time that it's easier to compute uh basic probabilities down here in this normal distribution. Down here, I'm going to take a little tiny aside, and I'm just going to give you some properties and facts of the normal distribution, and then we're going to talk about how to calculate some basic probabilities that you might want to compute uh for these distributions, but using this normal distribution. Okay, um, I'm going to kind of show you the old-fashioned way of, like, you know, how I learned it, and then I'll tell you how to do this with a modern computer in Python.
Okay, so this normal distribution is parameterized by two things: there is the mean μ and the standard deviation uh squared, σ². Here, and what I want to do is I want to show you what this normal distribution looks like in the kind of standard unit normal. So uh we literally call this the standard um unit normal. Sometimes we call this, you know, the the Gaussian distribution, the normal distribution, and this is where μ = 0 and σ = 1. And this is kind of the canonical Gaussian. When you think of a Gaussian, this is like the normalized standard uh normal distribution or Gaussian distribution where everything is centered around uh x = z, and the spread is exactly equal to one. The amount of spread has been normalized, so my standard deviation is equal to one. And so the way I would draw this, um, again we're looking at probability density functions, the probability of X being at some value, um, so that probability is going to look uh like this. I'm just going to draw some things and tell you some facts. My bell curve or my my probability distribution is going to look kind of like this, and it's symmetric. This is a symmetric function, symmetric about the mean, uh, and this is—I'm going to give this thing some units—this is my probability, and this is my variable uh X down here. This is called the PDF, the probability density function, the probability. And if you really want to be careful, what we would actually do is we would actually um take this distribution and we would look at the probability of being at X plus some little ΔX or dx, and you would have this—this is a continuous distribution, so there's like an infinitesimal probability—and so to compute the probability between um, you know, two values, the probability that X is between -1 and 1, for example, you would have to integrate this probability density function from -1 to 1.
Okay, and in fact, I'm actually just going to write that down. So uh the probability that uh X is between -1 and 1 is equal to the integral uh of my my probability function—sometimes we actually call this uh lowercase f because this is a continuous probability density function—um, so sometimes we would call it little f, and it's the integral from -1 to 1 of f(x) dx. So literally, if I took this function here, this f(x), and I integrated it from -1 to 1, I would get the probability that X is between -1 and 1. Okay, um, the integral of this probability density function is often called the cumulative density function. So PDF is probability density function; CDF is the cumulative density function, and it's literally just the integral of f(x) dx from negative infinity to X. Okay, that's what this uh value here is. This is—sometimes we call it capital F, sometimes we call it um big Φ—for the standard unit normal, it's actually called big Φ. This cumulative distribution and it starts off at at zero and it eventually goes up to one because the cumulative—the probability—um, you know, way out here at x = negative infinity is zero, and the probability of me being less than X increases as I go to the right, and to the right, the probability—this is literally the probability that big X is less than or equal to little x—that's this cumulative distribution function, and it will eventually—I'm going to try to draw this as well as I can—it'll eventually converge to one. Okay, and it hits its middle point exactly when my Gaussian has its peak—its peak—probability. So this cumulative distribution function tells me what's the probability that X is less than a certain number, and it's adding up all of the probabilities that x equals this number, this number, this number, this number, this number up to that uh that point there.
Okay, um, and for this standard unit normal, we give this thing a name; we call it Φ. And so, for example, this uh probability that X is greater than -1 and less than 1 is this integral from uh -1 to 1, and this is actually defined uh as Φ, the cumulative distribution of one minus the cumulative distribution—the CDF—of -1. And in the olden days, in the stone ages when I was learning statistics, you would have to go to your clay tablets and look up these values. Someone chipped them in uh in cuneiform, and you had to go look up these numbers in a tablet, and it was only tabulation; it was only written down and recorded for this standard unit normal distribution. So if I had a normal distribution with a different mean and a different standard deviation, I would have to kind of warp or distort it into the standard unit normal, then I could use my lookup tables—again, clay tablets—and then I could retranslate those back to probabilities over here. Now this is like trivial in computers. You can get the CDF of this normal distribution that is not mean zero, standard deviation one; it can have any mean and any standard deviation, and the CDF, the cumulative distribution of this, that's just a, you know, one-line Python command. So in, like, SciPy.stats, you have all of the tools to compute these cumulative distribution functions, and so you can compute the probability that X is between a and b. And I'm just going to write that out, like the probability for a generic uh normal, the probability that, you know, X is between A and B, again, it's just the integral from A to B of my probability density function, which, you know, for a generic μ and σ is this function here. Really, if I was being careful, I would say that this is, you know, uh how do I actually want to write this? So this is the probability—this f(x)—this is the probability that x equals little x; this is the infinitesimal probability that f—the x—that my random variable equals some value little x. Okay, good.
Um, and you can actually compute this probability. This one is relatively easy um to compute using these standard normals. I think it's something like 68%; um, this is about a 68% chance that if you randomly drew a normal variable, you would be between -1 and 1 standard deviation. So that's kind of what standard deviation means is, you know, if you—you're in the middle of the distribution; if you're in the middle 68% of the distribution, you're within one standard deviation. You could compute what the probability of being within two standard deviations are; you would just replace this one by a two, and uh that's actually a good homework problem for you: is what are the chances you're within, you know, two standard deviations uh of the mean in a normal distribution? And this kind of answers the age-old question, "Am I normal?" Uh, you can compute—if you're normal, you can compute your height relative to the distribution; you can compute your weight relative to the distribution, or lots of other things, um, you know, to see if you're normal.
Okay, um, I think that if you want to do a shorthand of this, um, I'll just tell you that Φ(1) is about 0.841, something like that. This is again from those clay lookup tables, and that would be the probability of being to the left of one. So that would be—that would be um all Φ(1), the cumulative distribution is the integral from negative infinity to 1 of my PDF, so that's the probability of being left of x = 1, of being less than one. This is the probability that big X is less than or equal to one; that's this thing here. And by symmetry, you can argue that this remainder over here is about 16%, which means that this symmetric part over here that we don't want to be counting is also about 16%, and if you subtract that, you get your 68%. So work through this, kind of figure this out, um, try this out yourself, try this in Python, and compute these things. It's uh relatively straightforward, but what I want to do is I want to use this to compute something more tricky. I want to show you how you can actually use this to compute some really, really nasty probabilities that would be hard to do in binomial using the normal distribution.
So let's say that I uh I flip a coin uh 400 times. Okay, and the probability P um is equal to 1/2, you know, so uh P = 1/2. The number of heads that I expect to get, so number of heads is—we're going to call that our random variable X—and that is binomially distributed with n = 400 and P = 1/2. So I could compute the probability of X equaling 15, the probability of getting 15 heads; I would take this distribution and I'd plug in k = 15; I can get that probability no problem, but it's actually a total mess to compute something like the following. Let's say I want to compute the probability that X is—that I get at least 190 heads, so I get at least 190 heads and no more than 230 heads. This is a monster to compute using the binomial distribution. So what you would have to do is you would have to actually add up all of these cases: the probability that that I get between 190 and 230 heads is the probability that x equals exactly 190 plus the probability that x equals exactly 191 plus ... plus the probability that x equals exactly 229 plus the probability that x equals exactly 230, and I'd have to add up all 51 of these probabilities, which each is kind of a nightmare to compute. So I'd be adding up all of these terms; it would be a big mess. Yes, on a computer I could do it, but it's still just like messy and ugly.
Okay, so instead, what we do is we use the normal distribution. We say, well, okay, we know what the mean is here, so X is very approximately normal with mean 200. I expect 200 heads with mean 200 and standard deviation squared of 400 * 1/2 * 1/2 um which is 100. So this means that uh this is σ², so σ = 10 and μ = 200. And so I can, you know, draw this normal distribution; it's going to have a mean of 200; it's going to have a pretty narrow spread. I'm actually going to expect to get, you know, plus or minus 10 uh heads within about 68% of the time, um, so that gives you some some rough estimate there. And what I would do now is I can compute the the cumulative distribution, the integral; I can use the SciPy.stats uh cumulative distribution function, or I can warp this into standard normal and use these Φ functions. I can go find my clay tablets and use my lookups, but basically what I'm going to do is find the area of this PDF between 190 and 230, and I can compute what that area is. And if I use my, you know, clay lookup tables or if I go to Python and I uh use the cumulative distribution function in SciPy.stats, you're going to find that there's something like an 85 uh % chance that you are between 190 and 230 uh heads for this coin flip.
Okay, um, that was a lot, but I—I hope what I convinced you of is that the normal distribution is interesting, and there's lots of things you can compute about it. This cumulative distribution tells you the probability um that X is less than some some value. Um, you can derive things pretty quickly, like the—you expect to be within one standard deviation of the mean 68% of the time, and you can use that to compute things like the number of heads being between 190 and 230 out of 400 coin flips. It's way easier to compute these two numbers, you know, Φ(230) and Φ(190), than it is to add up all of these exact binomial probabilities. This is a mess; this is way, way, way, way easier to compute, and you're going to get a lot of intuition for what it means to be within one standard deviation or two standard deviations um pretty quickly; you'll build a lot of intuition, and it's a lot easier to work in these uh kind of Gaussian, normal distribution coordinates. Okay, thank you.