Transcription
Welcome back. So we showed in the last couple of lectures how to define functions of a random variable X. So, for example, if X is a normally distributed function, we can build functions Y of X like this: linear, uh, translation and scaling. Okay? And in general, this is a pretty robust strategy. This is a really, really simple function of X; it's just a linear function. And we showed how to derive the PDF and CDF, the probability density and cumulative density of Y, given that we know the PDF and CDF of X. But today, I want to do a more sophisticated example that actually comes up all over the place in statistics. One of the most useful distributions around is the distribution of X squared, if X is a normally distributed random variable. Okay, so that's what we're going to do today. Um, and I'm actually going to—yes, I'm going to say that X is just a simple standard unit normal Gaussian distribution: mean zero, standard deviation one. And what we're going to do is we're going to introduce this new random variable Y, and we're going to say that Y equals X². Okay? And I'm just going to tell you—well, it's not the punchline—the name of this is called the Chi-Squared Distribution. So Y follows what's called the Chi-Squared Distribution. And if you've done any statistics in the past, you've almost certainly come across this Chi-squared distribution. It's the most useful distribution for hypothesis testing, or one of the most useful. If I have data, if I collect data and I think it belongs to some distribution, I can essentially test that hypothesis using this Chi-squared distribution. Roughly speaking, just very, very broad brush strokes: if I take my collected data and my putative model—the model I think the data follows—and I subtract them and square that error, those errors should approximately follow a normal distribution based on the central limit theorem. And if I add up the sum of the squares of those errors, that should follow something like a Chi-squared distribution. Okay, so Chi-squared is super, super useful for hypothesis testing, even if the distribution I think my data follows isn't normal, even if it's a different distribution; I can still often times use Chi-squared. Okay, so that's enough preamble. Now I'm just going to show you how to actually compute the PDF of this Chi-squared distribution.
Now, remember, if I have the probability density function of X, let's say this unit normal, I can't just take that function and square it. I can't just take this unit normal function, you know, e to the minus x²/2σ², and square it; that's not a well-defined PDF. So I have to go through this slightly more cumbersome procedure. I have to start with the—I have to write down what is the cumulative density function of this new variable Y. I have to relate it to the cumulative distribution of X that I have, and then I have to take its derivative to get the probability density function with respect to Y. Okay, I'm just going to show you how this works. So, um, and I'll stick with pink for a little while. So the cumulative density function of this new Chi-squared variable is f sub y of little y. This is the probability that my new random variable Y is less than some little, some number, some specific value y. Okay? Um, this is a function of a variable little Y, and this is the probability that my random variable happens to be less than little y, and that is—essentially—okay, how do I want to write this? If, if big—if my new variable Y is less than little y, that means X² is less than little y. I'm going to go—I'm actually going to write out all my steps; I don't want to skip steps here and get confused. So this is the probability that X² is less than little Y, which is the same as the probability that X is greater than negative square root of Y and less than the positive square root of Y. Okay? So the probability of X being less than Y, it means that X has to either be less than √Y or greater than −√Y; it has to be between plus or minus √Y. Okay, because X is squared. Good. Um, and this thing, because X was a standard unit normal, I can actually write down the answer. Remember, we have those—the cumulative distribution function of this—we have defined Fₓ for a standard unit normal as this Φ function. It has a special name because in the olden days you would actually have to look this up from a lookup table, so it was a named function—this error function or this sigmoidal function, which is the cumulative distribution of a standard unit normal, so it has its own name, Φ. And so this probability is Φ of √Y minus Φ of −√Y. Okay? That is the cumulative distribution function; it's this function that I know, and all I do is I plug in √Y and −√Y, and I get this cumulative distribution function here. But I don't just want the CDF; um, what I really want is I want the probability density function. So here it's a Gaussian normal; I want to know the probability density function of Y here. So what I'm going to have to do is to get the PDF; I'm going to have to take the derivative of this expression, of this expression, with respect to Y. Okay? And we're going to use the chain rule, just like we would normally do. And I'm going to go back to pink here. So the PDF of my variable Y, um, essentially Fᵧ(Y) is just equal to the derivative with respect to Y of my cumulative density function of Fᵧ(Y). These functions here, so it's the derivative of this with respect to its independent variable. So this is going to equal—f, sorry, F′(√Y) times the derivative of √Y with respect to Y, that's Y⁻¹⁄². Okay, times ½Y⁻¹⁄², and I have two copies; I have this one, uh, and I have this one. So, and then minus the same thing. Um, good. And how do I want to do that? So, so that's—this guy here, plus another—I'm going to get minus and minus, so plus another Φ′(−√Y) * ½Y⁻¹⁄². Good. Um, and I feel like there is some kind of a symmetry trick happening here because in my notes, uh, these two terms combine to equal Y⁻¹⁄²Φ(√Y). Okay, so, um, I think actually this is something you'll need to figure out is—Φ′(√Y). So, um, this Φ′ here, I think we're going to have to figure out why—why I can combine these two quantities here. Um, I think that's not entirely obvious, um, because this is a −√Y and this is a +√Y. So I want you to actually think through why that is the case. And I'm guessing it's actually because I'm taking the derivative of these things, and so the slope at plus and minus √Y are equal and opposite, and so it seems like I probably just missed a sign somewhere. I think this should be a minus here, and then—and then this all works out. Okay, the details are important, but you know that's not the main point here. The main point was for you to see the procedure here. We have this new random variable X², so X is normal, and we want to introduce what is the PDF of X². So we start with the cumulative density function; we relate that cumulative density function of our new variable to the cumulative density function of our old variable; we know this big Φ function; and then we take its derivative to get an expression like this down here. And remember, the derivative of this big Φ function—this is my big F function; it's my CDF of X—the derivative is just my PDF of X. So this equals Y⁻¹⁄² times my PDF of my X, PDF evaluated at √Y. So literally, I would take my normal distribution, my Gaussian, and I would plug in √Y every time I see an X, and I'd multiply it by Y⁻¹⁄². And so you can actually write this down; you would say that this is fᵧ of little Y is equal to Y⁻¹⁄² * e⁻ʸ⁄² / √2π. Okay? Um, and I'm skipping a step here. Remember that the Fₓ, the normal distribution for a standard unit normal, is just 1/√2π * e⁻ˣ²/². Okay? So this is the PDF that we know; this is, you know, F′, the derivative of my CDF; this is the PDF that I know for my standard unit normal. And we went through all of this math and differentiated our CDF for our new variable to get its PDF in terms of this PDF that I know. And so now what I do is I plug in √Y—this should be an X here—I plug in √Y every time I see an X, so—I evaluate my PDF of my X variable, but I now plug in √Y, and I multiply by Y⁻¹⁄², and I get this new PDF of my Y variable. This is the probability density function of X² when X is a normally distributed random variable. This is called the Chi-squared distribution, and it's super, super useful for hypothesis testing in statistics, for testing if your data matches some distribution that you think it should be matching. Okay?
Um, tiny, tiny, tiny last recap: you can't just take your PDF for X and square it. This is not just this PDF squared; that doesn't work; that would not be a well-defined probability density. So instead, what you do is you define the cumulative density function of your new variable; you represent it in terms of your old—your old random variable—to get the cumulative density function of Y in terms of functions you know, like the cumulative density of X; then you take its derivative to get the probability density function of this new variable Y; you take its derivative, derivative, derivative—maybe there's some steps in the middle here that are a little hairy—but you take its derivative, and now you have the PDF of Y in terms of functions you already know, the PDF of X. And so if you plug all this into here, you get the Chi-squared distribution, the Chi-squared distribution from statistics—super useful; we're going to use this a bunch in later lectures when we actually start hypothesis testing and statistics—all because we now know how to build functions of a random variable. Okay, thank you.