Transcription
Welcome back. Okay, so in the last lecture, we introduced this notion of co-variance and correlation, a really important property that allows me to tell kind of the joint dependence of two random variables X and Y. And I use this example of these two-dimensional Gaussian distributions, kind of joint probability distributions in X and Y, where, like you can imagine, you know, throwing a dart at a board, you get kind of a Gaussian pattern of of density of where where your dart actually hits.
If there is a tight distribution, I'm just going to draw these again in kind of contours. If I have a really tight uh distribution in X and Y, this has high covariance, high correlation. If I have a slightly lower slope and a slightly wider distribution, um kind of fatter and less slope, this will be a lower co-variance of X and Y. And eventually, if I have it be perfectly radially symmetric, um so there's no preferred direction, so it's just kind of there's no narrow direction, um then X and Y are have zero co-variance. Um, they are in fact independent uh random variables. Okay, so for independent X and Y, the covariance will be zero.
And I thought it would just be nice—I've actually hinted at this a few times—that this joint uh radially symmetric 2D Gaussian distribution—uh X and Y are independent—and I'm going to just kind of show that now because I think um I've been really dancing around this, and I think um it'll be helpful to actually kind of prove it. And this generalizes to what's known as a multinomial Gaussian distribution or multi uh multi multi-dimensional Gaussian distribution. Okay, um a higher dimensional normal distribution.
So if I have a PDF, um let's say, you know, fX of little x equals it would be 1/√2πuⁿe⁻ˣ²/², I'm saying that this thing is centered at zero and it's standard deviation of one, just to make my life easier; it doesn't have to be. And let's say that FY, the distribution of my Y variable, is um the exact same; it's a unit normal uh Gaussian centered at zero. This would be 1/√2πuⁿe⁻ʸ²/². Then if I multiply these two—and this is this is where the independence comes in—I'll switch colors for this. The Joint distribution F of x, y, I claim is just the product of these two. So let's multiply these two together and see what happens. I I I'm claiming that this is FX times FY of Y, and that's going to equal 1/2π * 1/2π * e to the minus and if you multiply these you're going to get this exponent plus this exponent because that's when you multiply e⁻ˣ²/² * e⁻ʸ²/², you get e to the minus uh x² + y² / 2. This is exactly radially symmetric; this is a variable r². So the density now is a function of uh of the radius away from this center point. This is what we expected; this is the the distribution of my 2D Gaussian, and we've just kind of shown that X and Y are actually independent.
So if I have a 2D multivariate Gaussian, then its X and Y components are independent variables, meaning the covariance of this uh X and Y should be zero. So I encourage you to actually simulate this: throw a random dart, you know, according to some Gaussian probability. Um, you can do this with RandNs, with a normally distributed random variable in Python; throw a bunch of random darts and actually compute, add this expectation up using, you know, the sum, actually add up all of those values and show that the sample expectation is very very close to zero. And this works in higher dimensions. I can have a three-dimensional multivariate Gaussian with X and Y and Z; I can have an N-D multivariate Gaussian with a bunch of independent uh or dependent variables, and I will get a formula like this, and I can still quantify notions of co-variance and correlation between the various uh components.
And so this is for independent X and Y. I could um if if these had different variances, different two Sigma squares on the on the um denominator of my exponent here, if they had different variances, I would get a stretched out ellipsoid, and then if I rotate the ellipsoid, then I'll get interesting terms in this PDF that make them jointly distributed. I'll essentially have to rotate this thing so it's no longer rotationally symmetric, and then we'll find that the co-variance in these cases is nonzero. So that's something we'll do either in a later lecture or as an exercise, is to start looking at what happens, how do you build these PDFs, these probability density functions, when you have nonzero covariance between X and Y. But in the simple case of a symmetric 2D Gaussian, X and Y are independent random variables, and the co-variance is equal to zero. Okay, thank you.