Transcription
Welcome back. Okay, last lecture we introduced the moment generating function M, which is the Laplace transform of the probability density function for a continuous random variable X. It has the very, very powerful property that this moment generating function M can be used to generate all of the moments of your probability distribution. So the first moment, uh, the expectation of X, is related to your mean; the second moment, the expectation of X squared, is related to your variance; the third moment, expectation of X cubed, is related to your skewness; fourth moment, curtosis; and so on and so forth. These give a unique kind of fingerprint or unique ID for your probability distribution, and this is a very, very useful way of thinking of PDFs in terms of their moments, their higher order moments. Okay.
So specifically, I wrote down this theorem. I'll write—I'll kind of circle this in pink because this is what we're going to talk about today—this theorem is that you can get the nth moment, the expectation of X to the n, where X is a random variable; you can get this nth moment by taking the nth derivative of your moment generating function and evaluating that at zero. Okay, this is going to be a useful way of computing these moments. So I'm going to prove this theorem, and then I'm going to give you an example on the Poisson distribution.
Okay, so the proof is pretty simple. Um, the proof here is essentially that the proof—uh, what we're going to do, we're going to write down the moment generating function. I'm going to write this down for continuous variables; this is also true for discrete variables. Um, so my moment generating function M of T is just this integral, minus infinity to infinity, of e to the TX F(x) DX. Nothing fancy here; this is just the definition. And we're going to write down the derivative M prime of T, and I'm just going to write this down: this is equal to the integral, uh, negative infinity to infinity, of x e to the TX F(x) DX. Now, how did I write this down so quickly? There's no chain rule; there's no integration by parts; there's nothing fancy here. This M prime is the derivative; this is the derivative of M with respect to T, not with respect to X. So taking the derivative of M with respect to T is really easy; this doesn't depend on T; the only thing that does is e to the TX, and the derivative of e to the TX is x e to the TX. So super easy to compute the derivative of this thing. Okay.
Um, similarly, I can write down the second derivative if I like. M double prime T equals integral minus infinity to infinity; this will end up being x squared e to the TX F(x) DX… dot dot dot. You know, the nth derivative is going to equal integral negative infinity to infinity x to the n e to the TX F(x) DX. And now, if I evaluate these at zero—e to the zero is one—and so what I'm left with is a really nice expression. This implies that M prime at zero is just integral negative infinity to infinity of x F(x) DX. So this becomes one; this becomes one. This also implies M double prime zero equals integral negative infinity to infinity of x squared F(x) DX… dot dot dot… uh, M nth derivative evaluated at zero is integral negative infinity to infinity x to the n F(x) DX. These are the definitions of these moments. The expectation of X is literally the integral of x * F(x) over all possible X; the definition of expectation of X squared is the integral of x squared F(x) over all possible values of X; the definition of the expectation of x to the n is the integral of x to the n F(x) DX. So this is by definition expectation of x to the power n; expectation, you know, uh, expectation of X and expectation of x squared. Okay.
So this is kind of a proof that uh you can get these moments, these expectations of x to the n, really, really easily by taking the nth derivatives of your moment generating function, okay, and evaluating at zero. So this is just a proof um that this can be used for calculating these moments. Very interesting, very useful. Now let's do a really quick example on the Poisson distribution. So we've already written down the moment generating function for this Poisson random variable. Remember, um, Poisson distributions are good for uh calculating how often a rare event is, you know, expected to happen. So, you know, if I have um light bulbs failing, you know, on average at a certain rate, and I have a thousand light bulbs, how many do I expect to fail in the next five days? You can kind of frame these as as Poisson type problems. Okay, so Poisson—quantify rare events—um, and this is the moment generating function for the Poisson distribution. So let's uh actually do this now. Let's write down from this moment generating function; let's compute the expected value, uh, the variance, and so on for this Poisson distribution.
So for Poisson, Poisson, we have M of T equals e to the lambda * (e to the T - 1). That's the moment generating function. So M prime of T—if I take the derivative of this with respect to T—now again, you should, you know, pause; use the chain rule; remind yourself that this is true—but it's a pretty easy fact that this is e to the lambda (e to the T - 1) times lambda e to the T. That's just the first derivative of this function. M double prime T is e to the lambda (e to the T - 1) * lambda e to the T squared plus e to the lambda (e to the T - 1) lambda e to the T. Okay, and dot dot dot; you can compute the third derivative, fourth derivative, and so on and so forth. This is the basic, you know, template here; these are the first and second derivatives. And so I can compute um, you know, expectation of X as my first derivative of my moment evaluated at zero. So I literally take this function and I plug in T equals zero. So e to the T's all become ones; this is um, you know, e to the (1 - 1) is 0; e to the 0 is—this whole thing becomes a one; this becomes a one; and I just get a lambda out. Again, I'm going kind of fast; convince yourself—if you plug in T equals zero—this becomes one; this becomes one; and I just get a lambda left. Expectation of X squared, similarly, is just M double prime evaluated at zero. So I take this whole big expression and evaluate it at zero; same thing; this expression becomes equal to one; all my e to the T's become um become ones if T equals zero; and I got a lambda squared plus lambda. So I get this equals lambda squared plus lambda. So my expected value of this distribution is lambda; that's actually true; we derived this, you know, lectures ago when we were introducing Poisson; this is definitely true. And now the variance of X, which we know is expected value of x squared minus the square of the expected value of x, so this equals lambda squared plus lambda minus the expectation squared is minus lambda squared, so the variance is also equal to lambda. And remember that's what we derived last, you know, a few lectures ago when we were working on Poisson: the expected value and the variance are both equal to lambda, which is kind of a fun and strange property. So I'm actually curious; I'd like you to do this as a homework: what's the third moment? What's the fourth moment? What's the fifth and sixth and seventh? I'm sure you're going to get an expression like lambda cubed plus lambda squared plus lambda and so on. You know, you'll get this kind of sequence, and then that can tell you something about the the skewness, the curtosis; all of these higher order moments um, you know, are easy to compute in this Poisson distribution. So moment generating function—this really powerful function, which is the Laplace transform of your PDF—allows you to compute moments like the expected value, expected value of x squared, x to the n, which are useful in computing things like the mean, the variance, the skewness, the curtosis, kind of the the profile or the fingerprint of your probability distribution. Okay, thank you.