Transcription
Welcome back. Okay, we're talking about the moment generating function, which is the Laplace transform of your PDF. Very useful function for computing things like expectations, variance, and higher order moments of a distribution. And also, this is going to be critical in proving the central limit theorem, one of the cornerstone results of probability and statistics. That says that if you have the sum of a bunch of IID (identical independent) random variables, their sum starts to become distributed like a normal distribution. Very powerful result, and the proof relies on the moment generating function. So I want to give you a property that I find super useful and kind of intuitive, based on what we know about Laplace transforms from differential equations. A property of the moment generating function.
So we're going to let X and Y, we're going to let X and Y be independent random variables. X and Y be independent random variables. They do not have to be from the same distribution, just two independent random variables. And they have two moment generating functions; they have MX and MY are their moment generating functions. Okay, we have two random variables, independent; they have two moment generating functions. And we are going to define a new variable. So define Z = X + Y. And what we're going to say is that the property I want to tell you today—this is the property—is that the moment generating function of Z is equal to the product of the moment generating functions of X and Y. It's equal to the moment generating function of X, this is a function of T, times the moment generating function of Y, this is a function of T. That is the moment generating function of Z.
So this is super cool. If I have two random variables and I add them up, the moment generating function of their sum is the product of their moment generating functions. Now this should make you think about differential equations where I have the Laplace transform of two signals. And before I would have to convolve them, but now if I Laplace transform that, I get the product. So this is a lot like transfer functions in differential equations and control theory. If you kind of, you know, add up blocks, then the transfer function is the product of those blocks. Where, in differential equations—that's just an aside in case you're comfortable with differential equations. If you've never seen differential equations before, this is just a new property in statistics that happens to be useful. And what specifically is useful? Again, I'm going to write down how this relates to the central limit theorem, because that's why I'm telling you all of this.
Is that if I have a bunch of random variables, X1, X2…Xn, and I define a new variable SN, which is the sum of all of these random variables, then this new sum, this random variable that's the sum of all of these, its moment generating function—the moment generating function of my sum—is just the product of all of these moment generating functions. It's the moment generating function of X, sorry, it's so squeaky, to the power n. Okay, and this is going to be very, very, very, very useful when we prove the central limit theorem because we're going to specifically be saying that this sum converges to a normal distribution, which means we're going to want to show that this moment generating function is the moment generating function for a normal distribution. That's how we're going to prove the central limit theorem, one of the most important results in probability and statistics. So this is the statement, this is the property, and I'll give you the thumbnail proof of why this is true.
So the proof is pretty simple. So the moment generating function of Z is specifically defined as the expectation of e to the T times my random variable Z, which is the expectation of e to the T times the sum of my random variables X + Y. Now there's a few ways you can do this. You could actually now plug this into the formula for expectation. This is now going to be the integral over X, integral over Y, e to the T(x + y) times the PDF. You could do that, and you can show that this is—you get what you want—or you can just write this as this is the expectation value of e to the Tx * e to the Ty, because that's what how exponential products work. And X and Y are independent, so the expectation of this and this are independent, and so this should give me the expectation value of e to the Tx times the expectation value of e to the Ty. This is defined as the moment generating function of X times the moment generating function of Y. I skipped some steps here. If you really, really, really wanted to be careful, I would actually start here and I'd write out the full integral of this expectation value, and I'd show that you can split that integral over X and Y into one integral over X and one integral over Y, and that you'll get these expectation values for that that result in the moment generating function. This is just meant to be a thumbnail sketch of this very, very important property here, that if I have two independent random variables—they have to be independent—then the sum of those random variables, its moment generating function is the product of the two moment generating functions. Very, very useful, and we're going to use this specifically when we prove the central limit theorem in a little bit. Okay, thank you.