Transcription
Welcome back. Okay, I'm going to work out two examples that involve the expectation and variance of a function of a random variable. Both of these are physical examples. So, in this case, we're going to assume that we measure some variable for temperature in degrees Celsius, and we want to convert it to Fahrenheit. Don't ask me why you'd want to do that, but there's a relatively simple, uh, linear formula to convert from uh X in Celsius to Y in Fahrenheit. You just use this linear scaling, a plus a constant offset, B.
And so we'll figure out what's the expectation value of my measurements in degrees Fahrenheit if I know it in Celsius, and what's the variance of this new, uh, this new variable. This is important because we want to know: if I change units, do I have to recompute everything? And if I change units and offset, how does that change this expectation and variance? So this is kind of an easy example.
Then, for a more sophisticated example, we're going to look at the kinetic theory of gases. So, of course, temperature is just a mean kinetic energy, uh, of the gas in my room. And we're going to look at the kinetic theory of gases where we have a random variable X, which is the magnitude of velocity of a gas molecule. It satisfies Maxwell's distribution, and we're going to try to find the expectation value, value of the kinetic energy. So I'm actually just going to write that down: the kinetic energy Y is going to be 1/2 M x^2. So X is the distribution of my velocities, so my kinetic energy has this distribution, and we're going to find the expectation of kinetic energy given Maxwell's distribution for velocities.
These are the two examples I'm going to work right now. Uh, let's get started. So this one's really easy. I'm going to start with the expected value of ax + b, and I'm just going to kind of write down some math and verify what I think is the common sense answer that this should be a * the expectation of X + b. So, let, let's try it.
So the expected value of aX + B, I'm actually going to write this whole thing out. I'm going to assume X is a continuous random variable, like maybe it's Gaussian distributed, a normal distributed. So I'm going to use the continuous form, but you could do this for discrete just as well. Um, okay, so this is the integral of ax + b times uh f(x) dx, and we're integrating over all of the domain of x, from negative infinity to infinity, if you like. Notice that here, um, this expectation value of a function of X, you take and you replace that function here with little x, uh, in that expectation. So the regular, the expectation of X would just be the integral of x * f(x). The expectation of G(X) is the integral of G(x) * f(x). I'm actually going to write that down: the expectation of G(X) is the integral over all of X of G(little x) times my probability density function f(x) dx.
Okay, this is kind of a, a definition from earlier, or not a definition, a property, uh, we wrote down from earlier that's really important; that's what we're using here. And because ax + b is linear, I can kind of expand this out, and I can say that this is equal to the integral of ax f(x) dx plus the integral of b * f(x) dx. And B is a constant; I can pull this constant out of the integral, and my integral of f(x) dx is equal to one. That's the law of total probability; this is just my probability density; area under the curve is one. So if I multiply it by a b, this had better just equal B. Okay, that's, that's pretty easy. This one, I can pull this constant a outside of this again; it has, has nothing to do with X, so I can pull it outside, and then my integral of x f(x) dx, that's just my expectation value of x. So this should equal a times my expected value of of X. And so the expected value of ax + B is equal to a * the expected value of x + b. And this is very intuitive; this is what we thought in our gut should be true, is that if I measure in Celsius or Fahrenheit, it doesn't matter. If I have the expected value in Celsius, I can convert that expected value directly to Fahrenheit, and I get the expected value in Fahrenheit. Okay, so I don't have to do anything magic to go between um these different units. I can either change all of my data directly to Fahrenheit and then compute the expectation value, or I can compute the expectation value in Celsius and then convert to Fahrenheit; kind of order doesn't matter. Okay, and that makes sense because this is a linear function of x, a linear transformation; that makes a lot of sense. Okay, good. Uh, let's do the next one; this will be slightly more involved, but not really much.
So now let's look at the variance of ax + b. Okay, so Var(ax + b) is um, so Var of um ax + b is the same as the variance of my y variable, my new random variable Y, and that's equal to the expectation value of y minus the mean quantity squared. This is uh the expectation of the squared deviation of Y from its expectation, from its mean squared, this expectation value. Okay, this is just the definition of variance. And so now I can plug in ax + b here, and I know this expectation value here, so I can plug both of these in here and expand these out. So, going to go all the way over here, this equals the expectation of ax + B; that's just y minus this is going to get a little hairy—minus this expectation of Y, and all of this I'm going to square—minus the expectation of Y is a expectation x uh minus B, all of that quantity squared. And now you'll notice that right away my B's cancel, and this is a good thing. If I take and I just shift my data, I shift my distribution; that shouldn't change the variance at all. That doesn't change the spread; it does change the mean, but it doesn't change the spread or the variance. So these B's should cancel; that's intuitive. Okay, good. Um, and now I'm just left with expectation of ax − a expectation of X quantity squared, and I can write that as expectation—I'm just going to, you know, pull out this factor of a—a squared times x minus the expectation of X quantity squared. Um, property of expectation values is that a constant, I can pull out constants, so this equals uh a^2 * the expected value of x minus its expectation quantity squared. This is just the variance of X. So this means that the variance of Y is equal to a^2 times the variance of X. Okay, so if I scale up, if I change my units from uh Celsius to Fahrenheit, and I scale up by a factor of 9/5ths, in these new coordinates my variance actually is going to be higher; my variance is going to scale up by (9/5ths)^2. So variance does get affected by scaling your distribution; if I make, if I, if I scale this, my variance apparently does scale; that's kind of interesting. Um, so this is important to know if you are computing mean and standard deviation, but you're going to be switching units. Okay, okay, so we have all of these classic examples of people going back and forth between these units and missing something, you know, that some conversion factor. This is a pretty important one, pretty important conversion factor here, but this all kind of makes sense intuitively. Good. That's one example. This example is a little hairier, and I'm only going to show you the broad brush strokes of how this works because the integral gets kind of nasty, and it's not worth, you know, uh, kind of going through 20 minutes of nasty integrals here, but I'll give the basic idea.
In the kinetic theory of gases, there is this Maxwell distribution that governs the probability density of finding a particle of gas, a gas molecule, traveling at a certain velocity X. Okay, so this is like the magnitude of velocity or the speed of a particle follows this Maxwell distribution. Now it looks kind of like a Gaussian, but it's not a Gaussian; it has this weird x squared out here; the variances don't match; it's kind of this weird function. Okay, this Maxwell distribution, and you can actually derive this uh from first principles with like entropy arguments; it's really quite beautiful, but that's a different, different course. So if we assume that the velocity of a gas is distributed according to Maxwell's distribution, then the kinetic energy, the kinetic energy would be distributed, kinetic energy would be distributed according to this variable Y, which is 1/2 Mass * x^2. Now remember, you can't just get this PDF by taking this PDF and squaring it; that doesn't work. If you wanted to write down a, a probability density function for the kinetic energy, you'd actually have to start with the cumulative density function, uh, and then take its derivative; that's a whole thing; it's an extra bunch of steps. But I'm going to write down now what is the expectation value of this kinetic energy. Okay, and I'm going to, again, broad brush strokes, not exact; I'm just going to show you kind of how this would work, but this is a really important calculation that people had to do when uh they were developing the kinetic theory of gases. This is like, what's the average kinetic energy? Well, the average kinetic energy is the temperature, so this is a really, really important quantity, this expected kinetic energy. Okay, good. So the expectation value of y, I'm just going to write this down; it's a bit of a mess, using this formula here, is the integral from 0 to infinity—why is it zero and not negative infinity? Well, because I can only have positive magnitudes of velocity; they can't be negative speeds—okay, so integral from 0 to infinity of the function 1/2 M x^2 times my PDF in x, f(x) dx. Okay, so now we're cooking; we actually have an integral; the expected kinetic energy, the temperature is going to be whatever this integral is, and look, it's, it's just an integral in X; it's kind of a nasty integral in X, but it's an integral in X, and it's got, you know, finite, it's got definite bounds, so this is going to pop out a number. Okay, uh, good. And so you could write this out um as—I'm just going to write a couple steps—m/2 (tk^2/π)^(1/2) / σ^3 times the integral from 0 to infinity of x^4 * e^(-x^2/2σ^2) dx. Now this is where it gets pretty nasty. Okay, so there is a change of variables that allows you to solve this integral; you could plug this into uh Mathematica; you could use a change of formulas and actually solve this thing. So I'll just say like, I'll tell you what the change of variables is… change of variables … where the change of variables is u = x^2/2σ^2. If you use that change of variables, you get another integral, which I'm not going to write out all of the gory details… it is—I'll write it out, why not—2mσ^2/(π)^(1/2) times the integral from 0 to infinity of u^(3/2)e^(-u) du. And if you're a calculus wizard, you'll know that this is uh 2mσ^2/(π)^(1/2) times the gamma function evaluated at 5/2. Now the gamma function generalizes the factorial to non-integer values; it's a very cool function; um, it actually comes up a bunch in probability; there's something called the gamma distribution; it's a really cool thing, but it, this gives you a number; you type this into Mathematica; it pops out a number. And so now all of this is just a number, and it turns out that that number is (3/2)mσ^2. So given that my velocities of my gas molecules are distributed according to Maxwell's distribution with these, uh, with this probability density function, the expectation of my kinetic energy, which is a function of my random variable, this, this function of my random variable, you go through all the math and you get this for the expectation value of kinetic energy. This is the temperature of your gas; very, very useful. And this is, you know, fundamentally a probability and statistics problem; this is, you know, physics, but we are assuming that our gas is distributed kind of randomly, according to some distribution, probably governed by, you know, principles of entropy and least action and things like that, but you go through all of this math using things we've learned from expectation values and probability, and you can actually derive the temperature of your gas. Okay, these are just two examples of how to work through, you know, expectation values and variances when you have kind of functions of a random variable X. Both of these are very physical examples of things you might actually care about as an engineer. Thank you.