Transcription
Welcome back. So we're talking about probability distributions, uh, and random variables X. And today I'm going to introduce two of the most useful random variables, uh, the Bernoulli distributed random variable and the binomial distributed random variable. And from binomial, we're going to get, uh, the normal distribution, Poisson processes—lots of really, really, really useful, uh, models of real-world phenomena come out of these two basic, basic, uh, random variables. So let's get started.
Um, we've talked a lot about—um, we we've talked kind of fast and loose about a lot of probabilities. We've computed the probability of coin flips. Uh, we've computed the probability of getting five heads in a row or getting at least five heads out of 10 coin flips. We've talked about dice rolls and poker hands and all kinds of stuff. We've made some kind of assumptions that, um, right now we're going to formalize. For example, I've been assuming that my coin has an equal probability of flipping heads or tails. So I've been assuming that each of these has one in two, or 50%, probability.
Um, I'm going to start with the Bernoulli random variable. So a Bernoulli, uh, random variable X can only take two values: zero or one. Can take, uh, two values. So we say that, uh, the range, uh, is either zero or one. It can either be zero or one. And I could pick—I could say heads is one, tails is zero, or vice versa; it's just about how you define it. I could also define my dice roll as a random variable. If my random variable is: did I get a six on that dice roll? If my random variable is whether or not I got a six on that dice roll, the range would be a zero or a one. Zero means I did not get it; one means I did get it. Okay. So, uh, the range is either 0 to 1. This means, uh, my event does happen, um, and this zero means my event did not happen, did not happen. So I essentially have to define what I mean by a successful event—I got a six, or I flipped a head, or whatever, you know, something like that. So this—I'm just going to write down some examples. This could be a coin flip; um, it could be, you know, getting heads; it could be dice, uh, roll getting a six; it could be if I have—if I'm manufacturing a part, let's say I'm manufacturing, you know, a composite panel or um, some widget, then it could be, you know, is my part a good part, you know, as opposed to a bad part? One could be a good part; zero could be a bad part. Okay. And when I'm dealing with coin flips, the probability of zero and one are equal; each one is equally likely if I have a fair, unweighted coin. But in general, my probability, um, of X being zero—my probability of X equaling 0 and my probability of X equaling 1 do not have to be equal. My probability of X equaling zero, uh, we're going to call that P. So in the case of a coin flip, P is 1/2, but in the case of my dice rolling a six, P would only be 1/6. And then the probability of—oh, I'm sorry, I did this wrong. The probability of X equaling one—of a success—is P. We're going to define the probability of X equaling one to be P. Um, in that case, rolling a six would be a 1/6 probability; P would be 1/6. The probability of my event not happening, X equals 0, is 1 - P, and sometimes we're going to define that to be equal to a variable called Q, so P + Q = 1. Okay.
Um, and so with this—this is a very, very simple formalization of an idea we've been kind of dancing around a lot with coin flips and defective parts and, uh, dice rolls and all kinds of things—you know, how likely it is to get an, you know, Ace of Spades, things like that. Um, but here we're just formalizing that this random variable has exactly two values, um, an event does or does not happen, and those do not have to be equally likely. Um, I can have a very likely or a very unlikely or an equally likely kind of split between these—that's totally fine. So lots of things can be measured by Bernoulli random variables, um, and you know, these, uh, are pretty useful. I actually use Bernoulli random variables like when I'm, uh, programmed; I will often create, um, a matrix of Bernoulli random variables because essentially it's a binary mask. Um, I might take an image and I might, you know, with some probability P, black out some proportion of those pixels, um, kind of randomly. That would be a Bernoulli random mask.
Um, and then from the Bernoulli random variable, you can get the binomial distributed random variable, and this is probably one of the most useful core foundational random variables and distributions in all of probability. Um, a binomial distribution—it's going to eventually limit, uh, to the normal distribution; we'll see that in a minute—but a, a binomial distribution is what happens if you add up a bunch of Bernoulli random variables. So if I do one coin flip, that's a Bernoulli random variable, but if I flip my coin 10 times in a row and I add up how many heads I got, that is a binomial random variable. If I roll a single die, that's a Bernoulli random variable—if, you know, if X is getting a six—but if I roll that die 20 times and I count how many sixes I got, that's a binomial distributed random variable. So let me write this down and define it. So, uh, if N independent Bernoulli random trials are performed—Bernoulli, uh, Bernoulli random trials are performed—and, uh, I'm going to call each of these Bernoulli random variables, let's call them B1, B2... all the way up to BN. These are independent. So basically, it's N coin flips or N dice rolls, and each of these, you know, could be between zero and one with some probability—either zero or one with some probability—then the sum of these Bernoulli random trials is a binomial distributed random variable X. So, um, let X equal the total number of successes—of this being one—total number of successes, uh, of successes. And we're going to say that this is, you know, B1 + B2 + ... + BN. This sum of independent Bernoulli random events is going to be binomial distributed—a binomial, uh, random variable. So we're going to say formally the way we write this is we say X ~ binomial, and there are two parameters you need; you need to know the number N, and you need to know the probability P. So you say it's a binomial distributed random variable with parameters N and P. And with those two numbers—that's all you need to determine all of the probabilities of X equaling 5 or of X equaling 3 and things like that. Okay. So this is really, really useful. You'll get used to the notation here. We say X ~ binomial; we don't say X = or X in; we say X is distributed as a binomial binomial random variable with these parameters. And specifically, that gives us our probabilities of X; that gives us the probability of X equaling some specific value K. It's going to be—and you can kind of rewind a few lectures when we actually computed this for, you know, coin flips and and card hands and things like that—the probability if I have N coin flips, um, of getting, you know, exactly K heads was N choose K. Okay, this is like the, the, you know, this is the number of ways I can get K successes out of N Bernoulli random variables. Now the probability—because the probability of a success is P and not 1/2—I have to multiply this by the probabilities here of, uh, P to the K times Q to the N - K. Um, and this is it; this is the probability of getting K successes out of N independent Bernoulli trials with probability of success P. Um, and the way you intuitively understand this—I'm just going to reason through this—so there is a counting—this is a combinatorial factor; we are counting how many ways I can get K successes out of N. And this is the probability of each of those successes; it means K of them had to be a success, so there's P to the power K, and N - K of them had to be failures—non-success—so that's Q to the N - K. So there's N choose K ways—like ordered, you know, if I get, uh, heads, heads, tails, tails, that's the same as tails, tails, heads, heads if I'm just counting successes—and there's N choose K ways of getting K successes out of N independent trials. But now because we're not talking about coin flips—if I was dealing with coin flips, P and Q would both be 1/2; okay, these would be equal—but if I'm talking about dice rolls, then P is 1/6 and Q is 5/6. Okay, so you have to actually keep track of the probability of each of these ways you can get P successes or K successes. Good. And so this is one of the most useful probability distributions, um, around, and we're going to see very quickly that as N becomes large, this is really well approximated by a normal distribution or a Poisson distribution. Okay, so that's going to be super, super useful for, um, you know, what if I—what if I roll a die—a fair die—100 times? What's the probability that I, you know, get 10 sixes? You'd never want to write out the combinatorial problem, but this is easy to compute, and the normal distribution, which this limits to, is even easier to compute. Okay, good.
Um, so let's talk through some like really, really simple examples here, and then I'll write down some, uh, some summaries again. Um, and actually maybe I'll write down—maybe I'll do an example first and then I'll write down the normal distribution, uh, at the end. So really simple example, um, we're essentially counting successes; we're counting how many of these Bernoulli random variables ended up being one, meaning we got the thing we were looking for—heads or sixes or good parts or whatever. So let's just, uh, do some examples of how to compute this thing, uh, for small N—for N equals 2, for N equals 3. Okay, so, um, I think I'm just going to do an example here. Example, uh, let—let's say N equals 2. So let's just compute the probability, um, and we're going to assume that, you know, our Bernoulli random variable has probability P of success and Q of failure, and we're just going to do N equals 2. So the probability of X equaling zero—zero successes—essentially means I'm going to use shorthand here; it essentially means I'm doing two trials; I'm—I'm flipping two coins or I'm rolling two dice or I'm testing two parts. It means that for zero successes—for zero Bernoullis equaling one—for zero successes, then I have to have a failure on my first trial, and I have to have a failure on my second trial. That's the only way I can get zero successes out of two trials. And the probability of failure on my first trial and failure on my second trial—they're independent—so this equals probability of fail on my first trial times probability of failure on my second trial. And the probability of a failure for a Bernoulli random variable is Q, so this just equals Q². I can do the same thing for the probability that X, uh, equals 1—that, uh, is the only—there there's two ways I can do that. I can have one success: either my first trial can be a success and my second can be a failure. So, so I can have success and failure, or I can have failure followed by success. And again, because each of these are independent random trials, the probability of these two things happening is just the probability of each of them happening, uh, and this is—so this is P—success one time—P—failure—2—success of my first trial, failure my second trial, plus the probability of my first trial failing, the probability of my second trial succeeding. And I'm just using shorthand here; I could be much more careful, and I could say the probability of B1 equaling 1 times the probability of B2 equaling 0 plus the probability of B1 equaling 0 times the probability of B2 equaling 1 if I wanted to be really careful; that's what I would do, but this—I don't have enough board space, so I'm using shorthand, and you'll get good at using shorthand too. And these probabilities are easy: probability of success is P * probability of failure is Q + Q * P. This equals 2PQ. Finally, the probability of X equaling two—the only way I can have two successes is if both of my Bernoulli variables were a success—so this is the probability of success on trial 1 times success on trial 2—that's the probability of success on trial 1 times the probability of success on trial 2—and that's equal to P². Okay, I'm really heavily using this property of independence here; this is really, really important because that's allowing me to take these—these events—and multiply their probabilities here. And you'll notice that this is exactly what we have here; this is the binomial coefficients 1, 2, 1 times, uh, P², P¹Q¹, Q². You can actually write out for N equals 2 and K = 0, 1, and 2, and you'll verify that you get exactly these coefficients. Now remember, um, you could have gotten this using Pascal's triangle. If I do like 1, 1, 1, 1, 2, 1—these are the coefficients from Pascal's triangle—um, the third row: 1, 2, 1. If I wanted to do an example where N equals 3—this is actually a homework problem, so you should do this—you should like pause the video and do this or do this after the video—do this same exact procedure for N equals 3, and the—the coefficients will be again the coefficients from Pascal's triangle—three choose zero, three choose one, three choose two, three choose three are going to be 1, 3, 3, 1. And you can keep, uh, kind of writing this down and reading this off of Pascal's triangle. Eventually, this is going to limit to a normal distribution, so that's what we're talking about is in the limit of large N, we're going to start getting a normally distributed, uh, number of successes. But you can verify and kind of confirm that this all works out and is tidy. So if you wanted to do this—do this for N equals 3—and then let's say the probability of, you know, X equaling one—let's say I'll do one example for you—so for N equals 3, the probability of X equaling one would be the probability of—the only way you can get a single success is either success on the first one followed by two failures or, um, a—a failure on the first one, a success on the second one and a failure on the third one, or a failure on the first one, a failure on the second one, and a success on the third one. This is the only way you can get a single success—a single successful Bernoulli trial out of three—and if you compute these probabilities—I'm guessing you'll get something like 3 * P * Q²—3 * P * Q²—this is going to equal 3PQ², which is exactly what this formula would predict. Okay, and you can fill out the rest for X equals 0, 2, uh, and 3. Okay, good.
Um, this is another exercise I think would be really good for you to do: verify that these probabilities add up to one. The sum of all of these probabilities over all the values X can take always has to add up to one. That's one of the most fundamental properties of all of probability is that probabilities add up to one. So convince yourself that these probabilities add up to one. Now the way I would do this is I would essentially say, well, Q² + 2PQ + P²—Q² + 2PQ + P²—is equal to (Q + P)² and Q + P adds up to one—Q + P is 1—this is just 1². So the probability of adding all of these up—the total probability—adds up to one in this case. Now you should verify for the N equals 3 case that the probability still adds up to one and generalize to the N—you know, arbitrary N case—that the probabilities still add up to one, and you get this really nice connection to the binomial coefficients and to these polynomial (P + Q)ⁿ. Um, okay, good. I'm almost done. I'm going to write out, um, in the limit of large N what these binomial, uh, distributions look like. I'm just going to write out a couple more examples for you, um, how you would actually write this. This was just some like, you know, checking details and doing some—some basic calculations. Um, but let's say like, you know, example, uh, what's the probability of getting—what's the probability of getting three sixes on 12 dice rolls? Of getting three sixes on 12 dice rolls? So a success means getting, uh, a six. So we're going to add up 12 Bernoulli variables where the probability of success is 1/6 in each case. So that means, um, that my random variable X, which is the number of sixes on 12 dice rolls, X is going to be binomial with 12 trials—with N equals 12—and a probability P of success of each of the trials is 1/6. So X is a binomial variable—the number of sixes on 12 dice rolls is a binomial variable, uh, with parameters 12, 1/6. And so I could compute the probability that X equals 3; probability that X equals 3, um, is—essentially I just plug in this formula—that is just, uh, 12 choose 3 * (1/6)³ * (5/6)⁹. And you can crunch through this on, you know, your calculator—type it into Google, do it in Python; it doesn't matter, uh, you know, you'll find some number. I don't know—I don't know what the actual probability is, but you can compute this, and it's a number—it's a probability—ility of three sixes in—in 12 dice rolls. It's a very simple formula; you can type this out and get the answer. Okay, good.
Um, last thing I want to show you: so zoom out—a single trial of a random variable that could have a success or a failure. So you define what that means. So even if my dice roll—there's six things that can happen—I can define what I mean by success to be: I roll a six or I roll a one. And then that establishes a Bernoulli random variable where it can only be success or failure with some probability. And then if I add up a bunch of those Bernoulli random variables, I get a binomial distribution, which has a very simple, compact formula that I can use to compute these probabilities. If I actually wanted to enumerate this—to count this the old-fashioned way—it would be a god-awful mess; like it would be pages and pages of—of terms. This is way simpler to compute because, you know, it's a binomial distribution. And in the limit of large N, these binomial coefficients—this binomial distribution—is going to start looking a lot like a smooth normal distribution—like a Gaussian bell curve kind of distribution. And so that's the last thing I'm going to show you is that for large N, uh, essentially as N goes to infinity or N gets large, what we have is we have our binomial—our binomial distribution—starts to become—so our binomial, which is with parameters N and P—so I'm just going to draw this really, really carefully—if X is binomial, uh, with parameters N and P, then for N large, this starts to look a lot like a normal distribution—a Gaussian distribution—with, um, the mean equal to NP. This is the expected number of successes—is the number of trials times the probability of success—and it's going to have a standard deviation which is √(NPQ). This is going to be the spread in the distribution. Okay, so for large N, this starts to look like a normal distribution, which is really, really convenient to calculate with. We're really good at doing math on normal distributions, and we know a lot about these; they're really well-behaved, uh, distributions. So that's if N—if P—ah, this is if N * P * Q is not too small. So success and failure are not too rare, then this is absolutely going to be true. If N * P or N * Q is small, then this would limit to a Poisson distribution; then X would become a Poisson random variable with parameter λ where λ = NP if, uh, NP or NQ is small. So these are two of the most important, useful distributions in all of probability and statistics are the normal distributed random variable and a Poisson distributed random variable. There—what happens when you have, um, kind of these binomial distributions with either rare events or really, really large N, um, and not-too-rare events—and we're going—the next two lectures we're going to talk about each of these two distributions—how to use them and examples of real-world phenomena that are actually distributed like normal and Poisson random variables. Okay, thank you.