Transcription
Welcome back. So this is one of our first lectures on probability in this series on probability and statistics. And so I'm going to start off really, really simple, with just defining what is a probability, giving some really simple examples with things like coin flips and dice rolls. And then I'm going to give you some idea of how probabilities can be used in much more complicated, um, uh, tasks in the real world; things like modeling the likelihood of part failures, or hurricanes, or you know, things like that. Okay. So at the at the very, very simplest, probabilities are how we calculate how likely some event is to happen. So the probability of an event A—now I'm going to define what I mean—so an event A could be, if I flip uh two coins, the event A could be that I get no heads. So what's the probability of an event happening? The probability—I'm going to try to color code here if I can—the probability of A happening is the number of ways that this event can occur; number of uh number of ways A can happen, can happen, divided by the total number of things that could happen in period; divided by the total number of things that can happen; like total, the total number of things that can happen.
So I'm going to give you some really simple examples with coin flips and dice rolls, um, but very, very soon we're going to start computing the probabilities of pretty sophisticated uh things. We're going to compute the probabilities of, you know, what's the probability I get a straight flush in poker? What's the probability that if I have a triple redundancy system I could still have a failure on a critical system? What's the probability that um if I have a positive test or a symptom that I actually have the disease? Okay, these are the kinds of things we actually want to compute the probabilities of, and we'll get to those very quickly, but let's start with some really simple examples. Okay. So let's start with um coin flips. That's super easy. I have an actual coin right here. This is a Washington state quarter; it has Mount Rainier and a salmon on it, and I can flip this quarter two times. Okay, I got a tails the first time, and I got a heads the second time. Okay, so this is a fair coin; I actually shaved this morning so you wouldn't second guess whether or not my coin is fair. So let's just set up a problem. Okay. So the example is, is I flip a coin twice. I flip a coin twice. Now two questions I could very easily ask are, what is the probability that I get um what's the probability I have at least one heads? So um let's say that my event A is at least one of these coin flips turns up heads. And two, I could say what's the probability—so another event B could be—what's the probability that there are no heads at all? Probability of no heads at all, period, out of the two coin flips.
So the way this is super easy to compute, and I think you know any of us uh school kids, right, could compute this pretty easily, but I'm going to formally do it so that we see that it really is a just about fancy counting. Okay, and then we'll do a more sophisticated example, and then we'll zoom out. So the first thing I'm going to do is I'm going to compute this this denominator here: what is the total set of things that could happen, period? So if I flip a coin twice, the things that could happen: I could get—maybe I'll do this in uh in orange—I could get um heads and then heads; I could get heads and then tails; I could get tails and then heads; or I could get tails and then tails. This is the set of all things that could happen, and we're going to just give this set a name; we're going to call it Omega, um, for right now. You don't have to worry about set theory, but very soon we're going to need, need to start defining what is a set and some set theory to do more sophisticated probability. But for now let's just say this is kind of a collection of all of the things that could happen as Omega. There's four things that could happen, so this denominator is four. Now A, being at least one heads, what are the set of coin flips where at least one of the coins was heads? So now A is the set: heads heads—one of those is a head—heads tails, and tails heads. But not tails tails; neither of those are heads, so it doesn't satisfy event A occurring. And so the probability of A is the number of ways A can happen—there's three ways A can happen—divided by the total number of things that can happen; so three-fourths of the events have at least one heads, or 75 percent. Okay, so that's really, really simple, and I'm just kind of codifying what you already know about basic probability. And one of the things I love about probability is that you know all of our favorite games—darts and backgammon and poker, you know, uh dice, D&D—all of them, you know, they have some element of randomness, of of of probability, and you can actually calculate the probabilities and get better at winning those games. So that's kind of fun. Okay. Uh, what's the probability that there are no heads? Well, so this set B, there's only one element in the set, and that's if I flip two tails. That's the only way I can have, you know, I don't get at least one head is if both of the coins are tails, and that means there's only one way that B can happen. So the probability of B is one in four, or 25 percent. Okay, really easy to compute these, but for example, what if I flip a thousand coins? What's the probability that I get 300 heads? That would be a huge pain to count all of the possible ways I could get a thousand coins flipped and then, you know, add those all up. So we're going to very, very quickly come up with formulas and models and better ways, clever ways of counting these probabilities. That's all probability is, at the end of the day, is just clever ways of counting how many ways that thing can happen out of the total number of things that can happen. This is actually, you know, also how we define entropy in statistical mechanics; it's how we think about, you know, the theory of gases and gas dynamics; it's actually a very, very big idea. I'm just showing you some baby examples right now, okay, to get kind of make us feel comfortable. Good. Um, maybe I'll do one more example, and then we'll we'll zoom out.
Second example is pretty easy. So uh my second example is, let's say I roll two dice. Okay, I roll uh I roll two dice. Now, depending on who my audience is, I have to specify that this is a six-sided dice with equal probabilities. Okay, again I'm making assumptions here, and you need to be defining those assumptions if you want to be really careful. This was a fair coin, and each of these flips was independent, meaning the second flip had nothing to do with the first flip. This is a fair six-sided die, and each roll is independent, meaning the two dice don't affect each other. Um, now that isn't always the case, but it's it's the easiest set of assumptions. So some of my questions I can ask would be things like, uh, what is the probability that at least one of these dice is a five? The probability um at least one die is a five. Okay, I'm like running out of space here, but what's the probability that at least one die is a five? The there's a lot of ways of solving this; the you know kind of most simple foolproof way, and it's not the best way, but the simplest way would be, you know, just list all of the possible combinations. What is how many possible ways can I roll two dice? Well, the set of all dice rolls: I could get a one on the first die and a one on the second one; and two, a one on the first die and a three on the second; a one on the first die, four on the second; and so on and so forth, dot dot dot dot dot. I could get a, you know, um a five on the first die and a two on the second; a five on the first die and a three on the second; dot dot dot dot dot. And eventually if I enumerate all of them, you know, the last one would be a six on the first die and a six on the second. And it's pretty easy to compute just with uh kind of basic arithmetic that there are 36 total things that can happen if I roll two dice. The first one, there's six possible dice rolls, and the second dice there's six other unique dice rolls for that die, and so there's 6 * 6, or 36 possible things that could happen. So now I want to count how many things can happen so that I get at least one die is a five, and I'm just going to go through it even though it's a little boring and you know how to do this, just to remind you that I'm just going to take out of all the things that can happen, all of the cases that satisfy this condition of at least one of the die being a five. Okay, so if the first die is a one, the second die has to be a five; if the first die is a two, the second die has to be a five; 3, 5; 4, 5. Here's where it gets interesting: if the first die is a five, the second die can be anything: 5, 1; 5, 2; 5, 3; 5, 4; 5, 5; 5, 6. And then again, if the first die is a six, the second die has to be a five for at least one of them to be a five. And if I count this: one, two, three, four, five, six, seven, eight, nine, 10, 11. The probability of A occurring is 11 out of 36; so just under one in three chances of getting at least one die being a five. So that's pretty easy. Um, now again, if I wanted to compute the probability if I rolled, you know, 15 dice, what's the probability of at least three of them being five? That would be very hard to compute using this counting method. The counting method is like it's like count on your fingers; you don't actually want to do it, but you can always fall back on it as a thought experiment; that's how you, you can always count that way. Another way to do this would be I could draw a picture; I could say, well, you know, I have six possibilities for the first die, and then for each of those I have six more possible outcomes, so each of these, if I drew it correctly, would have, you know, six more, and I could count that way. That's another way to do it. Um, maybe a slightly better way to do it would be to say, you know, to actually try to reason through the ways these could happen. I could say the probability of A is equal to the probability that the D1 equals a five plus the probability die one doesn't equal five times the probability that D2 equals five. So either die one is five, and if the first die is not a five, die two has to be a five. And I can compute this pretty easily: the probability that the first die is a five is one in six; the probability that the first die is not a five is five in six—there's five ways that it cannot be a, not be a five—times the probability that the second die is a five, that's one in six. And if I add these up, I get 6 over 36 plus 5 over 36 is 11 over 36. This is a slightly better way of calculating this without listing all of the possibilities, um, you know, ad nauseam. Good. Okay. So really, really simple. Let's zoom out.
Probability is one of the most important ways we model the real world. We model weather probabilistically; we model health outcomes probabilistically; we understand the likelihood of having cancer given symptom X as a probability; um, you know, we build probabilistic models of part failures and manufacturing outcomes and safety outcomes. You know, what is the probability that my self-driving car is going to hit a pedestrian? What are the probabilities that some critical component is going to fail, you know, uh while I'm driving or in midair or, you know, etc., etc., etc.? Those are probabilities that we're computing, and we want to model those. At the end of the day, you calculate how many ways that thing can happen divided by how many total things can happen; that's, you know, that that's the simplest way to think about probability. But again, zooming out, I made some big assumptions here. In these cases, the probability of a head or a tail, those are equal probabilities; the probability of, you know, all of the dice rolls, it's a fair dice, so one is as likely as two is as likely as three and so on and so forth. So we'll get into cases where um there are, you know, not these kind of uniform probabilities of things happening. We'll also get into cases where the probability of sequential events is not independent, where future things will depend on past things; that's interesting too. Um, there's this interesting idea that this coin is random. We haven't really talked about what we mean by random. Um, random is essentially this idea that the outcome of this coin flip is not deterministic, or is very hard to model deterministically, meaning we have to build a probability model for the outcome of this coin flip. It's a little circular, but if you think about it, flipping this coin actually is deterministic; it's a physical coin; it has mass; it's not, you know, quantum mechanical. When I flip this coin, there's wind resistance; there's mass; there's inertia; it's acting under the effect of gravity; it is a predictable, deterministic system; there is, you know, a formula essentially for the physics of this coin, and you could simulate coin flips. What we mean by random is that the outcome of this coin flip is too hard for me to predict with the information I have. So for me, it might as well be random; it might as well be a 50/50 chance. Now that's interesting in the machine learning era; I actually bet you could take a video of me flipping a coin, and if you had enough information, you could probably predict what the outcome is going to be better than a coin flip—better than 50/50 odds—from the information of that coin rising to its peak. I bet by the time you see it go up to its peak, you could call it in the air, heads or tails, with a machine learning algorithm. That might be a cool project to try, to build um actually build a classifier to see if you can tell, you know, what the coin flip is going to be in the air just from watching it flip. Okay, maybe you could, maybe you couldn't; I don't know. I can't. And so for me, it's a random event; it's a there's a probability associated with it, and that's how most of the world is. You know, weather is deterministic; there's physics that drives the climate that drives the weather that drives clouds and storms and tornadoes and hurricanes, but with the information we have, with the uncertainty in our measurements and the uncertainty uh in our information at some point in the future, all we can say are probabilistic estimates and probabilistic forecasts. The better we measure and the better we model those physical systems, the tighter those probabilistic estimates become, and eventually they might become deterministic; we might be certain that this is going to be a heads just from observing its its path in the air. But with increased uncertainty, things look more and more random. Now some things truly are random: the decay of a radioactive element; that's actually random; that is a stochastic process; it's kind of a whatever quantum mechanical process that is truly random. So radioactive decay um is an actual uh uh random process, but a lot of the things that we pretend are random are actually deterministic; they're just too complicated for us to actually understand and model, and so we model them with probabilities instead. It's a very, very useful way to model things like turbulence. If I think about the molecules of gas in this room, I can't model every single atom, and I don't want to; I model the average value of their velocities, and I call it a temperature; that's a a a probabilistic or a statistical way of quantifying information.
I guess parting thought here: very, very important; we're going to get to this soon. Probability is all about modeling what can happen and counting what can happen. Statistics is almost the inverse problem. Let's say I flip a coin 10 times and I get 10 heads in a row. What are the chances that that coin is fair? That's a statistics problem. So I have a model of what I think the coin is behaving like, but then I actually collect data, and it might disagree with my model. What are the chances that that is truly just from the randomness, the natural randomness of the system, and what are the chances that my coin, my model of the coin are wrong? So that's what statistics is all about: is hypothesis testing—testing your probability model, collecting data, and making hypotheses uh about that or inferring things. If I if I measure certain genes or certain um health traits, can I infer what is the likelihood of, you know, some underlying uh causal effect, um, you know, some some underlying disease that's causing those symptoms, that's causing those test scores? So those are all things that we're going to talk about soon in probability and statistics. We're going to start with real simple examples; very quickly we're going to learn how to calculate much more sophisticated things like poker hands, probability of part failures, how many parts you need to inspect to be 99 percent sure of a certain quality or a certain reliability, things like that. Tons, tons more coming. This is one of the most powerful, useful uh ways of modeling the world. We're going to question our assumptions; we're going to try lots of things; we're going to build lots of probability models; and soon we're actually going to start collecting data and running experiments and doing statistics against these probabilistic models. Okay, thank you.