Transcription
Welcome back. Okay, uh, one of the very, very important concepts in probability that you're going to use all over the place—you're going to hear this now that I tell you about it all the time—is independence, statistical independence of two events. So we say that two events A and B are independent of each other if having information about one of them doesn't change the probability of the other one happening.
So two events A and B are independent if knowing that one of them occurred—knowing one occurred, I hope that's how you spell occurred—gives no extra information about the other; uh, gives no extra info about the other. And you can think of a million examples of this. Um, you know, two events A and B being completely independent. Um, you know, um, did I drive to work today or not? Um, you know, am I going to pick the green pen or the blue pen next? Those are probably independent. Okay.
So two events A and B are independent if knowing that A happened doesn't change the probability of B, and if knowing that B happened does not change the probability of A. And the way that we write that down, uh, formally or mathematically, is really simple. We say that the probability of A given that we know B happened is still just the probability of A. We don't, we don't update our probability by knowing that B happened. Knowing that B happened doesn't change the probability, uh, of A happening. And similarly, the probability of B happening given that we know information about A is still just the probability of B. Really, really simple, but this is actually quite profound. This is, uh, one of the most useful ideas. You're going to hear this all the time. So you hear about IID events; that's, um, you know, identical independently distributed events. So if I flip a coin five times—you know, I'm not going to do it five times—if I flip a coin five times, those are independent trials. The second coin flip doesn't depend on the first; the third doesn't depend on the first two. You know, approximately good.
And so if this is true, there's a really, really, really important, uh, consequence of this, and that consequence is that the probability of A and B happening is equal to the probability of A times the probability of B times the probability of B. And this is really, really easy to see. You can basically just use, um, uh, the conditional probability formula. So I'm going to write this down. So we know that the probability of A is equal to the probability of A given B, and we know that the probability of A given B, using our kind of traditional conditional formula, is the probability of A and B divided by the probability of B. And so now I can just multiply both sides by the probability of B, and I get probability of A and B is probability of A times the probability of B. So that is really, really simple. Um, this is just kind of the identity of, of conditional probability, and if this probability of A given B is just probability of A, then I get this super, super useful—in fact, I'm going to box it because this is a really, uh, important, uh, formula here. So this is what I mean: if I have two events that are independent, then the pro—then knowing something about one of those events doesn't change the probability of the other event, and the probability of both events happening is the multiplicative probability of each event happening. Okay.
And this is really, really easy to picture. Um, the example I like to think of is I always think about cards. Maybe I'm, maybe I've got a problem, but let's talk about cards. Let's say that I have, um, I'm going to draw a picture of my sample space Omega, and let's say I have a deck of 52 cards, and I'm going to draw a card off the top, and it could be any one of four suits; it could be a club, a heart, a diamond, or a spade. So it could be, um, maybe I'll actually draw it; it could be a diamond, it could be a heart, it could be a club, and it could be a—how do I draw a spade?—a spade. Okay. So it could be one of those four, and then, uh, it could also be one of 13, um, one of 13, uh, uh, numbers; it could be a 1, 2, 3, 4 all the way up to 10, Jack, queen, and king. Okay. So there's 13 possible, uh, values, and there's four possible suits; 4 * 13 is 52; there's 52 cards. So this would be, you know, one, two, three, dot, dot, dot, dot, dot, 10, Jack, queen, king. Okay. Now event A could be, uh, that my card is a spade, and event B could be that my card is a queen. Okay. Um, and if I know that my card is a queen, it tells me nothing about whether or not my card was a spade. If I know that my card was a spade, it tells me nothing about my card being a queen; they're completely independent. Um, and the probability of my card being a Queen of Spades is the probability of my card being a queen times the probability of my card being a spade. So this probability of B is, uh, one in 13, because there's four queens in the deck; the probability of it being a spade is one in four because one quarter of the cards are spades. And so the probability of my card being the Queen of Spades, A and B, is the probability of A times the probability of B, which is one in 52; there's only one card out of all 52 that's the Queen of Spades. Really, really simple, whole idea of independence, and I like this because you can actually draw a really cool picture. So the set of all spades is this quarter down here, and the set of all queens is this little 13th of the vertical up here. Okay. So this is, I guess, uh, set A, and this is set B, and you'll notice that by some kind of a symmetry I've actually drawn this so that they're kind of orthogonal. If I restrict myself to just being spades, I still have a 1 in 3 chance of being a queen; my probability didn't change. And if I restrict myself to being—knowing that I had queens—I still have a one in four chance of getting a spade; my probability of spades didn't change. So it's a really, really simple idea to illustrate this notion of independence. Okay, good. And you can do this with coin flips; you can do this with, um, all kinds of things. Um, it's useful for things like, uh, redundancy and reliability. If you think about how many, many backup hard drives do I need if I assume that my hard drive has a 1% chance of failure, um, you know, if I have three drives, then there's a one in a million chance of failure; it's it's way, way better. That's assuming independence. If my, you know, uh, surge protector fails and all three drives go out at once, then I'm hosed. So independence is a, is, is actually an outfall of physics in a sense. So some systems you think are independent and they're not; some systems are actually independent like this one, and something you need to think about.
So last idea here I want to talk about is, uh, reliability, because again, when we think about those hard drives and reliability, if you can really assume that your parts are independent, you can start coming up with some pretty interesting, uh, notions here of, of reliability. So let's say I have, um, n components in series, uh, n components in series. I'm just going to draw my n components here: dot, dot, dot. Okay. This is component one, component two, component three, and let's say all of these components have a probability P of failure. So probability of, uh, failure of one component is P. Okay. Probability of one component failing is P. Let's say it's really small; let's say it's a 0.05; there's a 5% chance that one of these components fails. Okay. What is the probability that n components in series could fail? So if any one of these fails, the whole chain is broken. Imagine your string of Christmas lights; one of these fails, all of them fail. What's the probability, um, of this failing? So that's actually really hard to compute. So this is a probability of a single element failing is, is P. What's the probability of the total series failure? Of series failure? Um, the probability of series failure—this is very hard to compute; this is hard to compute—again, just like the birthday problem. I could compute the chances of exactly one failure, but what if I have two failures? What if I have three failures? Being sure that you're counting this and you're not double counting is actually really hard to compute this, but it's easy to compute the opposite. So this is equal to one minus the probability of, of success, and that's pretty easy to compute. The probability of all—the only way that this can actually not fail is if every single element doesn't fail. And so that is one minus the probability of all of these succeeding is 1 minus P to the power of n; all n components have to not fail, and this is the probability of that happening. Okay. Now if I have n equal 10 and P = 0.05, you see you can compute this; you know, just go type this into, um, you know, your Jupiter notebook, and you'll find that the probability of success, um, of this thing is only about 60%, and the probability of, of this failing is 40%. So even if your components are relatively reliable, putting them in series makes it way less reliable. So do you remember, uh, maybe some of you have used RAID array hard drives back in the olden days before cloud storage, um, someone else is doing RAID for you; you could do different types of redundancy; you could make it faster, um, but it was less reliable, or you could make it, you know, you could decrease your total capacity or make it slower, but make it much more reliable. So series typically makes things less reliable; parallel, the opposite. So if I stack my components in parallel now: dot, dot, dot. The idea here is the only way for this to fail is now for all of my elements to fail. If I have a single failure in parallel, I can just route around it. So the probability of failure, uh—does anyone listen to Uncle Roger? I can't hear failure in my head anymore—the probability of failure, uh, now is P to the N. This is way, way, way more unlikely in series—in parallel than in series—it's about, uh, 10 to the minus, uh, 13 here. The probability of this failing with n components—with 10 components in parallel—is now 10 to the minus 13. This has a high chance of failure; this has an almost astronomically low chance of failure. Independence: super, uh, important idea. It allows you to multiply probabilities; it says that information about event B doesn't tell you anything about event A and vice versa, and you can—we're going to use it all the time when we think about, you know, independent coin flips, uh, expected part failures of complicated processes, and and how—why we need redundancy and how to estimate redundancy. And when you think about redundancy, you had better be thinking about what it actually means to be independent. Um, you know, if I get a lightning strike, are all of my hard drives really independent? Probably not. Okay, thank you.