Transcription
Welcome back. So we introduced this notion of conditional probability: how you can compute the probability of A if you know that some other event B happened. You can compute the probability of A given that event happening as the probability that both happen divided by the probability of B happening. This makes a lot of sense, and it's one of the most useful kinds of ideas in probability: to take partial information of event B and use that to update your probability of event A happening. We also know that sometimes we're going to do the inverse of that; we might want to do the inverse or inference problem of estimating the probability of B happening given A. That's using Bayes' theorem; we'll talk about that in a minute. But there's an important corollary of this idea of conditional probability that's really, really useful.
The first corollary is that the probability of A and B happening—the probability of A happening and B happening—is equal to—this is so trivial I'm literally just going to multiply this over here—it's the probability of A given B times the probability of B. That's it. This is called the multiplication law, and it's so simple but it's going to be really, really useful. We're going to use it all the time, and so I just want you to remember that this is kind of an outfall of conditional probability. The probability of A and B happening is the probability of A given B times the probability of B. So you can write it this way; you can write it this way. These are both related to this notion of conditional probability, and I want you to really understand that. That's why I had a lecture on set theory; I want you to have this gut feeling for this symbol. A and B is the intersection of events A and B both happening; so both A and B have to happen. And this funny notation, vertical bar, probability of A given B, means that if we know that B happened, we can zoom in and then compute the probability of A after the fact that we know B definitely happened.
Okay. And the second kind of big corollary I want to point out here is what's known as the law of total probability. Again, very, very useful. Law of total probability. One of the things I think is interesting about probability is that if it's explained simply, it almost seems trivial or obvious or so intuitive; it's like hard to even understand how it could not be true. And so this is going to seem pretty obvious, I think. So what we're going to do is, just like we broke our set Omega of all things that could possibly happen into sets A and B, let's say now we break our set Omega into a bunch of disjoint sets B1, B2, B3, B4… dot dot dot all the way; you know, there's a bunch of these B's. So we're going to say that Omega is equal to the union of a bunch of disjoint B's from I equal 1 to n. So I'm going to have n of these disjoint sets. We're just going to say this is, you know, B1, B2, B3… Bn. So there's n of these sets, and we're going to say that if I, if I, you know, union all of them, if I take the, if I combine all of them with this union, I'll get my, my big set Omega. Okay. So this could be: Omega is all of the, you know, I, I have a deck of cards and I flip one card; there's 52 cards that could be, and these B's could be: it's a spade, it's a club, it's a diamond, it's a heart—four disjoint sets. Again, disjoint means that Bi intersection with Bj is the empty set. If it's a heart, it can't be a spade; if it's a club, it can't be a heart, and so on and so forth, if I does not equal J. So this is really mathematical, but what I mean is that these sets don't have an overlap; there's not an overlap region here because there is no, you know, no intersection between these two sets; they're disjoint. Okay. So we call this disjoint.
Okay. So if we break our pro—our—our sample space into a bunch of disjoint sets, then the law of total probability says: let's say that I have some event A here; this is A. Then the probability of A occurring can be—you can basically add up all of these kind of conditional probabilities over every single one of these disjoint sets. So I'm just going to write it out, and then we'll talk through it. It's the sum from I equals 1 to n of the probability of A given that I'm in subset I times the probability of subset I happening. Okay. This is the law of total probability. And essentially what it says is that the probability of A happening, I can compute—I can go into each of these disjoint subsets and compute: what's the probability of A given that I'm in that subset times the probability of that subset occurring? Okay. So, for example, let's say: what's the probability of my card being a red card? Okay, like there's cl—there there's diamonds and hearts; those are the red cards, and then there's clubs and spades. So, you know, let's say—let's make this really, really simple. So there's B1, B2, B3, B4. B1, B2, B3, B4. And let's say that these are, you know, hearts, diamonds, clubs, and spades. And let's say that A is that, you know, my card is red; so only hearts and diamonds. So A is that my, my card is red. So the probability of A—one way you could—I mean, this is not a useful way to compute this probability, but sometimes this is really useful—is I would go across all of these B's and I would say: what's the probability of my card being red given that it's a heart? That's 1 * 1/4 because there's 1/4 of the cards are hearts. Plus the probability of my card being red given that it's a diamond; that's 1 * 1/4 because 1/4 of the cards are diamonds. Plus the probability of my card being red if it's a club; that's 0 * 1/4 because a fourth are clubs. Plus the probability that my card is red given that it's a spade, which is 0 * 1/4 because 1/4 of the cards are spades. This is just another way of saying something has to happen; like the probabilities all have to add up to one. Okay. And this is a way of saying: if, if, if event A happens and I have all of my sample space divided into these disjoint sets B, it has to happen; it has to overlap with some of these subsets, and so the total probabilities add up this way. Okay. Very, very useful. We're going to end up using these a lot: the multiplication law and the law of total probability. And they're kind of common sense things that just have to be true, but I wanted to show you how to think about these and how to work with these before we just start using them. Okay. So probability A happening is the probability that A happens given, you know, event B times the probability of B, and you add up all of those disjoint B's and you get this total probability. Okay. I want you to think about this and really convince yourself that this is true.
One very useful disjoint set—let me draw this one more time—one really, really useful disjoint set is: let's say I have B; then I can divide my sample space—the, the set of all things that can happen—into B and not B, the complement of B. So that's what this notation B superscript C means. I can say that Omega is the union—maybe I'll write this a little bit differently—it is B union the complement of B. It's, it's the union: either B happens or not B happens; that's for sure true. Okay. So, you know, if I, if I roll a dice, B could be that I rolled a one; not B is anything else happening, and that has to—that union has to add up to the total set of things that can happen. So in the law of total probability, often I'll write this as: probability of A equals the probability of A given B times the probability of B plus the probability of A given B didn't happen times the probability that B didn't happen. This is an outfall of the law of total probability. This is a very common, useful disjoint covering of your sample space Omega. Okay. That's enough. Multiplication law, law of total probability. Next we're going to get into Bayes' theorem and how to solve for these inverse problems, but we're going to need these ideas here. Thank you.