Flash and JavaScript are required for this feature.
Download the video from Internet Archive.
Instructor: Vina Nguyen
Lecture Topics:
Review of Last Class, Conditional Independence
» Download English-US transcript (PDF)
The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
VINA NGUYEN: So we'll get started. As always, review. Can anyone tell me what Bayes' rule is? Intuitively or mathematically. OK. So Bayes' rule rule is kind of like the reverse of what we're given. So if you want the probability of some event given B, then you have to do probability of A times the probability of B given A over probability of Ai given probability of B given Ai, all the way to probability of A n given probability of B.
So it's probably a little hard to remember the formula. But if you think visually, it's like if you have a sample space, and say you have some random event, B, but you don't actually know what this is. All you know is that you have A1 maybe, and you know the probability of A1 and B. And then you might have A2, A3, et cetera, until A n. So n could be the number of partitions.
So Bayes' rule is figuring out probability of A given B when you're given these chunks. Does that makes sense? Do you guys need more time? No? OK. So what's the total probability theorem? I basically just wrote it, so you can point it out if you want. Yeah.
AUDIENCE: Is it the bit on the bottom?
VINA NGUYEN: Yep. So is this the total probability theorem, which gives us Pv. So it's called total because you have to have every one of these A's that have B in it. So if you know A1, A2, and A3, but you don't know this part, that's not complete, not total, so it won't work. So you have to make sure you have all of that.
Is this too high? OK. Does that makes sense? OK. And if I told you that A is independent from B, what does that mean intuitively?
AUDIENCE: P of A given B equals P of A. They're not related at all. So if you know one, it doesn't help to change anything here [INAUDIBLE].
VINA NGUYEN: So B basically doesn't give you any information about A. No new information about A. Yep. And that can be reversed. So A is independent from B. B is also independent from A. I won't write that because I figure that makes sense.
So how do we test for independence? What's that equation? You just said it, but anyone else? Yep.
AUDIENCE: We put the total probability theorem on top. Then you put P parenthesis A parenthesis.
VINA NGUYEN: Right. Can you say that again?
AUDIENCE: You put the total probability theorem on the top of the bracket.
VINA NGUYEN: You mean like this?
AUDIENCE: Yeah.
VINA NGUYEN: What do I put here?
AUDIENCE: A, B.
VINA NGUYEN: A something B.
AUDIENCE: A plus B.
VINA NGUYEN: Union. Yeah. Yep.
AUDIENCE: Over-- draw a line-- P parenthesis A parenthesis.
VINA NGUYEN: This?
AUDIENCE: Yeah.
VINA NGUYEN: Anyone have another answer?
AUDIENCE: Isn't it the probability of A is equal to the probability of A intercept B over the probability of A?
VINA NGUYEN: Yeah. OK. So like, for you, you got part of it if you switch these. But because this itself is just an expression, you're not saying equal to what. So you can't really evaluate it while giving some kind of resolve. So if you're just giving y, that doesn't really tell you if that's a test or not. But this does because this has to hold true.
So this is right. Does everyone see why that is? So we usually actually write it with Pv over here because we don't like dividing by 0. So if we just move it over here, that takes care of it. So you'll get PA, PB. So we like to write it like that, but they are essentially the same thing. Yeah. Just because the divide 0 thing. So here you can put in 0. Sorry.
Does that make sense? OK. All right. So we actually didn't get to conditional independence last class, so we're going to go over that now. Where's that eraser? Do you guys need this?
So we have this for independence. And if we want to do conditional independence, we just do given C, given C, because that's what conditional means. Given something. And you can also see it the second way, which means that even if you're given B and C, the addition of B doesn't mean anything.
So how do we get from the first to the bottom? So you have-- is this OK to read? Feels kind of messy. OK. So another way of writing this is if we have A union B union C over PC. It's kind of like we did with conditional probability.
And then you're given in that equation that this is PA given C, PB given C equals. And the way we get from here is you've seen the multiplication rule. So you have probability of C, probability of B given C, probability of A given B and C. Does this make sense? Over probability of C.
So if you don't fully understand that, you can think of it as C-- a tree-- and then B, not B, and then A. So that top part is just probability of C, probability of B given C happened, probability of A given B and C happened. OK. Does everyone see that?
So then these two cancel, these two cancel, and then you're left with this part, which is essentially that second line. So you can use either of those, depending on what you're given in the problem.
Does that make sense, how we got from one to the other? I'll give you more time to write it, OK? So we have an example. So we have two coins that are blue and red but they're biased. So the blue coin has heads 99% of the time, and then the red coin lands head only 1% of the time.
So what we're doing is first we're choosing one of these coins at random, and then we're flipping it twice. Does everyone understand that problem space? OK. So the way we broke it down is that event one, H1 means the first toss is a head, and the second event is H2, which means the second toss is a head.
So the question pretty much is, are they independent of each other, these tosses? The thing is that they're independent of each other conditionally. So if you already know what coin you're given, then each toss is independent. So I'm going to prove that.
So we're going to assume B, which means we got the blue coin. So are they conditionally independent? You have that equation again. You have P H1 union P H2 given B. Is that equal to probability of heads? The first toss is a head given B plus probability of second toss heads given B.
So that's what we're trying to answer right now. Does everyone understand how this is conditional? You're given that you have B. So it is kind of obvious. If you're given B, then it has 99 times 99 chance of being heads. And then this is 0.99 times 0.99. So obviously, they are equal. So they are conditionally independent.
But the next question is if you don't know which coin you have, if the tosses are still independent. So that's answering the question. So this was our original test for independence. There's nothing given, so we don't know what coin we have. So now we're trying to answer, are these tosses independent without given information?
Can you see the difference? So how do we calculate this? Probability of the first toss being a head. So we're going to do this. You've see the total probability theorem, right? So we have probability of first head given that it's blue plus the probability that we got a blue coin plus probability that we have the head given that it's not blue.