Probabilistic graphical models (PGMs) are a rich framework for encoding probability distributions over complex domains: joint (multivariate) distributions over large numbers of random variables that interact with each other. These representations sit at the intersection of statistics and computer science, relying on concepts from probability theory, graph algorithms, machine learning, and more. They are the basis for the state-of-the-art methods in a wide variety of applications, such as medical diagnosis, image understanding, speech recognition, natural language processing, and many, many more. They are also a foundational tool in formulating many machine learning problems. This course is the first in a sequence of three. It describes the two basic PGM representations: Bayesian Networks, which rely on a directed graph; and Markov networks, which use an undirected graph. The course discusses both the theoretical properties of these representations as well as their use in practice. The (highly recommended) honors track contains several hands-on assignments on how to represent some real-world problems. The course also presents some important extensions beyond the basic PGM representation, which allow more complex models to be encoded compactly.
Reasoning Patterns
Loading...
來自 Stanford University 的課程
Probabilistic Graphical Models 1: Representation
820 個評分
從本節課中
Bayesian Network (Directed Models)
In this module, we define the Bayesian network representation and its semantics. We also analyze the relationship between the graph structure and the independence properties of a distribution represented over that graph. Finally, we give some practical tips on how to model a real-world situation as a Bayesian network.
與講師見面
Daphne KollerProfessor
School of Engineering
Having defined the Bayesian network, let's look at some of the reasoning
patterns of work allows us to perform. So let's go back to our good old student
network with the following CPDs. We've already seen those.
So I'm not going to dwell on it. And let's look at some of the
probabilities that one would get if you took this busy network, produced the
joined distribution using the chain rule for busy network.
And now compute it say the values of different marginal probabilities.
So for example now we're asking what is the probability of getting a strong
letter, and we're not going to go through the calculation, because wanting to be
tedious to sum up all these numbers and I can just tell you that the probability of
the the of L1, is 0.5 but we can do more interesting queries.
So we can now condition on one variable. Remember we talked about conditioning of
probability distribution. And and ask how that changes this
probability. So for example, if we're going to
condition on low intelligence, we're going to use red to denote the false
value. And it's going to point to turn out that
the probability not surprisingly goes down.
It goes down to 0.39 because if the intelligence goes down, the probability
of getting a good grade goes down and so does the probability of getting a strong
letter. So this is an example of causal
reasoning, because because, intuitively the reasoning goes, in the causal
direction from top to bottom. We could also make things more
interesting. So we can ask what happens if we make the
difficulty of the course low and in this case, we have the probability of L1,
given i0 and b0. And what you expect the probabilities to
do, well, if it's an easy course, one would expect the grade to go up.
And sure enough the probability goes back up and we're back to 50/50, more or less.
Okay, so this is another example, of [UNKNOWN] in this case with a little
more, evidence. You can also do evedentual reasoning.
Evedentual goes from the bottom up. So we can in this case condition on the
grade and ask what happens to the probability of, of variables that are
parents or, or general ancestors of the grade.
So does it matter that this poor student takes the class and he gets a C.
Initially the probability that the class was difficult is 0.4 and the probability
that the student was intelligent is 0.3. But now with this additional evidence,
again this is not surprising, the probability that the, that the student is
intelligent goes down a fair amount but the other alternative hypothesis, that
the class is difficult also the probability of that goes up as well.
Hm. Now however there is an interesting type
or reasoning that is not quite as standard.
And that is reasoning that is called inter-causal because effectively it's
flow of information between two causes of a. of a single effect.
So remember we had the we're going to continue with the scenario before where
our poor student gets a C but now I'm going to tell you, wait a minute.
This class really is difficult so I'm going to condition on on v1 and notice
that the propability that the student his intelligence has gone up, it went up from
0.08 to 0.11 so that's not a huge increse and as you'll, see when you play around
with Bayesian networks, that often the changes in probability are somewhat
subtle. and the reason is that, you have to, I
mean, even in a hard class if you go back and look at the CPD it's kind of hard to
get a C, according to this model. which is that the students get a B.
and so now, have that the probability of high intelligence still goes down, it
goes down from 0.3 to 0.175 but now if I tell you the class is hard, the
probability goes up, in fact it goes up even higher than this, okay?
So, this is an illustration where this, where this intercausal reasoning can
actually make a fairly significant difference in the probabilities.
So intercausal reasoning is a little hard to understand, I mean she's a little bit
mysterious because after all, these are, I mean look at these you look at
difficulty you look at intelligence there's no edge between them how, how
would how would one cause affect another. So let's drill down into a concrete
scenario which is this one and just to sort of really understand the mechanism.
So, this is the most sort of purest form of intercausal reasoning.
Here we have two random variables x and, x1 and x2.
We're going to assume that they're distributed uniformally so each of them
is one with probability 50% and zero probability 50%.
And we have one effect one joined effect which is simply the deterministic oar of
those two parents. And in general we have the terministic
variable we're going to denote with these with these double lines.
So, in this case, there's only four assignments that have nonzero
probability, because, the value of Y is completely determined from, by the values
of X1 and X2. And so we have we have these four
distributions over here and now, I'm going to condition on the evidence y = 1.
Now let's look at what happened. Before I conditioned on this evidence,
the X2 were independent of each other, right?
I mean, look at this. They're independent of each other.
What happens when I condition on y = 1? Well we talked about conditioning.
This one goes away, and we have 0.33, 0.33.
0.33 or rather one third, one third. Okay.
In this probability distribution x1 and x2 are no longer independent of each
other. Okay.
Why is that? Because if I now condition on say x1
equals 0, then okay, so actually before we do that,
so that, in, in this probability distribution the probability of x1 equals
1 is equal to two-thirds and the probability of x, two equals one.
Is also equal to two thirds. I think,
and now if I condition on x1 = 1. So now, we're going to condition on,
conditions x11. = 1.
So that means we're going to remove this line.
And all of a sudden, the probability of x21 = 1 given x11 = 1 is back to being
50%.. So with 60% of 4, it went up to
two-thirds and then if we condition on x11, = 1, it goes back to 50%..
And the reason for this is the following, think about it intuitively.
If I know the y1 = 1 there's two possible things that could have made y1, = 1.
Either x1 was 1 or x2 was 1. If I've told you that X1 was one, I've
completely explained away the evidence that Y equals one.
I've given you a complete explanation of what happened, and so now I just want to
go back the way it was before, because there is no longer anything to suggest
that that it should be, anything other than 50/50.
So this particular type of [INAUDIBLE] because it's so common.
It's called explaining away. And it's when one cause explains a way
reasons that made me suspect a different cause.
And if you think about it it's something that people do all the time, when they're
reasoning about for example in the medical setting you, you're very sick,
you think you're very worried you have you don't know if you have the swine flu,
you go to the doctor the doctor says don't worry it's just the common cold.
You don't know if, that you don't have the swine flu, but because you have
explained away your symptoms you're not worried as much anymore.
Okay. Finally.
Lets look, lets go back to our example, and look at an interesting reasoning
pattern that is not that, that involves even longer sort of paths in the graph.
So lets imagine that we have the student, the student got a C.
But now we have this additional piece of information that the student actually
aced the SAT. So hopefully what happens there.
Remember that when we just had the evidence regarding the grade we had the,
the probability that the students being intelligence was only 0.08.
But now we have this additional conflicting piece of evidence, and all of
a sudden the probability went up very dramatically to 0.58, okay?
What do you think is going to happen to difficulty?
So, now it's explaining a way in action going in a different direction right?
Because if it's not the fact that student, I mean if the student didn't get
a C because he wasn't very bright, probably, the reason is that the class is
very difficult, and so that probability goes up.
And so we have effectively and we're going to talk about this an inference
that folds like that.
