3

My question is related to the second part of this question here.

Assume we consider an example $\mathbb{P}(A,B|C) = \mathbb{P}(A|C) \mathbb{P}(B|C)$, this means if the event $C$ occurs then the events $A$ and $B$ are independent.

An appropriate representation should be something like this below:

enter image description here

However I am seeing many text books and other graph related study materials depicting this relation as:

enter image description here

That is just not correct. I am curious to find out what the community thinks.

Where am I going wrong ?

Chill2Macht
  • 6,249
  • 3
    I suspect that (as always) the devil is in the details. So without a concrete reference for these "many text books and other graph related study materials" we cannot comment. Text books contain mistakes, but more often text books are misunderstood. Without the context we have no way of distinguishing between these two possibilities. – Maarten Buis Dec 12 '15 at 08:44
  • @MaartenBuis, I agree , if you go on google and search using these words conditional independence graph , you will find a lot of materials and they all mention the same thing...for example, slide 3 in this url https://www.cl.cam.ac.uk/teaching/1011/L101/ml4lp-lect3.pdf represents conditional independence in way that dosent make sense and not intuitive to understand. – King Frazier Dec 12 '15 at 14:56

1 Answers1

1

1. To make sure we are on the same page, in what follows I use this definition of (undirected) conditional independence graph which corresponds (at least roughly) to Markov random fields:

Definition: The conditional independence graph of $X$ is the undirected graph $G=(K,E)$ where $K=\{ 1, 2, \dots, k \}$ and $(i,j)$ is not in the edge set if and only if $X_i \perp \! \! \! \perp X_j | X_{K \setminus \{i,j\}}$. (Where $X_{K \setminus \{i,j\}}$ denotes the vector of all of the random variables except for $X_i$ and $X_j$.)

From p. 60 of Whittaker, Graphical Models in Applied Mathematical Multivariate Statistics (1990).

Note: The above definition is regarding random variables, rather than for events (as you mention in your question). The definitions are related to one another of course, but subtle differences can occur. In order to more closely follow the logic of the argument given in the notes you linked to, where $A$, $B$, and $C$, represent random variables, rather than events, in everything that follows I will also consider $A$, $B$, and $C$ to represent random variables, rather than events. I don't think though that the arguments below would be substantially (if at all) affected if one considered $A, B$, and $C$ to be events rather than random variables; the logic should be exactly the same in this case.

2. Let's compare the two examples given and see whether or not they are the same.

In the first example you give, we have that: $$\mathbb{P}(A,B|C)= \mathbb{P}(A|C)\mathbb{P}(B|C) \,, \quad \text{i.e.} \quad A \perp \! \! \! \perp B | C \,.$$

Using the definition of conditional independence graph given above, this means that the only missing edge should be that between $A$ and $B$, since they are conditionally independent given all of the remaining random variables (in this case just $C$).

And indeed, that is exactly the graph you have, so you are 100% correct that.

Regarding the second example, from slide 3 of the lecture you linked to, there are actually two distinct/different situations being described.

The first situation described in that slide is the joint distribution of any three arbitrary random variables, $\mathbb{P}(A,B,C)$. By the definition of conditional probability, this can always be re-written as: $$\mathbb{P}(A,B,C) = \mathbb{P}(C|A,B)\mathbb{P}(A,B)= \mathbb{P}(C|A,B)\mathbb{P}(B|A)\mathbb{P}(A) \,. $$ (Koller and Friedman call this fact the "chain rule of conditional probabilities". Specifically, see equations 2.2 and 2.3 on p. 18 in their book Probabilistic Graphical Models.)

The second situation described in that slide is when $A$ is conditionally independent of $C$ given $B$, $$\mathbb{P}(A,C|B)=\mathbb{P}(A|B)\mathbb{P}(C|B) \,, \quad \text{i.e.} \quad A \perp \! \! \! \perp C |B \,. $$

As a consequence of this, in the second situation one can write that: $$\mathbb{P}(C|A,B)= \frac{\mathbb{P}(A,B,C)}{\mathbb{P}(A,B)} = \frac{\mathbb{P}(A,C|B)\mathbb{P}(B)}{\mathbb{P}(A,B)} = \frac{\mathbb{P}(A,C|B)}{\mathbb{P}(A|B)}= \frac{\mathbb{P}(A|B)\mathbb{P}(C|B)}{\mathbb{P}(A|B)} = \mathbb{P}(C|B) \,,$$ where we only applied the conditional independence relationship expressed above in the last equality (everything else was just applying the definition of conditional probability).

So substituting this into the general case result given above ("chain rule of conditional probabilities") we get that, in the special case when $A$ is conditionally independent of $C$ given $B$: $$\mathbb{P}(A,B,C) = \mathbb{P}(C|B)\mathbb{P}(B|A)\mathbb{P}(A) \,, $$ which is exactly what the lecture slide says.

Then, using both your and my definition of conditional independence graph, the only missing edge should be between $A$ and $C$, because they are conditionally independent given all of the remaining random variables (which in this case is just $B$).

Conclusion: Both the first and second example in the question are correct; they have different conditional independence assumptions, and therefore they have different conditional independence graphs.

Chill2Macht
  • 6,249