If 'B is more likely given A', then 'A is more likely given B'

Question

I am trying to get a clearer intuition behind: "If $A$ makes $B$ more likely then $B$ makes $A$ more likely" i.e

Let $n(S)$ denote the size of the space in which $A$ and $B$ are, then

Claim: $P(B|A)>P(B)$ so $n(AB)/n(A) > n(B)/n(S)$

so $n(AB)/n(B) > n(A)/n(S)$

which is $P(A|B)>P(A)$

I understand the math, but why does this make intuitive sense?

Answer 1

I think another mathematical way of putting it may help. Consider the claim in the context of Bayes' rule:

Claim: if $P(B|A)>P(B)$ then $P(A|B) > P(A)$

Bayes' rule: $$ P(A \mid B) = \frac{P(B \mid A) \, P(A)}{P(B)} $$

assuming $P(B)$ nonzero. Thus

$$\frac{P(A|B)}{P(A)} = \frac{P(B|A)}{P(B)}$$

If $P(B|A)>P(B)$, then $\frac{P(B|A)}{P(B)} > 1$.

Then $\frac{P(A|B)}{P(A)} > 1$, and so $P(A|B) > P(A)$.

This proves the claim and an even stronger conclusion - that the respective proportions of the likelihoods must be equal.

Answer 2

Well, I don't like the word "makes" in the question. That implies some sort of causality and causality usually doesn't reverse.

But you asked for intuition. So, I'd think about some examples, because that seems to spark intuition. Choose one you like:

If a person is a woman, it is more likely that the person voted for a Democrat.
If a person voted for a Democrat, it is more likely that the person is a woman.

If a man is a professional basketball center, it is more likely that he is over 2 meters tall.
If a man is over 2 meters tall, it is more likely that he is a basketball center.

If it is over 40 degrees Celsius, it is more likely that there will be a blackout.
If there has been a blackout, it is more likely that it is over 40 degrees.

And so on.

Answer 3

By way of intuition, real world examples such as Peter Flom gives are most helpful for some people. The other thing that commonly helps people is pictures. So, to cover most bases, let's have some pictures.

What we have here are two very basic diagrams showing probabilities. The first shows two independent predicates I'll call Red and Plain. It is clear that they are independent because the lines line up. The proportion of plain area that is red is the same as the proportion of stripy area that is red and is also the same as the total proportion that is red.

In the second image, we have non-independent distributions. Specifically, we have expanded some of the plain red area into the stripy area without changing the fact that it is red. Clearly then, being red makes being plain more likely.

Meanwhile, have a look at the plain side of that image. Clearly the proportion of the plain region that is red is greater than the proportion of the whole image that is red. That is because the plain region has been given a bunch more area and all of it is red.

So, red makes plain more likely, and plain makes red more likely.

What's actually happening here? A is evidence for B (that is, A makes B more likely) when the area that contains both A and B is bigger than would be predicted if they were independent. Because the intersection between A and B is the same as the intersection between B and A, that also implies that B is evidence for A.

One note of caution: although the argument above seems very symmetrical, it may not be the case that the strength of evidence in both directions is equal. For example, consider this third image.
Here the same thing has happened: plain red has eaten up territory previously belonging to stripy red. In fact, it has completely finished the job!

Note that the point being red outright guarantees plainness because there are no stripy red regions left. However a point being plain has not guaranteed redness, because there are still green regions left. Nevertheless, a point in the box being plain increases the chance that it is red, and a point being red increases the chance that it is plain. Both directions imply more likely, just not by the same amount.

Answer 4

To add on the answer by @Dasherman: What can it mean to say that two events are related, or maybe associated or correlated? Maybe we could for a definition compare the joint probability (Assuming $\DeclareMathOperator{\P}{\mathbb{P}} \P(A)>0, \P(B)>0$): $$ \eta(A,B)=\frac{\P(A \cap B)}{\P(A) \P(B)} $$ so if $\eta$ is larger than one, $A$ and $B$ occurs together more often than under independence. Then we can say that $A$ and $B$ are positively related.

But now, using the definition of conditional probability, $\frac{\P(A \cap B)}{\P(A) \P(B)}>1$ is an easy consequence of $\P(B \mid A) > \P(B)$. But $\frac{\P(A \cap B)}{\P(A) \P(B)}$ is completely symmetric in $A$ and $B$ (interchanging all occurrences of the symbol $A$ with $B$ and vice versa) leaves the same formulas, so is also equivalent with $\P(A \mid B) > \P(A)$. That gives the result. So the intuition you ask for is that $\eta(A,B)$ is symmetric in $A$ and $B$.

The answer by @gunes gave a practical example, and it is easy to make others the same way.

Answer 5

If A makes B more likely, A has crucial information that B can infer about itself. Despite the fact that it might not contribute the same amount, that information is not lost the other way around. Eventually, we have two events that their occurrence support each other. I can’t seem to imagine a scenario where occurrence of A increases the likelihood of B, and occurrence of B decreases the likelihood of A. For example, if it rains, the floor will be wet with high probability, and if the floor is wet, it doesn’t mean that it rained but it doesn’t decrease the chances.

Answer 6

If A makes B more likely, this means the events are somehow related. This relation works both ways.

If A makes B more likely, this means that A and B tend to happen together. This then means that B also makes A more likely.

Answer 7

You can make the math more intuitive by imagining a contingency table.

$\begin{array}{cc} \begin{array}{cc|cc} && A & \lnot A & \\ &a+b+c+d & a+c & b+d \\\hline B& a+b& a & b \\ \lnot B & c+d& c & d \\ \end{array} \end{array}$

• When $A$ and $B$ are independent then the joint probabilities are products of the marginal probabilities $$\begin{array}{cc} \begin{array}{cc|cc} && A & \lnot A & \\ &1 & x & 1-x \\\hline B& y& a=xy & b=(1-x)y \\ \lnot B & 1-y& c=x (1-y) & d=(1-x)(1-y)\\ \end{array} \end{array}$$ In such case you would have similar marginal and conditional probabilities, e.g. $P (A) = P (A|B) $ and $P (B)=P (B|A) $.

• When there is no independence then you could see this as leaving the parameters $a,b,c,d $ the same (as products of the margins) but with just an adjustment by $\pm z $ $$\begin{array}{cc} \begin{array}{cc|cc} && A & \lnot A & \\ &1 & x & 1-x \\\hline B& y& a+z & b-z \\ \lnot B & 1-y& c-z & d+z\\ \end{array} \end{array}$$

  You could see this $z$ as breaking the equality of the marginal and conditional probabilities or breaking the relationship for the joint probabilities being products of the marginal probabilities.

  Now, from this point of view (of breaking these equalities) you can see that this breaking happens in two ways both for $P(A|B) \neq P(A)$ and $P(B|A) \neq P(B)$. And the inequality will be for both cases $>$ when $z$ is positive and $<$ when $z $ is negative.

So you could see the connection $P(A|B) > P(A)$ then $P(B|A) > P(B)$ via the joint probability $P(B,A) > P (A) P (B) $.

If A and B often happen together (joint probability is higher then product of marginal probabilities) then observing the one will make the (conditional) probability of the other higher.

Answer 8

Suppose we denote the posterior-to-prior probability ratio of an event as:

$$\Delta(A|B) \equiv \frac{\mathbb{P}(A|B)}{\mathbb{P}(A)}$$

Then an alternative expression of Bayes' theorem (see this related post) is:

$$\Delta(A|B) = \frac{\mathbb{P}(A|B)}{\mathbb{P}(A)} = \frac{\mathbb{P}(A \cap B)}{\mathbb{P}(A) \mathbb{P}(B)} = \frac{\mathbb{P}(B|A)}{\mathbb{P}(B)} = \Delta(B|A).$$

The posterior-to-prior probability ratio tells us whether the argument event is made more or less likely by the occurrence of the conditioning event (and how much more or less likely). The above form of Bayes' theorem shows use that posterior-to-prior probability ratio is symmetric in the variables.$^\dagger$ For example, if observing $B$ makes $A$ more likely than it was a priori, then observing $A$ makes $B$ more likely than it was a priori.

---

$^\dagger$ Note that this is a probability rule, and so it should not be interpreted causally. This symmetry is true in a probabilistic sense for passive observation ---however, it is not true if you intervene in the system to change $A$ or $B$. In that latter case you would need to use causal operations (e.g., the $\text{do}$ operator) to find the effect of the change in the conditioning variable.

Answer 9

You are told that Sam is a woman and Kim is a man, and one of the two wears make-up and the other does not. Who of them would you guess wears make-up?

You are told that Sam wears make-up and Kim doesn't, and one of the two is a man and one is a woman. Who would you guess is the woman?

Answer 10

It seems there is some confusion between causation and correlation. Indeed, the question statement is false for causation, as can be seen by an example such as:

• If a dog is wearing a scarf, then it is a domesticated animal.

The following is not true:

• Seeing a domesticated animal wearing a scarf implies it is a dog.
• Seeing a domesticated dog implies it is wearing a scarf.

However, if you are thinking of probabilities (correlation) then it IS true:

• Dogs wearing scarfs are much more likely to be a domesticated animal than dogs not wearing scarfs (or animals in general for that matter)

The following is true:

• A domesticated animal wearing a scarf is more likely to be a dog than another animal.
• A domesticated dog is more likely to be wearing a scarf than a non-domesticated dog.

If this is not intuitive, think of a pool of animals including ants, dogs and cats. Dogs and cats can both be domesticated and wear scarfs, ants can't neither.

1. If you increase the probability of domesticated animals in your pool, it also will mean you will increase the chance of seeing an animal wearing a scarf.
2. If you increase the probability of either cats or dogs, then you will also increase the probability of seeing an animal wearing a scarf.

Being domesticated is the "secret" link between the animal and wearing a scarf, and that "secret" link will exert its influence both ways.

Edit: Giving an example to your question in the comments:

Imagine a world where animals are either Cats or Dogs. They can be either domesticated or not. They can wear a scarf or not. Imagine there exist 100 total animals, 50 Dogs and 50 Cats.

Now consider the statement A to be: "Dogs wearing scarfs are thrice as likely to be a domesticated animal than dogs not wearing scarfs".

If A is not true, then you can imagine that the world could be made of 50 Dogs, 25 of them domesticated (of which 10 wear scarfs), 25 of them wild (of which 10 wear scarfs). Same stats for cats.

Then, if you saw a domesticated animal in this world, it would have 50% chance of being a dog (25/50, 25 dogs out of 50 domesticated animals) and 40% chance of having a scarf (20/50, 10 Dogs and 10 Cats out of 50 domesticated animals).

However, if A is true, then you have a world where there are 50 Dogs, 25 of them domesticated (of which 15 wear scarfs), 25 of them wild (of which 5 wear scarfs). Cats maintain the old stats: 50 Cats, 25 of them domesticated (of which 10 wear scarfs), 25 of them wild (of which 10 wear scarfs).

Then, if you saw a domesticated animal in this world, it would have the same 50% chance of being a dog (25/50, 25 dogs out of 50 domesticated animals) but would have 50% (25/50, 15 Dogs and 10 Cats out of 50 domesticated animals).

As you can see, if you say that A is true, then if you saw a domesticated animal wearing a scarf in the world, it would be more likely a Dog (60% or 15/25) than any other animal (in this case Cat, 40% or 10/25).

Answer 11

The intuition becomes clear if you look at the stronger statement:

If A implies B, then B makes A more likely.

Implication:
  A true  -> B true
  A false -> B true or false
Reverse implication:
  B true  -> A true or false
  B false -> A false

Obviously A is more likely to be true if B is known to be true as well, because if B was false then so would be A. The same logic applies to the weaker statement:

If A makes B more likely, then B makes A more likely.

Weak implication:
  A true  -> B true or (unlikely) false
  A false -> B true or false
Reverse weak implication:
  B true  -> A true or false
  B false -> A false or (unlikely) true

Answer 12

There is a confusion here between causation and correlation. So I'll give you an example where the exact opposite happens.

Some people are rich, some are poor. Some poor people are given benefits, which makes them less poor. But people who get benefits are still more likely to be poor, even with benefits.

If you are given benefits, that makes it more likely that you can afford cinema tickets. ("Makes it more likely" meaning causality). But if you can afford cinema tickets, that makes it less likely that you are among the people who are poor enough to get benefits, so if you can afford cinema tickets, you are less likely to get benefits.

Answer 13

Suppose Alice has a higher free throw rate than average. Then the probability of a shot being successful, given that it's attempted by Alice, is greater than the probability of a shot being successful in general $P(successful|Alice)>P(successful)$. We can also conclude that Alice's share of successful shots is greater than her share of shots in general: $P(Alice|successful)>P(Alice)$.

Or suppose there's a school that has 10% of the students in its school district, but 15% of the straight-A students. Then clearly the percentage of students at that school who are straight-A students is higher than the district-wide percentage.

Another way of looking at it: A is more likely, given B, if $P(A\&B)>P(A)P(B)$, and that is completely symmetrical with respect to $A$ and $B$.