This Math Overflow community question asked for "examples of bad arguments which involve the application of mathematical theorems in non-mathematical contexts" and produced a fascinating list of pathologically applied mathematics.
I am wondering about similar examples of pathological uses of Bayesian inference. Has anyone encountered academic articles, eccentric blog posts that use Bayesian methods in crankish ways.
Yep. I was hired as a statistical consultant recently to scrutinize a particular(ly awful) article whose authors managed to make themselves look even worse in a letter to the editor using Bayes' theorem. They began with a miscalculated positive predictive value from their article (PPV = 95% supposedly). They basically disregarded a critical letter about this by Ricci (2004) that tried (and failed) to tell them how they should've calculated it (he suggested 82.3%). Then they found a biostats textbook (Elston & Johnson, 1994) and misquoted it. We bought the book and checked, but in retrospect, this was just as unnecessary as I suspected. Get a load of this mess (from Barsness et al.'s reply letter to the editor):
Bayes' theorem1 generally states that a low prevalence of a particular disease (NAT) strengthens the positive predictive value of a positive test (rib fracture) to define the disease state (victim of NAT)...According to Bayes' theorem,1 the probability of an event is defined by the following equation: $$\rm P=\frac{P(S/D_1)}{P(S/D_1)+P(S/D_2)}$$ P is the probability of a true event (victim of NAT), P(S/D1) is the probability of a positive test (PPV of a rib fracture to predict NAT) and P(S/D2) is the posterior probability of a positive test (prevalence of NAT). Substituting our data, the probability that a rib fracture is a true event $[p = 95/(95 + 1.6)]$ is 98.3 percent. Using the aforementioned lower PPV calculation of 82.3 percent, the probability of a true event is 98.1 percent.
See anything odd coherent here? I sure don't...
This is Bayes' theorem as Elston and Johnson (1994) apply it to an example of hemophilia heredity:
$$\rm P(D_1|S)=\frac{P(D_1)P(S|D_1)}{P(D_1)P(S|D_1)+P(D_2)P(S|D_2)}$$
The discrepancies speak for themselves, but here's a quote from their discussion of the example:
The fact that she had one son who is unaffected decreases the probability that she inherited the hemophilia gene, and hence the probability that her second son will be affected.
Where Barsness and colleagues got the idea that low prevalence strengthens PPV, I do not know, but they sure weren't paying attention to their own textbook of choice.
They don't seem to understand that PPV is the probability of a "true event" (D1) given a rib fracture (S). Thus, in a poetically complete demonstration of "garbage in, garbage out", they input their PPV as numerator and denominator, add the prevalence to the denominator, and get a higher PPV. It's a shame they didn't realize they could continue this circularly ad nauseum: $$p_1 = 95/(95 + 1.6)=98.3\rightarrow p_2 = 98.3/(98.3 + 1.6)=98.4\rightarrow\dots$$ Though 98.4 is actually $\lim_{k\rightarrow\infty} p_k(p_{k-1},1.6)$; i.e., any PPV could be converted to 98.4 with prevalence = 1.6 if their version of the equation were correct by applying it iteratively.
When using their prevalence info and some reasonable estimates of sensitivity and specificity from other studies on the topic, the PPV turns out to be much lower (maybe as low as 3%). The funny thing is that I wouldn't have even thought to use Bayes' theorem if they hadn't tried to use it to strengthen their case. It's clearly not going to work out that way given a prevalence of 1.6%.
References
· Barsness, K. A., Cha, E. S., Bensard, D. D., Calkins, C. M., Partrick, D. A., Karrer, F. M., & Strain, J. D. (2003). The positive predictive value of rib fractures as an indicator of nonaccidental trauma in children. Journal of Trauma-Injury, Infection, and Critical Care, 54(6), 1107–1110.
· Elston, R. C., & Johnson, W. D. (1994). Essentials of biostatistics (2nd ed.). Philadelphia: F.A. Davis Company.
· Ricci, L. R. (2004). Letters to the Editor. Journal of Trauma-Injury, Infection, and Critical Care, 56(3), 721.
