2

I have used Bayesian reasoning in my research work and it has been extremely useful. The book I have read is E.T. Jayne's Probability theory. The idea is to formulate propositions and then probability theory tells how to assign numbers (viz. probability) to those propositions, conditional on one's information and beginning from some prior probabilities.

A proposition is something that is decidedly either true or false, irrespective of any observer. Therefore for a given coin "Probability of Heads is $p$" is not a proposition, because in Bayesian view probability depends on the observer (his/her information) and is not an objective property of the coin (like its mass or temperature).

Suppose I toss the coin once and get Heads. I ask "What's the probability of Heads in the next toss?" Consider the propositions: $$H_k\equiv\textrm{Heads in $k$-th toss}\\ T_k\equiv\textrm{Tails in $k$-th toss}$$

Then my question is: $P(H_2|H_1)=?$

I assume uniform prior probabilities: $P(H_k)=P(T_k)=1/2$ for any $k$. The result of first toss must change the probability of Heads in the second toss (I'm not assuming that the coin is fair; if say 100 tosses were to turn up Heads then I would suspect the coin to be biased in favour of Heads and probability theory must indicate the same to me).

Bayes rule gives: $$P(H_2|H_1)=\frac{P(H_1|H_2)P(H_2)}{P(H_1)}=P(H_1|H_2)$$

This gets me nowhere. How do I get a number and thus update the probability of Heads with each toss?

In this post and some articles I read on the net, this issue is resolved by taking "Probability of Heads is $p$" as a proposition and then seeking its probability (which amounts to seeking the probability of a probability). This does give an answer. My only problem (which I believe is a major problem) is that the aforementioned statement is not a proposition and so asking for its probability is nonsense. What's a way out of this conundrum?

Deep
  • 121
  • 3
  • Not enough framework provided here. // If you assume coin tosses are independent, then $P(H_2|H_1) = P(H_2).$ If you have some mechanism by which the coin 'remembers' its past behavior, then you need to describe that. // If you assume coin has same probability of heads on each toss then $P(H_1) = P(H_2).$ // In a true Bayesian context, you would start with a prior distribution on $\theta = P(H).$ – BruceET Sep 17 '18 at 07:35
  • @BruceET I don't have a reason to assume independence. As I said in the post, if suppose 100 successive tosses came up Heads, I would suspect that the coin is biased in favour of Heads, despite not knowing the mechanism that causes bias (in fact Bayesian inference doesn't require one to know the real world mechanisms that may affect an experiment and that is precisely its strength); the probability of Heads in 101-th toss, conditional on the fact that previous 100 tosses were Heads, ought to be more than 0.5. Bayes updating should result in this reasonable conclusion. – Deep Sep 18 '18 at 05:36
  • (contd.) If you take it to an extreme and say 1 billion tosses came up Heads, then it is reasonable to conclude that the coin is heavily biased in favour of Heads (and again you don't need to know the underlying mechanism that causes the bias), and therefore the probability of Heads coming up in the next toss must be close to 1. Also in the post I have assumed the prior to be 1/2 for Heads. – Deep Sep 18 '18 at 05:38
  • So you have the ultimate in informative priors, degenerate at $1/2.$ Interesting approach. Certainly simplifies things. – BruceET Sep 18 '18 at 07:12
  • @BruceET I am not sure I understand your comment. Should I have assumed a different prior? The problem is that I don't know how to do Bayesian updating in my case (so that I get a number for probability of Heads in the next toss) and this problem persists no matter which prior I choose. – Deep Sep 18 '18 at 08:15
  • First, not familiar with Jayne's book. Various authors calling themselves "Bayesians" seem to have fundamentally different definitions and philosophies. // Traditional Bayesian approach to coin-toss problem would be to have a beta prior distribution on $\theta = P(H).$ Data from actual coin tosses provide binomial likelihood. From prior and likelihood one gets posterior beta dist'n, from which to make interval estimates.// E.g.: Prior BETA(330, 270), then 620 H's in 1000, leads to posterior BETA(950,650) and 95% int. est (.57,.62) for $\theta.$ ... – BruceET Sep 18 '18 at 17:14
  • ... According to this framework, starting with degenerate $\theta$ identically equal to 1/2 [as in $P(\theta = 1/2) = 1]$ as prior would lead to ignoring data altogether and to the degenerate posterior $P(\theta = 1/2) = 1.$ – BruceET Sep 18 '18 at 17:21
  • @BruceET Ok your method is for finding the probability that the probability of Heads is $p$. That would be alright if wouldn't view probability theory as generalized propositional logic as Jaynes (and other Bayesians before and after him) has done. – Deep Sep 19 '18 at 04:59
  • In almost-religious debates among Bayesians and between frequentists and Bayesians, I prefer to be an 'agnostic'. I use various Bayesian and frequentitst methods to suit whatever practical situation is at hand. – BruceET Sep 19 '18 at 05:15
  • @BruceET You are right. I am an engineer too and I use probability for applications. If I tossed a coin 1000 times (say) and found all Heads, I am sure even a kid would bet on Heads in the next toss. I only thought Bayesian updating must somehow support that conclusion if I started from an undecided state (prior 1/2 for Heads). Frequentist approach is useless here because it demands infinite number of tosses. The only way out seems to be to seriously consider probability of a probability being $p$ because it at least gives some answer. – Deep Sep 19 '18 at 05:23

0 Answers0