help computing the beta likelihood when we only observe the number of successes and failures (not the latent probability of success)

Question

Imagine we have a biased coin that generates heads with unknown probability $\theta$ where $\theta$ is drawn from a beta distribution with known parameters $(\alpha, \beta)$. Next imagine that we flip the coin some number of times and we get $m$ heads and $n$ tails. I'd like to compute the quantity $p(m,n|\alpha,\beta)$.

What I have so far is $$ \begin{align} p(m,n|\theta) &= \begin{pmatrix}m+n\\n\end{pmatrix} \theta^m (1-\theta)^n \\ &\\ p(\theta|\alpha,\beta) &= \frac{\theta^{\alpha-1}(1-\theta)^{\beta-1}}{B(\alpha,\beta)} \\ &\\ p(m,n|\alpha,\beta) &= \int_0^1 p(m,n|\theta) \, p(\theta|\alpha,\beta)\,d\theta \\ &= \int_0^1 \begin{pmatrix}m+n\\n\end{pmatrix} \theta^m (1-\theta)^n \frac{\theta^{\alpha-1}(1-\theta)^{\beta-1}}{B(\alpha,\beta)} \, d\theta\\ &= \begin{pmatrix}m+n\\n\end{pmatrix}\bigl(B(\alpha,\beta)\bigr)^{-1} \int_0^1 \theta^{m+\alpha-1}(1-\theta)^{n+\beta-1}\,d\theta \\ &= \begin{pmatrix}m+n\\n\end{pmatrix} \frac{B(m+\alpha, n+\beta)}{B(\alpha,\beta)}. \end{align} $$

The final expression is pretty nice, but I'm wondering if it can be simplified even further. For instance, is it equivalent to a binomial distribution with some parameter $\theta'$?

Answer 1

The beta binomial and binomial distributions are distinct. The binomial distribution has mean $kp$ and variance $kp(1-p)$ where $k$ is the number of trials and $p$ is the probability of success.

The pmf that you have written is a beta-binomial distribution (as you know). The beta binomial distribution has mean $$\begin{align} \mu &=(n+m)\frac{\alpha}{\alpha+\beta} \\ &=q(n+m) \end{align}$$ This is suggestive! Perhaps we can use the substitution $q = \frac{\alpha}{\alpha+\beta}$ and show that the beta-binomial distribution is secretly a binomial distribution.

Unfortunately, this plan falls apart when we look at the beta-binomial variance $$\begin{align} \sigma^2 &= (n+m)\frac{\alpha}{\alpha+\beta}\cdot\frac{\beta}{\alpha+\beta}\cdot\frac{\alpha+\beta+n+m}{\alpha+\beta+1} \\ &= q(n+m)(1-q)\frac{\alpha+\beta+n+m}{\alpha+\beta+1} \end{align}$$ and the first three factors are exactly what we want to find in a binomial distribution... but the fourth factor is not equal to 1 for $n + m > 1$.