17

how can I calculate the variance of p as derived from a binomial distribution? Let's say I flip n coins and get k heads. I can estimate p as k/n, but how can I calculated the variance in that estimate?

I'm interested in this so that I can control for variance in my ratio estimates when I'm comparing between points with different numbers of trials. I'm more sure of the estimate of p when n is greater, so I would like to be able to model how reliable the estimate is.

Thanks in advance!

example:

  • 40/100. The MLE of p would be 0.4, but what is the variance in p?
  • 4/10. The MLE would still be 0.4, but the estimate is less reliable, so there should be more variance in p.
draco_alpine
  • 135
Jautis
  • 618

1 Answers1

24

If $X$ is $\text{Binomial}(n, p)$ then MLE of $p$ is $\hat{p} = X/n$.

A binomial variable can be thought of as the sum of $n$ Bernoulli random variables. $X = \sum_{i=1}^n Y_i$ where $Y_i\sim\text{Bernoulli}(p)$.

so we can calculate the variance of the MLE $\hat{p}$ as

$$\begin{align*} \text{Var}[\hat{p}] &= \text{Var}\left[\dfrac{1}{n}\sum_{i=1}^n Y_i\right]\\ &= \dfrac{1}{n^2}\sum_{i=1}^n Var[Y_i]\\ &= \dfrac{1}{n^2}\sum_{i=1}^n p(1-p)\\ &= \dfrac{p(1-p)}{n} \end{align*}$$

So you can see that the variance of the MLE gets smaller for large $n$, and also it is smaller for $p$ close to 0 or 1. In terms of $p$ it is maximized when $p=0.5$.

For some confidence intervals you can check out Binomial Confidence Intervals

bdeonovic
  • 10,127
  • I think the link is similar to what I'm looking for, but I want a value that is equivalent to the variance of p. How can I get that from the confidence interval? – Jautis Jan 19 '16 at 20:49
  • I edited my original answer to more closely answer your question. – bdeonovic Jan 19 '16 at 21:13
  • How do you deal that the formula of the variance requires p but you only have an estimation of p? – Heberto Mayorquin Sep 03 '18 at 11:15
  • 1
    You could consider using a variance stabilizing transform such as $arcsin(\sqrt{\hat{p}})$ and then you get that the variance of the transformed variable is $\tfrac{1}{4n}$ – bdeonovic Sep 06 '18 at 18:45