8

I know the standard formula for the Bernoulli CI is:

$$\hat{p}\pm z_{1-\alpha/2}\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$$

If $\hat{p} = \frac{m}{n}$ how do I estimate the confidence interval when$\ n$ is small and$\ m = 0$? This case would collapse the above equation to $\ 0 \pm 0$, which implies that the confidence interval does not improve with larger $\ n$.

In my mind, the CI should start at [0,1] and the upper bound should decrease as $\ n$ increases, given that $\ m$ remains at 0.

japata
  • 183
  • 1
    You could use the actual likelihood function of your data L(p) which will be proportional to $p^m,(1-p)^{n-m}$ Given some prior on p, for example some Beta distribution you can get the posterior and the credible intervals on p. – sega_sai Nov 02 '16 at 18:09
  • Notice that the same applies to $\hat{p}=1$. – Alexis Nov 02 '16 at 18:27
  • Running a Bayesian analysis of the data will produce a credible interval even when $\hat{p}=0$. – Xi'an Nov 02 '16 at 20:00
  • One possibility that's sometimes used in some application areas is called "the rule of three". Also see Wikipedia's page on it – Glen_b Nov 02 '16 at 21:28
  • Many similar questions: https://stats.stackexchange.com/questions/559916/inference-in-binomial-with-zero-successes-and-fixed-number-of-trials, https://stats.stackexchange.com/questions/82720/confidence-interval-around-binomial-estimate-of-0-or-1, https://stats.stackexchange.com/questions/464672/should-a-two-sided-binomial-confidence-interval-for-an-observed-frequency-of-zer, https://stats.stackexchange.com/questions/56562/calculating-confidence-intervals-for-a-proportion-when-there-are-no-successes, – kjetil b halvorsen Feb 16 '24 at 01:01
  • Also search this site for "rule of three" – kjetil b halvorsen Feb 16 '24 at 01:01

1 Answers1

9

The reason the usual "CLT" confidence interval becomes 0 is because when $p$ is very close to 0 or 1 (and the relative number of samples is low), the CLT becomes a bad approximation. This is because when $p=0,1$, your random variable is constant. On the other hand, when $p$ is very close to 1 or 0, you need a very large amount of samples to distinguish $p$ from exactly 1 or 0.

There are a couple of approaches to get the true confidence interval. The easy way is to appeal to the Wilson score interval:

$$\frac{1}{1 + \frac{1}{n} z^2} \left[ \hat{p} + \frac{1}{2n} z^2 \pm z \sqrt{ \frac{1}{n}\hat{p} \left(1 - \hat{p}\right) + \frac{1}{4n^2}z^2 } \right].$$

The second option is to numerically estimate the true confidence interval by explicitly using the binomial distribution, as opposed to appealing to the normal distribution.

Alex R.
  • 13,897
  • Thanks for the help! The Wilson score interval behaves closer to my expectations. I'm going to do some reading on these different methods for confidence intervals, as all I've been exposed to are the CLT methods. – japata Nov 02 '16 at 19:36