Confidence interval for proportion assuming binomial distribution when observed proportion is exactly 1 (or 0)

Question

I have a simulation (a set of 200 iterations) which I used to compute a confidence interval for average statistical power. In binomial approximation, I know that the standard error is computed as $\sqrt{\frac{p(1-p)}{n}}$ where $p$ is the success probability and $n$ is the sample size. As I mentioned before, my $n=200$ as 200 iterations. Assume that in some instance I got statistical power exactly equal to 1 (when averaged from 200 iterations) and it is clear that the standard error is equal to 0 (computed as $\sqrt{\frac{p(1-p)}{n}} = \sqrt{\frac{1(1-1)}{200}} = 0$), and hence the confidence interval is no longer an interval, but a point (which is 1). If I need to get a confidence interval where there is no overlap in lower limit and upper limit,

1. can I use continuity correction (if this is not called so, what is it)?
2. if yes, what will be the computational formula?
3. reference(s)?

I could not find a proper reference which is applicable for my case (when sample size is 200 range – in case if there's any such constraint of sample size).

ETA: As a community suggestion, I was given reference to a similar question posted earlier Confidence interval around binomial estimate of 0 or 1, but I do not see it is helpful since the sample size is too small (typically less than 100 cannot be approximated normal). My case n=200, so I am not sure how to relate that response to my question. Plus I would appreciate references of a valid well-established method, and therefore, the quoted link will not provide answers to my question, though it looks similar.

Answer 1

Confidence intervals for a binomial probability parameter can be formed using the Wilson score interval and this interval respects the allowable parameter range for the probability parameter. It also accommodates the extreme case where all observations are in one of the binary categories. This case is discussed in O'Neill (2021) (p. 5), which also outlines some other properties of these intervals. When we have $n$ data points with sample mean $\bar{x}=1$ and the confidence interval with confidence level $1-\alpha$ is given by:

$$\text{CI}(1-\alpha) = \bigg[ \frac{n}{n+\chi_\alpha^2}, 1 \bigg],$$

where $\chi_\alpha^2$ is the critical point corresponding to an upper tail area of $\alpha$ for the chi-squared distribution with one degree-of-freedom. For example, with $n=200$ data points that are all ones and using a 95% confidence level (giving $\chi_\alpha^2 = 3.841459$) you get the confidence interval:

$$\text{CI}(0.95) = \bigg[ \frac{200}{203.841459}, 1 \bigg] = [0.9811547, 1].$$

Answer 2

I'd like to point out two misunderstandings I suspect in the question:

---

As a community suggestion, I was given reference to a similar question posted earlier Confidence interval around binomial estimate of 0 or 1, but I do not see it is helpful since the sample size is too small (typically less than 100 cannot be approximated normal). My case n=200, so I am not pretty sure how to relate that response to my question.

The typical recommendation about when it is OK to use the normal approximation for binomial proportion confidence intervals is that $np$ and $n(1-p)$ need to be $\geq 5$. So the absolute sample size alone is not sufficient: whatever the sample size is, $p$ can be too close to 0 or 1 to allow this approximation. And that is what happens in your case. The linked question is thus relevant pretty much regardless of what $n$ is. I'll thus also vote to close this question as duplicate.

---

Secondly I suspect that the underlying source of confusion may be the following:

As you (indirectly) say, the variance of a binomial distribution is $s^2 (k) = np(1-p)$, i.e. the variance in the number of observed "successes" $k$ for a binomial distribution that truly has probability $p$ after $n$ trials. Thus, the variance in the observed proportion $\hat p = \frac{k}{n}$ is $s^2 (\hat p) = \frac{p(1-p)}{n}$.

Now, it is crucial to realize that this variance is still formulated as a function of the true probability $p$, not the observed proportion $\hat p$. And for that it is correct: the observed proportion must have variance 0 given that the true probability is 0 or 1.
However, for the confidence interval, you'd rather need the variance given the observed proportion.

To put it slightly differently, for $\hat p = 0$ (or 1), the plug-in approximation of $p \approx \hat p$ does not work well: it disregards the obvious and important possibility that other true $p$ than 0 (or 1) can lead to 0 observed successes (failures).
For example, with $n = 200$ an true probability of $p = 0.99$ has a 13 % chance of yielding $k = 200$ and thus an observed probability of $\hat p = 1$.

If you follow through with this thought and apply Bayes theorem, i.e., you calculate the $Pr(\hat p|p, n)$ for all possible $p$ and then normalize* this to get the distribution of $Pr (p | \hat p, n)$ you can derive intervals as provided e.g. R package binom does in binom.bayes().

* and weight: if you do not weight but consider all $p \in [0, 1]$ equally likely before that binomial trial, that corresponds to a uniform prior. Others choose to use Jeffrey's prior (which does not differ much for $n = 200$ - you may say, the two priors differ by half a trial with half a success). That's the interval called "Jeffrey's" in the Brown paper in the linked question's answer.