1

Suppose I collected 3 random samples (each of size n) from the same population. I calculated three estimates (of the same parameter) and three 95% confidence intervals (for the same parameter).

What is the probability that all 3 intervals failed to capture the true parameter value? If I assume the confidence intervals are independent and all the statistical model assumptions are met:

$$F = (1-0.95)^3$$ $$F = 0.000125$$

There is a 0.0125% probability that all 3 intervals failed to capture the true parameter.

I have several questions:

  1. What is the harm of interpreting a 95% confidence interval as there is a 95% probability that a random interval contains a fixed parameter value? A single confidence interval may or may not contain the true parameter. Why can't I quantify the "not knowing" with probability?

  2. Why do frequentists have to come up with round-about statements like, "There's a 95% chance that the next sample would generate a 95% confidence interval that contains the true value." If the probabilities are independent, isn't the 95% probability applicable to both the current and next sample?

  3. How could you answer the "all 3 intervals fail to capture the parameter" question without assuming that the "95%" is a probability?

  4. Does the answer of 1.25% depend on whether the 3 confidence intervals are realized (calculated) or yet to be realized (yet to be calculated)? Does it matter whether the first 2 sentences are in past tense?

The question is heavily related to Why aren't these sentences about confidence intervals equivalent?.

Update: 08/17/2022: Henry found a math mistake.

William Chiu
  • 790
  • 2
    $(1-0.95)^3 = 0.000125$ which is $0.0125%$ not $1.25%$ – Henry Aug 17 '22 at 07:26
  • 1
    Your questions 1 and 2 have been asked many times before, such as at https://stats.stackexchange.com/questions/26450 and https://stats.stackexchange.com/questions/6652 and https://stats.stackexchange.com/questions/11609 while your question 4 (from a frequentist perspective) has the answer "yes" and that affects the answer to question 3 – Henry Aug 17 '22 at 08:16
  • (1) is the standard interpretation of a confidence interval. The alleged frequentist interpretation in (2) is incorrect. (3) is baflling--what would the 95% mean otherwise? (4) also is strange due to an unconventional interpretation of both confidence and probability. – whuber Aug 17 '22 at 13:53
  • Thank you, Henry. I fixed the math mistake. – William Chiu Aug 17 '22 at 16:19
  • @whuber - I got #2 from Wiki (which cites Neyman 1937). https://en.wikipedia.org/wiki/Confidence_interval#Interpretation – William Chiu Aug 17 '22 at 16:33
  • Thank you. What bothers me about that phrasing is the reference to the next sample. That's irrelevant to discussing this sample. So, although technically the description is correct, it's misleading. – whuber Aug 17 '22 at 17:02

0 Answers0