0

I'm trying to learn about when there is a 'statistically significant difference' and the size of the effect.

Imagine that we have a new medicine, and there is a Randomized Controlled Trial (RCT). The results are (where more is better). We don't have access to the raw data.

Placebo:

  • N = 33
  • Mean = 85
  • Standard Deviation = 14

New Medicine:

  • N = 33
  • Mean = 94
  • Standard Deviation = 13

The paper shows these results along with the p-value, which is p = 0.001. The paper's conclusion is that there is a statistically significant difference in the results.

I calculated the individual confidence intervals (CI) with t-student distribution and obtained (lower limit, upper limit): Placebo: (80.04, 89.96) New Medicine: (89.39, 98.61)

In the individual CI, there is an intersection between the placebo and new medicine. In other words, in this calculation, there is no statistically significant difference in the results.

However, I also calculated the confidence interval for the difference between the two means, and the result was: (-15.77432892, -2.225671077)

With this result, it's possible to notice a statistically significant difference in the results.

After all these calculations, is there or isn't a statistically significant difference in the results?

Why do individual confidence intervals have a different interpretation than the confidence interval for the difference between the two means?

Arthur A
  • 1
  • Non-overlapping CIs is a much more stringent requirement than the CI of the difference excluding zero. See for example this answer illustrating that under some assumptions, for a t-test, non-overlapping 95% CIs means the difference is significant at P < 0.005. – PBulls Jan 30 '24 at 07:45
  • The two-sample CI is the correct choice since you want to test for the difference between two populations. – utobi Jan 30 '24 at 08:49

0 Answers0