0

I have used statsmodel in python to compare the means between a control and test group. The difference of the means is 29 and I get a low p-value making my results statistically significant. The 95% confidence interval is [18, 25] which does not include 0 but also doesnt include my actual difference which is 29. Is there something wrong with these results or can I assume that the 29 value just happened to be in the 5% of the times that is outside the CI?

nicnaz
  • 77
  • 2
    How did you calculate the confidence interval and the difference in means? A coding error sounds like the likely culprit. – Dave Jul 13 '20 at 18:38
  • I just used this code for the confidence intervals https://www.statsmodels.org/stable/generated/statsmodels.stats.weightstats.CompareMeans.html for the difference in means I just did the mean of each group and subtracted – nicnaz Jul 13 '20 at 18:44
  • Please provide the specific code you used, and if possible show density plots or histograms of all the values for the 2 groups. Are there any large outliers in one or the other of the groups? – EdM Jul 13 '20 at 18:50
  • Are these results impossible to happen? – nicnaz Jul 13 '20 at 19:02
  • can you include the code? – StupidWolf Jul 13 '20 at 22:27
  • In bootstrapping, for example, bias in the estimates of the quantity of interest can lead to apparent confidence intervals that don't include the mean. That seems highly unlikely to be the case in what you describe, but without further details on the code and the data I would hesitate to say "impossible." – EdM Jul 14 '20 at 12:50

0 Answers0