0

This may sound simple but I'm lost. I have a simple experiment 15 controls and 15 treatments. 13 of the 15 controls were positive, but only one of the treatment samples was positive. This makes the SD = 0 so t-tests and the like fail. It's obvious there is a difference in the samples, but I have no test to verify.

I could make a model based off the control results and simulate some data, but the sample size is so small that model would stink. I could use probabilities, thus 86% are expected to be positive, but only 0.06% were found. I could use a hypothesized error rate, like 13% (2 of the 15) and the results are still well beyond expectations.

Any advice? I see this question has been asked similarly but there has not been a clear solution to significance.

Thanks

OK thanks for thanks for that null hypothesis testing link. Here is the answer I've come up with to justify the rareness of the results. Although it is still tricky to explain. But here is a probability table for my example. Essentially there are 155117520 possible combinations, but only 225 in which there is only a single positive, so 0.0001%. Compared to the control which had 13 positives which had 11025 possibilities, which might occur 13% of the time. So being the control this would be 87%. In this case SE is 1.41 and I want a 95% confidence interval so I would expect 95*1.41% points, which is 1.34%. So, to accept the null that the treatment was the control I would expect a value of 87% +- 1.34% or really 12 to 14 positives. The question then become how to explain. I would think I would expect my treatment to be with in 95% CI of my control. This sound correct? Pro table

Sycorax
  • 90,934
J Slowik
  • 1
  • 2
    How the standard deviation in your example is zero? – Tim Mar 24 '23 at 19:06
  • 2
    https://en.wikipedia.org/wiki/Lady_tasting_tea. See https://stats.stackexchange.com/search?tab=votes&q=lady%20tea&searchOn=3 for discussions here on CV. – whuber Mar 24 '23 at 19:37
  • When you have no variation in the sample the SD will be 0. So if I had 15 samples all of the same value my SD would also be 0. But a 0 SD in the t-test it fails, can't divide by 0 right. – J Slowik Mar 25 '23 at 21:18
  • 1
    But you said “only one of the treatment samples was positive” implying that not all the samples were negative. – Tim Mar 27 '23 at 19:27
  • The only way the SD in the two-sample t-test that might be applied here could be zero is when all 15+15=30 results are the same value. That doesn't seem to be the case, so perhaps you haven't applied the test correctly. BTW, the mean treatment value should not be within a 95% CI computed from the control: that property defines a prediction interval, not a confidence interval. See https://stats.stackexchange.com/questions/16493. – whuber Mar 27 '23 at 20:57
  • I see your point. I had been just using Excel which throws out the error because one of the groups has a SD of 0. But using a pooled SD then even if the SD of one of the groups is 0 shouldn't matter. Sound's like I should also maybe try the Welch's modification. – J Slowik Mar 28 '23 at 21:32

0 Answers0