Test if two binomial distribution are significantly different from each other

Question

I obtained two groups of data from my experiment and calculated the binomial distribution for each group, trying to see if they are significantly different from each other. However, the z sore I got was way smaller than -4, which was a little bit odd to me. Here is how I did my calculation:

The control group: # of trails n1=34; # of successes k1=2, success rate p1=2/34=5.9% The experimental group: # of trails n2=25; # of successes k2=2, success rate p2=2/25=8.0%

average p=6.8% standard error=0.0044 test statistic z=(5.9%-8%)/standard error=-4.77 It's a two-tailed test. I couldn't find a p-value corresponding to the z score I got (-4.77) on the table. Does that mean my p-value was just really small or I made mistakes here?

Your calculations are erroneous: you appear to have typed an extra zero in the standard error. Your approach is incorrect, as well, for several reasons: a Z-test is inapplicable and the method of computing the SE is not appropriate. Search our site for http://stats.stackexchange.com/search?q=two+sample+binomial+test, for instance. — whuber, May 16 '16 at 20:40
These may help: https://stats.stackexchange.com/questions/113602/test-if-two-binomial-distributions-are-statistically-different-from-each-other http://www.itl.nist.gov/div898/software/dataplot/refman1/auxillar/binotest.htm — Quinn Asena, May 16 '16 at 22:28
What is the process of acquiring the data? Why do you believe that the data is binomial distrbuted? — Sextus Empiricus, Feb 25 '19 at 13:52
You are using
$$\frac{p_1-p_2}{\text{s.e.} (\bar{p})}$$

Where did you got the idea to use the difference in succesrate divided by the standard error of the average succes rate? (the test which seems most close is the 'z-test to compare the equivalence of two proportions' but it is using an estimate for the standard deviation instead of an estimate for the standard error, and also it does not work well with low number of expected counts) — Sextus Empiricus, Feb 25 '19 at 14:02
Does this answer your question? Test if two binomial distributions are statistically different from each other — Scott, Jan 16 '20 at 08:00

score 0 · Answer 1 · answered Feb 25 '19 at 13:22

0

If you had obtained a z-score of -4.77, then you would have observed a very significative difference However, the calculations are wrong, as (2 in 34) vs (2 in 25) should return a much closer to 1 p-value

answered Feb 25 '19 at 13:22

David

2,596

score 0 · Answer 2 · answered Feb 25 '19 at 13:36

Your calculation of the standard error is way off. It should be closer to 0.07

In any case, the normal approximation is not applicable when there are barely any successes observed. You should be using the exact binomial distribution to do this calculation. The R function prop.test will do that test. If I'm not mistaken, you will find a p-value of 1 when you do that.

Test if two binomial distribution are significantly different from each other

2 Answers2