4

The R function binom.test performs an exact binomial test. In trying to understand how many trails would be necessary to reject the null given zero successes, I plotted the p-value as follows

p <- 1/16
n <- seq(2,200)
pval <- lapply(n, function(x) binom.test(0,x,p)$p.value)
plot(n, pval)

plot of pval vs n

I was surprised to see that the p-value did not decrease monotonically with increasing trials. What is the reason for this and is it accurate?

jarmond
  • 143
  • 3
    It is because the binomial random variable is discrete. See my paper in the American Statistician coauthored with Christine Liu. "The Saw-toothed Behavior of Power vs Sample Size and Software Solutions: Single Binomial Proportions using Exact Methods. American Statistician (2002) Vol. 56 pp 149-155. – Michael R. Chernick Mar 15 '18 at 00:25
  • I took @StephanKolassa advice. I realize that the result is counter-intuitive and it may take more detailed accounts in the references to get a clear picture. – Michael R. Chernick Mar 15 '18 at 14:31

1 Answers1

3

The reason is because the binomial random variable is discrete. My paper in The American Statistician [1] explains this and gives other references. Think about the fact that the planned type 1 error cannot be achieved exactly for all values of the sample size n.

[1] Michael R Chernick & Christine Y Liu (2012). The Saw-Toothed Behavior of Power Versus Sample Size and Software Solutions. The American Statistician, 56:2, 149-155, DOI: 10.1198/000313002317572835

Stephan Kolassa
  • 123,354
Michael R. Chernick
  • 42,857