1

Wikipedia says that p-values are uniformly distributed over [0,1] if the null hypothesis is true and for continuous data.

What is the expected p-value distribution if the test statistic is discrete? E.g. occurrences in an enrichment analysis.

Glen_b
  • 282,281
slm
  • 31
  • 3
    See the discussion at Non-uniform distribution of p-values when simulating binomial tests under the null hypothesis -- a step function that touches the $y=x$ line at the top of each step is the general case. I suggest there that it could be called quasi-uniform while whuber has argued in comments elsewhere on site that is should just be called 'discrete uniform'. However, that doesn't match a fairly conventional usage of the term, so I don't think that suggestion will catch on. – Glen_b Oct 19 '17 at 09:20
  • 2
    Note that it's not the data but the distribution of the test statistic that is at issue. Consider, for example, a two-sample Kolmogorov-Smirnov test; the data can be continuous but the discreteness of the distribution of the test statistic is the issue. I have edited the question accordingly. – Glen_b Oct 19 '17 at 09:27
  • 1
    Another (very common) situation in which the Wikipedia claim is false is a "composite null" hypothesis, such as testing $H_0:\mu \le 0$ against the alternative $H_A:\mu \gt 0$ for data from a Normal$(\mu,\sigma^2)$ distribution. The p-value will be uniformly distributed only when $\mu=0$; if $\mu\lt 0$ (which is still part of the null hypothesis), the p-value will be concentrated closer to $1$. – whuber Oct 19 '17 at 14:22

0 Answers0