3

I have found this quote from a (to me obscure) power calculation software:

For a given effect size, sample size, and alpha, a one-tailed test is more powerful than a two-tailed test (a one-tailed test with alpha set at .05 has approximately the same power as a two-tailed test with alpha set at .10).

I am interested in the claim in the brackets.

  1. Is this generally true for "any" hypothesis test?
  2. $\rm Power_{one-tailed}(\alpha) \approx Power_{two-tailed}(2 \times \alpha)$ for any $\alpha \in (0,1)$, ceteris paribus?
  3. Does anyone know of a reference?

(sorry for the trashy notation)

User1865345
  • 8,202
hangingprawns
  • 45
  • Probably easier to think about the critical threshold for the test statistics rather than power. For t-tests these are identical in each case so the power is identical. For other tests where the alpha for the two-tailed test is split equally on both sides of the null distribution then the critical threshold should be identical as well? – George Savva Jul 03 '23 at 14:08
  • In a given situation, draw the two power curves ... and see where (and thereby why) they differ. – Glen_b Jul 04 '23 at 19:12

1 Answers1

1

Yes and no...the statement is almost correct. The power for a 1-tailed and 2-tailed tests are usually approximately the same. However, the power is technically smaller for a 1-tailed test (not larger).

The significance level determines where the critical value(s) will be for the distribution for the null hypothesis. And, you will have the same critical values for a 1-tailed test with $\alpha$ and a 2-tailed tests with $2\alpha$.

Assuming you want to reject the null, this means you want a value to fall in the critical region (that is "beyond" the critical values). If you fail to reject, you make a type II error, and the probability of this happening is $\beta$. Power is defined as $1-\beta$ (the probability of NOT making a type II error).

The calculation for $\beta$ is to find the probability of being in the non-critical region of the null distribution for the actual distribution (which you estimate using the expected effect size). For a one-tailed test, this region is (essentially) unbounded, usually something like $(-\infty,\nu_\text{c.v.})$. Whereas for a two-tailed test, this region if bounded, usually something like $(-\nu-\text{c.v.},\nu_\text{c.v.})$...and thus smaller. So, $\beta$ is smaller for a 2-tailed test than a 1-tailed test...which means the power is bigger for a 2-tailed test than a 1-tailed test.

However, the assertion that they are approximately the same is usually reasonable, as that "extra" non-critical region area in the 1-tailed test, $(-\infty,-\nu_\text{c.v.})$, the probability of that occurring for the "real" distribution is usually quite small.

Happy to provide a numerical example using a normal distribution if this will help clarify.

Gregg H
  • 5,474