2

Is it okay to calculate one tailed p-values from post-hoc tests? I read this from a source but I just want to confirm it. "First, post hoc tests by their nature are two-tailed (you use them when you have made no specific hypotheses and you cannot predict the direction of hypotheses that don’t exist)."

Neal
  • 162
  • 1
  • 13

1 Answers1

2

A post hoc test means that you’re doing the test after seeing some results, such as conducting ANOVA on 4 groups and then testing that group A has a different mean than group B. Since you’ve observed the direction in that case, you must do a two-tailed test, since it’s possible to observe that A has the greater sample mean than B even if B has a greater population mean than A.

(This is for the same reason that, if your two-tailed test gives you a p-value of $1.5\alpha$, you don’t get to do a one-tailed test in the direction you’ve observed the difference and then claim significance at $\alpha$.)

However, it is possible that, prior to seeing results, you know that you specifically want to test that A has a greater mean than B. Then it would be okay to do a one-tailed test of that hypothesis. But if you observe that B has a greater sample mean than A, you still would test that A has a greater mean than B, and even if you skip that test because you know you will get $p>0.50$, you still count this test when you’re counting how many tests you’re doing for multiple comparisons (such as a Bonferroni correction).

I struggle to think of when you’d want to do that, but I suppose it’s possible.

Dave
  • 62,186
  • Thanks Dave. So essentially it would be the same as for other tests? If I have a non-directional hypothesis, then run and keep the 2- tailed results. However, if I have a directional hypothesis, then I can calculate the one tailed tests (although I can't change my hypothesis after I run the test). – Neal Apr 15 '20 at 15:50
  • 1
    As long as you’re coming up with the hypothesis by being a scientist and deciding what interests you, instead of snooping and looking at the data, then you are fine to do one-tailed testing or to skip tests entirely. Just make those decisions before you look at the results, or give the appropriate penalty for looking (such as adding one more comparison to your number of multiple comparisons and making the Bonferroni correction (for all tests, not just the new one) based on $k+1$ instead of $k$ observations). – Dave Apr 15 '20 at 16:07