3

My 1-way ANOVA test is not significant. Is it possible that a Scheffé test as a post hoc test be significant?

If not, please say why not.

Nick Cox
  • 56,404
  • 8
  • 127
  • 185
Bernard
  • 383
  • In general, the process of iteratively seeking tests that will deliver significant results is called "statistical fishing" and will greatly inflate your type 0.05 error rate. If the data truly aren't consistent with a trend, consider instead how you can communicate those results in a significant way to your collaborators. – AdamO Oct 04 '14 at 15:11
  • user48162 - No doubt @AdamO meant to type "Type I error rate" there. Is this for some exercise? – Glen_b Oct 04 '14 at 21:44
  • @Glen_b HAH. That's what happens when you switch from coffee to tea and check CV in the morning. Or maybe I'm onto a Fisherian epiphany regarding a fractal Hausdorff dimension of error rates. – AdamO Oct 04 '14 at 21:59

2 Answers2

2

It is NOT possible. The ANOVA $F$ test is exactly equivalent to finding the most significant possible contrast and testing it against the Scheffé critical value. (Actually, that is how the Scheffé method is derived.) So if the $F$ is nonsignificant, so is every possible contrast.

Nick Cox
  • 56,404
  • 8
  • 127
  • 185
Russ Lenth
  • 20,271
1

@rvl's answer is correct for Scheffé's test, which is what the question was about. With Scheffé it is not possible for any comparison to be statistically significant when the overall ANOVA is not.

But for others reading this question and answers, note that it would be a mistake to generalize this answer to all multiple comparison tests. With the methods of Tukey or Dunnett, it is possible for a comparison between two group means to be statistically significant even though the overall ANOVA is not.

Nick Cox
  • 56,404
  • 8
  • 127
  • 185
Harvey Motulsky
  • 20,456
  • Why this happened for scheffe but not for happened for other multiple comparison tests? (just for scheffe?) – Bernard Oct 05 '14 at 08:34