-1

In the three-factor (all categorical) ANOVA, two-way interaction is very significant, but when qualified with the third factor, it became non-significant. But the pattern seems different according to the different levels of the third factor(like in the picture). So I'm confused why is three-way interaction dropped dramatically?enter image description here

(copied from later duplicate because it's useful to have both pictures explained in the same thread -- the same explanations work for both)

I have two-way interaction on the two levels of the third categorical variable, the pattern looks different, why is three-way interaction non-significant?enter image description here

Glen_b
  • 282,281
Lin Peng
  • 9
  • 1
    I do not see any strong evidence of a three-way interaction there. Can you expand on why you think there might be one? – mdewey Aug 17 '17 at 12:16
  • Thanks for commenting.In my understanding the three-way interaction means the two way interaction varies according to different levels of the third factor, is it right? The two-factor interaction patterns are different based on two different levels of the third factor, so I expect a three-factor interaction – Lin Peng Aug 17 '17 at 12:50
  • In the left hand panel the reds are above the blues at the left but fall below them at the right. In the right hand panel the reds are above the blues at the left but fall below them at the right. So the qualitative interaction is the same. There may be a small quantitative difference but in the absence of a scientific hypothesis about this I would say that even if the three way reached some arbitrary level of statistical significance there is no meaningful three-way interaction here. – mdewey Aug 17 '17 at 15:42
  • Possible duplicate of why three-way interaction is not significant – kjetil b halvorsen Aug 24 '17 at 11:48

2 Answers2

2

Your picture shows the averages of different groups, but not how spread out the data are. You will likely find your answer here: Even though the averages appear to show a different pattern, the data are quite spread out, and the difference is not significant.

When you omit the interaction, you have smaller standard errors for the effects you do test, and you find a significant effect.

However, in general, you should use p-values cautiously and only for pre-specified hypotheses, not for model comparison.

Caspar van Lissa
  • 121
2

  1. The different appearance of the "pattern" in the second pair of pictures may be removed simply by moving the first red line down a little; that kind of shift wouldn't change the 3 way interaction at all. (In short you aren't reading the plot correctly; the pattern you should be looking for doesn't seem to differ)

    With either plot, if you look at the left plot and consider the "shift in red going from left to right" minus the "shift in blue going left to right" that's about the same as the corresponding thing as in the right plot. That implies that the three-way interaction would be very small in size (close to $0$).

  2. You don't present any information about sample size or residual variation, so even if there were a difference in pattern there's not enough information in your question to suggest that it would result in significance.

Community
  • 1
Glen_b
  • 282,281