1

In my model, some potentially confounding continuous variables had to be taken in account as well as an interaction of this variable with the main factor of interest, which is categorical. Initially, I had been doing some modelling in R but I switched to JMP for some extra plots. There the p-values of my main factor of interest changed dramatically. After some digging, I found out that JMP centers variables when they are involved in an interaction and that this option could be disabled. As expected, the p-values agreed with the R output as soon as I disabled it. I found an explanation as to why the p-values change when an interacting variable is centered here: p-values change after mean centering with interaction terms. How to test for significance?

However, while this explains how it could happen, I'm still in the dark as to how you could then decide which is more relevant? Somewhere in the link it is noted that the correct test for significance should involve all coefficients of the interacting effects at once but I'm completely unaware of how to do that?

Stephan Kolassa
  • 123,354
Jobbe Goossens
  • 21

1 Answers1

0

The coefficient reported for an individual predictor by most statistical software needs careful interpretation when the predictor is involved in interactions. The coefficient is for its association with outcome when all of its interacting predictors have values of 0 (continuous interactors) or are at reference levels (categorical interactors). The p-values evaluate whether the coefficient value under those conditions differs "significantly" from 0. When there's an interaction, the value of an individual coefficient estimate and its apparent "significance" of a difference from 0 (in that highly restricted interpretation) thus changes as you center the interacting predictors.

Nevertheless, as you found, the model is fundamentally the same whether or not you center the predictors. All model predictions would be the same either way. I that context, I'd say that no individual coefficient is "relevant" on its own (except for the coefficient for the very highest level of interactions, which isn't affected by centering).

Testing whether a predictor including its interactions has a "significant" association with outcome requires a combined test of some type. These tests typically have a name like "anova" although they apply to situations beyond the classic ANOVA situation with categorical predictors and continuous outcomes.

For a single predictor, you can fit two models: one with it (and its interactions), one without, and use anova() to perform a likelihood-ratio test between the two models. Alternatively, without fitting an additional model, you can perform what's called a Wald test on the entire set of coefficients involving the predictor. That takes into account those coefficients and their standard errors and the covariances among the coefficient estimates. That's the default for the Anova() function (note the capital "A") in the R car package and for anova() when applied to models built with the R rms package. Be careful in using the basic R anova() function on a single model, however, as its default can be misleading when the study isn't completely balanced.

EdM
  • 92,183
  • 10
  • 92
  • 267
  • Well essentially the same, depends on how you look at it. The p-value of the interaction stays the same, however the p-value of the main effects of interest swing all over the place. I can do a joint test but on which set of coefficients do I do it than? centered or uncentred? Or does this heavily depend on the specifics of my experiment (interacting variable has a sensible zero point, although none of the included samples has a value of zero for it. Making it centered around zero seems strange (the variable has to do with degradation due to decay, thus it seems weird to center around mean?) – Jobbe Goossens Feb 13 '23 at 11:41
  • As for the suggestion for Anova, I don't really now if it answers the question. The interacting variable would also point towards an effect off the main factor of interest, just one that is no longer clearly registrable due to issues with decay. Joint inference seems more logical but I struggle with choosing on which coefficient to do it. Furthermore: does joint inference on heavily multicolinear parameters solve anything or do the p-values stay very volatile/untrustworthy? – Jobbe Goossens Feb 13 '23 at 11:44
  • 1
    @JobbeGoossens Centering doesn't matter for combined tests on all coefficients involving a predictor. It never makes a difference for a nested likelihood-ratio test, comparing models with/without a predictor and all terms including it. A Wald test on all coefficients involving a predictor is invariant to linear transformations like centering/scaling. Multicollinearity often isn't a problem, illustrated here, as coefficient estimates can have offsetting covariances. – EdM Feb 13 '23 at 13:58