MLM in R: build up procedure, is it ok to add all predictors at once?

Question

I am a novice when it comes to MLM, but had to try my best for my master thesis. Hopefully you can help me with my following study:

I conducted a study with n = 56 participants (interpreters). The study offers longitudinal data with three measurement periods (baseline, follow up 1 after 6 months and follow up 2 after 12 months). The outcome variable is: Sum_PCL (PTBS symptoms), the predictor is TP (therapy participation) and the interactions I needed to add are (genderTP, flight experienceTP). My first hypothesis is whether therapy participation is associated with Sum_PCL. The second and third hypothesis tests whether gender and flight experience moderate the association between Sum_PCL and therapy participation.

My supervisor recommended to do MLM (2-level model with participants at level 2 and observations at the three measurement periods at level 1), so here are the R codes I came up with:

#Null model
Nullmodel <- lmer(Sum_PCL ~ 1 + (1 | Participant), data = data1, control = lmerControl(check.nobs.vs.nRE = "ignore"))
summary(Nullmodell)
#Random Intercept
RandomIntercept <- lmer(Sum_PCL ~ TP * measurement period + (1 | Participant), data = data1, , control = lmerControl(check.nobs.vs.nRE = "ignore"))
summary(RandomIntercept)
#Random Slope
RandomSlope <- lmer(Sum_PCL ~ TP * measurement period + (1 + measurement period | Participant), data = data1, control = lmerControl(check.nobs.vs.nRE = "ignore"))

—>the random slope model turned out too complex for the data so we stuck to the random intercept with fixed slope model (i got the following error: boundary (singular) fit: see help('isSingular'))

#Random Intercept - Adding Interactions
RandomInterceptInteraction <- lmer(Sum_PCL ~ TP * measurement period + TP * gender + TP * flight experience + (1 | Participant), data = data1, REML = FALSE, control = lmerControl(optimizer ='optimx', optCtrl=list(method='nlminb')))
summary(RandomInterceptInteraction)
anova(Nullmodel, RandomIntercept, RandomInterceptInteraction)

—> the random intercept interaction model turned out as the best overall model fit (likelihood ratio test), so it was taken for analysis

—> my question: is it ok to add all predictors / interactions at once? Or should I have added them separately in separate models? I found different opinions on the procedure (adding predictors / interactions at once or not). —> in the interaction model: do I have to add the single predictors (gender and flight experience) too as main effects in the code or is it ok just to add them in the interaction? I mean in the R output, the main effect will automatically be reported

Answer 1

The way that you wrote the R code, the "main effect" of each of the predictors in the interactions is automatically included in the model, as is proper practice in almost all situations. Note that each "main effect" will be reported at the baseline level of its interacting predictors, so changing the coding of a predictor will change the apparent "main effect" of each predictor with which it interacts. Thus, be wary of interpreting the "main effect" coefficients; focus on particular predictor combinations of interest.

In general, it's best to include as many predictors in the model as you can without overfitting. Frank Harrell's course notes and book go into much detail about how to formulate regression models. That allows evaluation of each predictor to take into account the values of all the other predictors, minimizing omitted-variable bias. That said, you might learn something about the relationships among the predictors and their associations with outcome by evaluating separate simpler models carefully against the full model.

With this simple experimental design you did not need to use a mixed model. The Fox and Weisberg appendix on multivariate models to "An R Companion to Applied Regression" shows how to analyze such data as a multivariate (multiple-outcome) repeated-measures regression. Harrell's notes and book linked above have chapters on generalized least squares for such longitudinal data. You could also consider the baseline outcome values as predictors and then only evaluate the post-intervention outcomes. There isn't anything necessarily wrong with how you proceeded, just different assumptions and tradeoffs.