How to choose between linear mixed model and GLS

Question

My study is a randomized clinical trial. How to choose between linear mixed model and GLS Linear Model Using Generalized Least Squares? What are the advantages or disadvantages of both? I have some responses continuous, some binary. My understanding GLS does not apply to binary outcomes. Thanks

Answer 1

I summarize this in Chapter 7 of RMS course notes. The main advantage of mixed effects models is that they apply to a wide variety of models include models for categorical Y. Disadvantages include computational issues, interpretation, and problems arising from using the default correlation structure. If you only put random intercepts in the model you are assuming compound symmetry which doesn't fit well for serial data especially over long time spans. We tend to see things like AR(1) serial correlation much more often than compound symmetry.

A generalization of GLS thinking that applies easily to all types of models is Markov processes. Details including detailed case studies may be found here.

One philosophy you might use in selecting an approach is this: If you are interested in estimating subject-specific effects, use a mixed effects model. If your interests are at the group level (e.g., how to subjects on treatment B compare with those on treatment A) then use a marginal (i.e., not conditional on subject) model such as GLS or a Markov model.

Nowadays Markov models are my go-to longitudinal models cause they can easily be semiparametric to handle ordinal Y or continuous Y with a strange distribution. The R Hmisc package has functions for unconditioning on previous states for early computing state occupancy probabilities from state transition probabilities. This works particularly well with Bayesian semiparametric models as in the R rmsb package. Again you can find fully worked out examples and simulations with R code here.