2

I have a panel data set I'm using to model a binary response. I have developed 3 alternative models:

  1. A simple logistic regression, pooling all the observations
  2. A conditional logistic regression, stratifying the data by individual (I understand this is the binary equivalent of a fixed effects model)
  3. A random effects logistic regression, with a random intercept for each individual

The purpose of these models is to establish whether there is a significant trend over time. They do not agree.

With a continuous response and true fixed and random effects models, I could use the Hausman test to choose between them. Is there an alternative I can use with these binary response models? In particular in R...

There is a similar question on Cross Validated but the answer basically says "always use random effects". There's a question on SO too which adapts a function from plm but has a major disclaimer and I rather suspect it's not a valid approach.

Tom Wagstaff
  • 245

1 Answers1

3

If you are trying to do inference, your model should be based on theory, or at least on reasonable assumptions that you can justify in your context.

If you are trying to do prediction, you can probably get just as good predictions with xgboost or something where you don't have to think about random effects structures at all.

There are, in theory, a few suggestions on how to do this (see e.g., this thread and especially the GLMM FAQ) like using AIC to compare models, but not everyone agrees on them. There are some debates about the best way to do this, and not everyone agrees whether you should select fixed effects or random effects first (it's hard to do both at the same time). In short there's no canonical automatic process for this, which is why it's best to base the random effects structure on theory.

wzbillings
  • 693
  • Thanks so much @wzbillings, good to see there's not a trivial answer to this at least! It's an inference job really, so theory comes to the fore, but I'm not sure what a sensible assumption would be for this. I certainly expect unobserved individual heterogeneity to be important and worth controlling for, but not sure about FE vs. RE. To motivate RE, my individuals are drawn at random from a larger population, so it seems reasonable to see their intercepts as draws from some distribution. But I hear FE demands fewer assumptions at the cost of less precision, which does not seem to be my case.. – Tom Wagstaff Oct 19 '22 at 19:10
  • 1
    I don't think fixed effects makes fewer assumptions, I just think it makes different assumptions :). I would probably always use random effects to control for differences across individuals -- IMO this should be the default for experimental units like individuals (or study centers, or anything like that). At least to me it is much easier to justify individuals having random intercepts than justifying why they shouldn't. – wzbillings Oct 19 '22 at 20:38