0

I'm running a logistic regression model in R using glmer() from lme4. I only want to get the random effects estimates, so I have an intercept-only model with just the random effect. The model is of the probability of subjects (which each have one observation, and are within a location, which is the random effect) getting an illness. Obviously I have the raw data to hand and can see the actual occurrences of illness within each location. After running the model, I used ranef() to extract the location random effects and converted them to odds and then probabilities. They are different - have a lower variance/SD/range - from the actual "probabilities" for each location in the raw data.

Perhaps I misunderstood how to interpret these random effects estimates, or how to convert them to probabilities, or maybe there is an issue with the model?

To be clear, I exponentiated each of the random effects intercept values, then converted them to probabilities with the usual formula odds/(1+odds). I also considered that the random effects estimates might not be the actual intercepts for groups but either the difference from the fixed intercept or odds ratios, but the probabilities from these calculations are even further away from reality.

DoggedGeddog
  • 1
  • 1
    What exactly do you hope to get out of a subject-level prediction without conditioning on any other covariates? The model would just predict ~1 where that's observed and ~0 otherwise, ignoring separability issues. The reason why the random effects are "smushed together" is because of shrinkage, see this question and in particular the second answer by Paul for a thorough but non-technical explanation on the difference between fixed and random effects. – PBulls Oct 18 '23 at 12:24
  • Because I just wanted to get a measure of the variation in the response that's explained by the random effect. There too many locations to model it as a fixed effect. While I had forgotten about shrinkage and understand its importance for estimating populations, what is the point of extracting intercepts for each group if they are not telling you the real group values? – DoggedGeddog Oct 18 '23 at 13:31
  • 1
    The whole point of the random effect is the shrinkage. They are not meant to reproduce the observed values exactly; they are meant to be biased estimates of the true values that are more precise because they borrow information from the other locations. If they reproduced the values exactly, you would need a parameter for each location, which is exactly the fixed effects model you seem to want to avoid. – Noah Oct 18 '23 at 17:15

0 Answers0