4

I was wondering if there exists an option to avoid reference classes in logistic regression by transformation estimaters (especially the intercept)? Normally the intercept contains the information of all reference classes. Is it possible to get three estimators for a variable with three categories?

It would be very much appreciated if someone has got an example.

Edit: I've read about models using intercept and having estimaters for each class.

kjetil b halvorsen
  • 77,844
T. Beige
  • 307

2 Answers2

5

Normally, with three categories, you will obtain an intercept, reflecting the log odds of the outcome in the reference category, and two effect terms, indicating how the log odds for the other two categories differ from the reference. You can change the contrasts used in the model to change how this information is encoded (see https://stats.oarc.ucla.edu/r/library/r-library-contrast-coding-systems-for-categorical-variables/ ).

However, what I think you're looking for is just an estimate of the log odds for every category, or in other words, an intercept for each category. This is done by suppressing the overall intercept, using the syntax outcome ~ -1 + predictor in R.

Update

As mentioned in the comments, it is technically possible to fit a model where with an intercept and effects for each category. However, this model is not identified, since there is no unique best-fitting set of parameters. This model can be estimated, though, using regularisation - applying a small penalty to the weights (but not the intercept) such that parameters with smaller squared magnitudes are preferred (this can also be achieved using something called the Moore-Penrose pseudoinverse; see here). This yields a unique solution: the intercept is the average of the log odds across categories, while the effects for each category indicate how much each category differs from the average. It should be easy to see why this solution has the smallest squared parameter sizes (again, not counting the intercept).

Again, though, this is an advanced topic, and unless you really know what you're doing, you probably don't want this! I only know of this approach because some people use it to analyse fMRI data using the popular but idiosyncratic SPM software.

Eoin
  • 8,997
  • Yes, there is the option to get a logistic regression without intercept, but I read about a model with intercept but with estimators for each category and I can't image how that works. – T. Beige May 20 '22 at 08:15
  • Indeed, that model would be under-identified. The model with intercept 0 and category betas 2, 4, and 6 gives exactly the same predictions as the model with intercept 1 and betas 1, 3 and 5. – dash2 May 20 '22 at 08:26
  • 4
    Technically this is possible, using regularisation or the pseudoinverse, but it's an advanced topic and not useful under most circumstances. I'll try to edit the response later. – Eoin May 20 '22 at 10:16
5

This question has nothing to do with logistic regression per se, the problem and answers is the same for all generalized linear regression models. If you have only one categorical variable, just leave out the intercept, with more than one that will not be useful. See the earlier posts for details:

But, however you parametrize your model (there will be many ways, no perfect), you can always just test the contrasts of interest after the fit!

kjetil b halvorsen
  • 77,844