1

When you penalize logistic regression using l1, l2 or both penalizations, the coefficients are penalized towards 0. I would like to do the same thing but penalizing the coefficients towards other different numbers. For instance, penalize the first coefficient towards 1, the second towards 10 and the third towards -3.

Note that I am aware of this question and its answer, but it wouldn't quite work in a logistic regression scenario as substracting some combination of the predictors to the target variable doesn't give a binary variable, therefore the transformed regression is not a logistic regression anymore.

If somebody finds an R package or a way to do this with glmnet I would appreciate it a lot, thanks.

kjetil b halvorsen
  • 77,844
David Masip
  • 247
  • I think one could replace the weighted least square step in the IRLS algorithm (cf algo here https://stats.stackexchange.com/questions/178492/which-optimization-algorithm-is-used-in-glm-function-in-r) by a nonzero-centered ridge regression (see https://www.sciencedirect.com/science/article/abs/pii/S0047259X21001603 and refs therein), which can be obtained by row-augmenting the covariate matrix sqrt(W)X with a diagonal matrix with sqrt(lambdas) along the diagonal and adding a vector center sqrt(lambdas) to the outcome variable z * sqrt(W) in each iteration? – Tom Wenseleers Jun 11 '23 at 07:40

1 Answers1

3

This will depend on the software used, and with R possibly with the package. But the concept that you need is that of an offset. An offset is a variable which you include in a linear predictor with a known, given constant coefficient. In R modeling functions which accepts a formula argument, it can be given as y ~ offset(x) which uses x in the linear predictor with an offset of 1. So, if you want that offset to be 3, use y ~ offset(3*x). If you want regularization against 3, you would need y ~ x + offset(3*x).

But glmnet do not accept a formula argument, but it has an extra optional argument offset which can be used likewise. So read ?glmnet. But there is a contributed package glmnetUtils which offers a formula argument, so I would use that package.

kjetil b halvorsen
  • 77,844
  • Ok, I get it, and I guess that y ~ x + offset(3*x) doesn't imply any collinearity problems among the predictors, as it deals with offsets in a different way? – David Masip Jan 04 '19 at 12:19
  • No collinearity problems, as it is just an additional constant in the linear predictor. – kjetil b halvorsen Jan 04 '19 at 12:25
  • Are you sure this is the best way? Can one not replace the weighted least square step in the IRLS algorithm (cf algo here https://stats.stackexchange.com/questions/178492/which-optimization-algorithm-is-used-in-glm-function-in-r) by a nonzero-centered ridge regression (see https://www.sciencedirect.com/science/article/abs/pii/S0047259X21001603 and refs therein), which can be obtained by row-augmenting the covariate matrix sqrt(W)X with a diagonal matrix with sqrt(lambdas) along the diagonal and adding a vector center sqrt(lambdas) to the outcome variable z * sqrt(W) in each iteration? – Tom Wenseleers Jun 11 '23 at 07:39