1

This question is a follow up to one of my previous questions asked on this site. The goal was to create a composite score for biomarkers related to a binary outcome and then use that in a regression to see if the composite score can significantly predict the outcome. I had 30+ biomarkers and I ended up selecting 4 of them which were bivariately ($p<0.10$) related to the outcome. I made a composite of these 4 biomarkers using ridge regression following the helpful answer by EdM. That way I could account for the natural correlation present among these markers and get adjusted $\beta$'s (adjusting for other biomarkers and covariates like age, sex, etc.). I had 109 complete observations. The coefficients look as follows:

> ridge.mod.bestlam <- glmnet(x, y, alpha = 0, lambda = 0.2387845, standardize = TRUE, intercept=TRUE)
> coef(ridge.mod.bestlam)
10 x 1 sparse Matrix of class "dgCMatrix"
                                s0
(Intercept)          -0.0252900970
Age                   0.0003756038
female                0.0603410625
Premorbid_depression -0.0338846415
antidep12             0.0556264177
nGCS_Bestin24         0.0135018439
log_med_IL_10         0.0530590200
log_med_ITAC          0.0478298328
log_med_sIL_6R       -0.0881823906
log_med_RANTES        0.0568835030

I multiplied the last 4 coefficients with the respective (scaled) marker values and obtained the composite score that I'd call ILS.ridge here. I used it as an input in a final logistic regression model. The odds ratio was 423.3499, extremely high. I must be doing something wrong but cannot figure it out. I checked the VIF and it was well below 1.5 for all variables. I also provide with the final regression results here.

glm(formula = nPTDCategory_m12 ~ Age + factor(female) + factor(nGCS_Bestin24) + 
    factor(Premorbid_depression) + factor(antidep12) + ILS.ridge, 
    family = "binomial", data = data2)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-1.0708  -0.6266  -0.4577  -0.2850   2.6085  

Coefficients:
                                Estimate Std. Error z value Pr(>|z|)   
(Intercept)                    4.5892763  2.6980108   1.701  0.08895 . 
Age                           -0.0008613  0.0170169  -0.051  0.95963   
factor(female)1                0.4465424  0.6081925   0.734  0.46282   
factor(nGCS_Bestin24)1        -0.0261555  0.6160321  -0.042  0.96613   
factor(Premorbid_depression)1 -0.7174396  0.8567616  -0.837  0.40238   
factor(antidep12)1             0.7393719  0.6429819   1.150  0.25018   
ILS.ridge                      6.0481991  2.3258686   2.600  0.00931 **

> exp(6.0481991)
[1] 423.3499

I'd like to know your thoughts about this problem. Can anyone tell if I'm doing something wrong?

Blain Waan
  • 3,595
  • 2
    This will not explain such a huge odds ratio (for that, first make sure the odds ratio is computed over a valid range of X such as its quartiles) but once you go with penalization you may need to stay within the penalized model and not use an unpenalized approach. – Frank Harrell Sep 03 '19 at 19:39
  • 2
    Based on the code that you show, your ridge.mod.bestlam seems to be based on linear rather than logistic regression with glmnet. If that's not an error in copying then that could contribute to your problem. I'm also curious what the distribution of ILS.ridge values was. As a continuous predictor its reported coefficient for the logistic regression would be for a change of 1 full unit, so if ILS.ridge only varies over a range of, say, +/- 0.01 then this result might make sense. As @FrankHarrell put it: "make sure the odds ratio is computed over a valid range of X such as its quartiles." – EdM Sep 03 '19 at 19:51
  • @EdM Thanks for spotting out my error. Thanks to Frank Harrell too. What a foolish mistake I made! I forgot to mention family="binomial". Now the odds ratio looks fine. – Blain Waan Sep 04 '19 at 02:57
  • 1
    For the benefit of those who might find this page later, why don't you post (and accept) an answer showing just what happened and how well things worked after you fixed the problem. You are not the first person to have a problem like this and you will not be the last. It's fine to post and accept an answer to your own question on this site, and doing so may save someone else the headaches that you just had. – EdM Sep 04 '19 at 14:44
  • Okay, thanks for the suggestion. Yes, it will be helpful for others. – Blain Waan Sep 05 '19 at 21:53

1 Answers1

0

As suggested by EdM in the comments, I post here an answer to help others who have similar problems. I used family="binomial" while finding the best $\lambda$ by k-fold cross-validation. But forgot to add it when running the model again with the chosen $\lambda$. For my case $\lambda=0.2387845$.

The following codes give a stable odds ratio.

> ridge.mod.bestlam <- glmnet(x, y, family="binomial", alpha = 0, lambda = 0.2387845, standardize = TRUE, intercept=TRUE)
> coef(ridge.mod.bestlam)
10 x 1 sparse Matrix of class "dgCMatrix"
                               s0
(Intercept)          -3.393086872
Age                   0.001080965
female                0.270751918
Premorbid_depression -0.124371600
antidep12             0.237535918
nGCS_Bestin24         0.104369776
log_med_IL_10         0.235349603
log_med_ITAC          0.235589152
log_med_sIL_6R       -0.350081857
log_med_RANTES        0.284487664

After this, I create the composite ILS.ridge and use it in the GLM. 

glm(formula = nPTDCategory_m12 ~ Age + factor(female) + factor(nGCS_Bestin24) + 
    factor(Premorbid_depression) + factor(antidep12) + ILS.ridge, 
    family = "binomial", data = data2)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-1.0516  -0.6327  -0.4368  -0.2928   2.6054  

Coefficients:
                                Estimate Std. Error z value Pr(>|z|)   
(Intercept)                    1.6434409  1.6416256   1.001  0.31678   
Age                           -0.0009967  0.0170669  -0.058  0.95343   
factor(female)1                0.4294085  0.6102366   0.704  0.48163   
factor(nGCS_Bestin24)1        -0.0431719  0.6181712  -0.070  0.94432   
factor(Premorbid_depression)1 -0.7156341  0.8591330  -0.833  0.40486   
factor(antidep12)1             0.7141079  0.6423839   1.112  0.26629   
ILS.ridge                      1.3835981  0.5207700   2.657  0.00789 **

Now, the odds ratio looks stable. 

> exp(1.3835981)
[1] 3.989229
Blain Waan
  • 3,595