5

I know glmnet(x,y) generates $\lambda$ but I am very curious to know the actual formula that is behind this, generating $\lambda$.

Nick Cox
  • 56,404
  • 8
  • 127
  • 185
bison2178
  • 487
  • Are you talking about the function glmnet in the R package glmnet? – Glen_b Nov 17 '13 at 22:38
  • @Glen_b yes library(glmnet) in R – bison2178 Nov 17 '13 at 23:35
  • I discovered one thing I had missed before: here (scroll down to lambda.min.ratio) it says that the largest value of $\lambda$, lambda.max is the (data derived) entry value (i.e. the smallest value for which all coefficients are zero). Once lambda.max is given, obtaining the rest of the $\lambda$ sequence should be pretty simple. The question remains, how is lambda.max calculated? – Richard Hardy Feb 04 '15 at 20:46

2 Answers2

2

I had this same question and also ran into confusion in the F90 code in the glmnet package. In the end I took some code from the quadrupen package (at the end of quadrupen.R) and modified it for my purposes. I can confirm that the maximum lambda value produces all zero coefficients in glmnet with alpha=1. I'd love to hear better answers to this question or an implementation of the glmnet fortran version in R --- at least to help with teaching and learning.

### from quadrupen
## GENERATE A GRID OF PENALTIES IF NONE HAVE BEEN PROVIDED
get.lambda.l1  <-  function(xs,y,nlambda,min.ratio) {
  ##xs     <-  as(x, "dgCMatrix")
  ## currently not robust to missing values in xs or y
  ybar <- mean(y,na.rm=TRUE)
  xbar <- colMeans(xs,na.rm=TRUE)
  x      <-  list(Xi = xs@i, Xj = xs@p, Xnp = diff(xs@p), Xx = xs@x)
  xty    <-  drop(crossprod(y-ybar,scale(xs,xbar,FALSE)))
  lmax  <-  max(abs(xty))
  return(10^seq(log10(lmax), log10(min.ratio*lmax), len=nlambda))
}
Jake
  • 211
1

From the documentation, it seems that cross-validation is used on a self-generated sequence for lambda.

This results in the lambda.min being the lambda value in the sequence which produces the smallest cvm (mean cross-validated error) and lambda.1se being the largest lambda in the sequence such that error is within 1 standard error of the minimum.

There is some discussion and illustration in section 6 of the JStatSoft article

Henry
  • 39,459
  • 2
    Your answer addresses a different question than the one being asked. The question of interest is how the sequence of lambda is generated by the function glmnet in the glmnet package in R. I have been wondering about this question and checked the code behind the glmnet function, but ultimately faced some Fortran code (since the glmnet package is not entirely coded in R) which I had trouble with... I was unable to find the answer in the documentation nor in the JStatSoft article. – Richard Hardy Dec 05 '14 at 10:13