Elastic Net Regression in R formula

Question

This is not a question of how to do Elastic Net Regression in R, but understanding the objective function of Elastic Net Regression from the glmnet package.

From the package itself the objective function is as follows found here: $$\min_{B_0,B}\frac{1}{N}\sum{w_il(y_i,B_0+B^Tx_i)+\lambda[\frac{(1-\alpha)}{2}||B||_2^2+\alpha||B||_1]}$$.

So the summation is the sum of the squared residuals and the $\lambda$ is penalization from Lasso and Ridge.

Then it states for the Gaussian case, for $l(y,n)$, it becomes $\frac{1}{2}(y-n)^2$. This simplifies the summation such that instead of multiplying by $\frac{1}{N}$ it becomes $\frac{1}{2N}$ as suggested link1 and link 2. However, in this link, there is no division by N to begin with in the objective function of elastic net regression.

My questions are

1. Why is there a difference such that one divides by the number of observations and the other does not?
2. What is $w_i$ in the objective function?

Answer 1

1. This difference is simply a matter of how the penalty $\lambda$ is scaled. Note that if you multiply the objective function as you wrote it by $N$ you would then have a penalty of $N \lambda$ instead. Either way can work at a fixed sample size. The way you wrote it has the potential advantage that the penalty term is independent of sample size.

2. The $w_i$ are case weights. Sometimes in regression it's useful not to weight all observations equally in an analysis, for example if some observations are known to have less reliability. The $w_i$ provide for that possibility.