If there is a Elastic-net criterion function: $$\mathcal{L}(\boldsymbol{\beta}) = \frac{1}{2}\sum_{n=1}^N(\boldsymbol{\beta}^{\top}\boldsymbol{x}_n - y_n)^2 + \frac{1}{2}\lambda(1-\eta)\|\boldsymbol{\beta}\|_2 + \lambda\eta\|\boldsymbol{\beta}\|_1,$$ and $$\boldsymbol{\beta}_{opt} = arg\min\limits_{\boldsymbol{\beta}} \mathcal{L}(\boldsymbol{\beta}).$$ My question is : Is there an upper limit $M$ such that $$\|\boldsymbol{\beta}_{opt}\|_2 \leq M$$ and what is $M$ related to?
Asked
Active
Viewed 25 times
0
-
Since elastic-net is a shrinkage method, you might intuitively guess in a handwaving sense that the ordinary linear regression estimates gave an upper bound, approached when $\lambda$ was close to $0$ – Henry Apr 05 '23 at 07:22
-
I'm not sure there is an upper bound without some assumptions – Firebug Apr 05 '23 at 08:14
-
@Fire Observe that when $||\beta||_2^2\gt ||y||_2^2/(\lambda(1-\eta)),$ $\mathcal L(\beta)\gt \mathcal L(0).$ That gives a universal upper bound. – whuber Jun 07 '23 at 16:52