When should you use L1 vs. L2 regularization?

Asked Nov 01 '22 at 22:41

Active Nov 01 '22 at 22:41

Viewed 63 times

Can't seem to find a good explanation online of concrete examples of where you would use one over the other?

I also read somewhere that L1 is supposedly slower than L2, but not sure how that is since L2 involves squaring and L1 is just taking the absolute value.

asked Nov 01 '22 at 22:41

Katsu

Pretty big question so oversimplifying things for a comment here: L1 helps with explanatory variable selection while L2 helps with explanatory variables' covariances. L2 has a closed-form solution too so it can be often much faster to compute. Be without much context I would suggest using the "cheat code" and perform elastic-net. :) – usεr11852 Nov 02 '22 at 00:51
What do you mean by 'L2 helps with explanatory variables' covariances'? – Katsu Nov 02 '22 at 01:13
When we have collinearities and/or explanatory variables that are somehow interconnected, L2 helps us get more robust (and "shrunk") coefficients. – usεr11852 Nov 02 '22 at 01:16
Not sure this is a duplicate, but maybe… – Dave Nov 02 '22 at 01:19
2

Actually, this is a duplicate of When should I use lasso vs ridge? – usεr11852 Nov 02 '22 at 04:42
1

If I remember correctly, Tibshirani's original paper where he proposed lasso contains some illuminating comparisons with ridge. – Richard Hardy Nov 02 '22 at 07:43

When should you use L1 vs. L2 regularization?

0 Answers0