3

We have some nice posts (1, 2 and likely more) illustrating multicollinearity geometrically. Now, ridge regression ($L_2$ regularization) is known to be a remedy of multicollinearity. What is the geometric intuition for that?

We have a nice answer 3 that illustrates the geometry of the objective function and shows how a ridge under OLS becomes a peak under ridge regression. In addition to that, would it be possible to illustrate the geometry of fitting a plane in a 3D space (as in 1) with OLS vs. ridge when $X_1$ and $X_2$ are (almost) collinear? Or perhaps use yet another geometric perspective that provides intuition?

Richard Hardy
  • 67,272
  • Isn't https://stats.stackexchange.com/a/164546/919 good enough? – whuber Feb 08 '23 at 00:08
  • 1
    @whuber, I guess it is. Thank you! I had overlooked it in my search. When I am trying to picture it in my head, can I imagine the mean of $y$ is zero and the $X$ matrix does not contain a columns of 1s? (Otherwise I am having a hard time. It then seems that adding the ridge terms is making the best fitting hyperplane steeper if the mean of $y$ is positive.) – Richard Hardy Feb 08 '23 at 05:25

0 Answers0