0

I have fitted Lasso and Ridge regressions on the same training data and having checked the training MSE error seems more-less the same:

from sklearn.linear_model import Ridge, Lasso
from sklearn.metrics import mean_squared_error

lasso = Lasso()
lasso.fit(X_train, y_train)


print(lasso.coef_, lasso.intercept_)
print(mean_squared_error(y_train, lasso.predict(X_train)))

Gives:

[-0., 0., -0., 0.07240139, -0., 2.78814019, -0., -0., -0., -0., -1.2595198, 0.13041602, -3.58735814] 22.862623762376256

28.099545911959265

And:

ridge = Ridge()
ridge.fit(X_train, y_train)

print(ridge.coef_, ridge.intercept_)
print(mean_squared_error(y_train, ridge.predict(X_train)))

Gives:

[-0.74031532, 1.19138487, 0.09161603, 0.7471897, -1.81315326, 2.62583585, -0.04210494, -2.95522009, 2.79949698, -1.97974729, -2.02004127, 0.87593085, -3.89141515] 22.86262376237627

21.691959974765535

I expected the coefficients of the respective models to somewhat correlate (with the cancellation of very small Ridge coefficients to be zero in Lasso) but this is apparently not the case. What could be the explanation?

Fredrik
  • 725
  • 5
    Why would you expect that? – Firebug Jan 24 '23 at 15:02
  • If there were any useful "correlation" in the results generally, the duplicate thread wouldn't even exist. – whuber Jan 24 '23 at 15:23
  • I expect correlation because both model works on the same way on the same data except the regularization (actually setting alpha=0.0 gives identical results as also expected). – Fredrik Jan 24 '23 at 15:29
  • So you already discovered a fact: the correlation between coefficients is inversely proportional to the penalization factor, unless some specific (trivial) conditions are at play – Firebug Jan 25 '23 at 09:07

0 Answers0