Why does L2 regularization smooth the loss surface? went over my head.
Asked
Active
Viewed 257 times
1 Answers
1
You can think of minimising the MSE as finding an interpolating surface between the $(x_i, y_i)$ pairs. Regularisation is introduced to reduce the contribution of highly noisy single $ y_i $. From that perspective we'd expect the cost surface to become smoother, as we're biasing it be smoother!
Tom Kealy
- 115
-
1"we're biasing it be smoother"... how? I thought we're biasing weights to be smaller. very confused. – alwayscurious Feb 19 '18 at 17:22
-
smaller weights lead to a smoother function. How does it lead to a smoother cost function too? – alwayscurious Feb 19 '18 at 17:28
-
Could you make your question more specific please? – Tom Kealy Oct 02 '21 at 10:51