1

enter image description here

Can anybody let me know the definition of these terms? I know we solve this for Beta but I want to have the definition

Ahmad Turani
  • 239
  • 2
  • 7
  • Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. – Community Feb 07 '22 at 15:00
  • Have a look at Introduction to Statistical Learning. Well explained there https://www.statlearning.com/ – Peter Feb 12 '22 at 20:59

1 Answers1

1

The function you are trying to minimize when using ridge regression consists of two parts, a standard loss function that describes how well the model fits the data (in your example this is the mean-squared error) and a penalty term. The penalty term is simply the summed absolute values of the model's parameters multiplied by lambda, which is a hyperparameter of the ridge regression. This describes how much of an impact the penalty term has on the total loss, a large lambda means that the model will likely favor a simpler model with smaller (in value) parameters. The model therefore has to make a trade-off between the two, improving the fit with regards to the data while making sure to keep the values of the parameters relatively small to prevent the penalty term from becoming too big.

Oxbowerce
  • 7,472
  • 2
  • 8
  • 23
  • Thank you so much. so what is n exactly as a term definition? SECOND TERM AS YOU SAID IS SUM OF THE MODEL FEATURES BUT WHAT IS N EXACTLY IN TERM DEFINITION? WHAT DOES IT SHOW? – Ahmad Turani Feb 07 '22 at 13:49
  • when you talk about penalty term that becomes too big, do you refer to overfitting ? – HelpNeederStudent Feb 07 '22 at 13:52
  • @AhmadTurani The n you see above the sum symbol simply means that the sum is taken over all of the parameters, i.e. parameter 1 to n. If the model has three parameters all three are squared and added together. – Oxbowerce Feb 07 '22 at 14:08
  • Yes yes @Oxbowerce. But what the n exactly is? for example parameters (data set numbers or ...)? – Ahmad Turani Feb 07 '22 at 14:09
  • @HelpNeederStudent Within this context I indeed had overfitting in mind, but I wouldn't say that a large penalty term always means that you are overfitting. There could be cases where the model can offset the large penalty term by a significant decrease of the MSE. – Oxbowerce Feb 07 '22 at 14:10
  • @AhmadTurani As mentioned in my comment, n simply refers to the number of parameters/coefficients in the model. – Oxbowerce Feb 07 '22 at 14:11