0

According to the algorithm for the backward stepwise selection from the book ISLR which is shown below:

enter image description here

says that we need to choose the model among the $k$ models by having a smallest RSS or highest $R^{2}$, while other resources tells that for choosing the predictor/s to be included in the model based on their $p$-values as stated here.

Can someone tell what is the right process?

isemaj
  • 207

1 Answers1

0

These are two different methods, each with advantages and disadvantages.

The method in ISLR is more computationally intensive. It fits 1+p(p+1)/2 models, where p is the number of predictors. The other backward selection method fits at most p+1 models.

On the other hand, the method in ISLR is more statistically sound. In general, the practice of comparing p-values to each other is frowned upon by statisticians. See principle number 5 in the ASA's Statement on p-Values: https://www.tandfonline.com/doi/pdf/10.1080/00031305.2016.1154108?needAccess=true