Questions tagged [stepwise-regression]

Stepwise regression (often called forward or backward regression) involves fitting a regression model and adding or removing predictors based on $t$ statistics, $R^2$ or information criteria to arrive in a stepwise manner at a final model. This tag can also be used for forward selection, backward elimination & best subsets variable selection strategies.

Note that performing inferential statistics via $p$-values after stepwise regression is invalid unless the $p$-values have been adjusted to account for the stepwise model building step.

330 questions
3
votes
1 answer

Clarification-Forward stepwise regression

I'm learning about forward stepwise and there are some things which are not so clear: If I have $p$ predictors, is it true that forward stepwise does $p$ iterations? If I add the predictors in each iteration is it true that I eventually converge to…
Ville
  • 857
2
votes
2 answers

Does the number of IV after stepwise model selection depend on the amount of data?

I have 3 different DV that I try to model with 3 distinct models (linear mixed models) using the same set of IV. I found that the DV that I have the least amount of data for also has the lowest number of IV's in the final model, while the DV that…
Frize
  • 115
1
vote
1 answer

Co-linearity/Stepwise regression

I was wondering if we need to check the multicollinearity between variables when we run the stepwise regression, If yes, why? Also, If we have several IV, how we should know which interaction we should include in the model?
1
vote
0 answers

Forcing variable selection to keep certain predictor in R

I am looking for a variable selection technique in R to reduce the number of my regression predictors, where I can force the method to keep a specific variable within the model. Here is a toy example from the R help of ?step, the variable…
Marco
  • 29
0
votes
3 answers

How can I do Stepwise Method if only independent variables are given?

This is the data set How can I determine if this 4 groups differed significantly in their performance
0
votes
1 answer

Confusing stepwise regression process

According to the algorithm for the backward stepwise selection from the book ISLR which is shown below: says that we need to choose the model among the $k$ models by having a smallest RSS or highest $R^{2}$, while other resources tells that for…
isemaj
  • 207
0
votes
1 answer

Why stepwise feature selection method do not perform well when there is a large number of explanatory variables

I have the feeling that when I have a large number of predictors, it will be better to use feature selection regression model such as lasso, to fit the model, and better not use the stepwise feature selection method to select important predictors.…