1

I'm new to stepwise regression and I've been asked to conduct one for my boss. In doing so, they also asked for the effect sizes from each predictor in the model. Disregarding any debates around whether or not one should conduct a stepwise regression (remember, this is for my boss), is it possible to get the requested effect sizes?

Here is the model that I am working with:

faModel <- lm(respTime ~ separation + exp + field, data=faM)

response time is measured in seconds
separation is continuous
exp and field are categorical with 3 and 3 levels respectively and were converted to factors.

Here is the stepwise regression code:

faP <- ols_step_both_p(faModel, penter= 0.05, prem = 0.05, 
        details = TRUE)

Note the ols_step_both_p() is from the olsrr package.

Here is an image of the final model output:

enter image description here

So you can see that R-squared for exp is 0.274 and the final R-squared with field is 0.286, which suggests that the field R-squared is 0.012 (I think?). So can I use the formula: f = R-squared/(1-R-squared) to find the final model effect size? And the same formula for each predictor included in the model (e.g., 0.274 and 0.012)?

john connor
  • 113
  • 3
    Stewing aside the legitimate claims that stepwise regression is pants, why wouldn’t you calculate the effect size the exact same way as you would if you estimated the stepwise-selected model from the beginning. // It’s important to be able to tell people when they are using incorrect methodology, rather than just knuckling under to any bad idea someone suggests. See the last paragraph of my answer here. – Dave Sep 13 '22 at 21:01
  • @Dave Please refer to my third sentence as I am not posing a debate, rather I am asking for a helpful response. – john connor Sep 14 '22 at 00:29
  • Please tell us (as an edit to the post) from where is the function ols_step_both_p – kjetil b halvorsen Sep 14 '22 at 00:40

1 Answers1

3

Given that you have to use stepwise, why not bootstrap it? See for instance How to evaluate collinearity or correlation of predictors in logistic regression?

Then, as @Dave said in a comment, calculate effect sizes the way you would do it with a prespecified model, but repeat it in every bootstrap resample. The same you can do with other fit statistics. The variability between resamples will give some indication of the reliability of the effect size estimates.

kjetil b halvorsen
  • 77,844