6

I am currently working with a multiple regression with three parameters; $\beta_1, \beta_2, \beta_3$. The individual T-test for each parameter contains large p values (>.05) while the p-value of the F-test for the null hypothesis that all coefficients are 0 is quiet small (approx. 0).

Is this suggestive of the issue suggested here of multicollinearity? And thus, would you reject or accept the null hypothesis, either for individual parameters or the model as a whole?

Bepop
  • 307

1 Answers1

6

It’s okay to reject the idea that all three parameters equal zero while not being able to pin down which of the three are nonzero. That is logically consistent and seems to be what is happening, statistically.

Dave
  • 62,186
  • Thanks! Would you say it makes sense to remove 1 or 2 predictors and repeat the regression? – Bepop Jul 07 '23 at 18:06
  • @Bepop Then you’re doing a form of stepwise regression, which can be highly problematic for the kind of inferential work you’re doing (could work well if you just care about predictions, however). Briefly, when you fit the model with the variables removed, your statistics all assume that you had not checked those variables and then removed them, rather than reflecting the reality of y workflow, and you can wind up being too optimistic with your claims. – Dave Jul 07 '23 at 18:15
  • Thanks again! This is slightly unrelated but do any extensions of Lin Reg (from variable selection to LASSO/Ridge techniques, etc) have mainstream inferential usage in statistics? For ex, I know Ridge regression doesn't really have the confidence intervals that OLS allows for – Bepop Jul 07 '23 at 18:45
  • 1
    @Bepop That really warrants its own posted question, one that I would expect to be well-received if not for it almost certainly being a duplicate (so a good question, just one that I suspect has been asked and answered). Working from memory, it seems that there has been work to give the usual inferences from regularized regression (maybe stepwise, too), but they are not in the mainstream of statistics (by which I mostly mean that I do not know any offhand). – Dave Jul 07 '23 at 18:50
  • Related: https://stats.stackexchange.com/questions/151403/significance-of-individual-coefficients-vs-significance-of-both/151410#151410 – Christoph Hanck Jul 11 '23 at 14:47