Efficiently add a new predictor to an estimated multiple linear regression model

Question

I want to use forward selection to choose predictors in a multiple linear regression model. If you have a regression with N predictors and want to add another predictor, is there a way to update the coefficient estimates that is more efficient than just estimating a new regression from scratch?

In R, the update function performs such a job, although I do not know if it does so with special regard for efficiency. In any case, you may want to inspect that function's code. — Christoph Hanck, Jun 15 '18 at 14:43
The update function do not even try to be fast, it just updates the model formula and reruns. — kjetil b halvorsen, Jun 15 '18 at 15:15
You should not use forward selection. It may help you to read my answer here: Algorithms for automatic model selection. — gung - Reinstate Monica, Jun 18 '18 at 14:28

score 1 · Answer 1 · answered Jun 17 '18 at 14:36

The least-squares solution to X*b = Y is given by the normal equations

X′X b = X′Y

Adding another predictor is equivalent to adding a column to X. Given the old matrix of cross-products, to get the new X'X you only need to calculate the dot product of the new column with the old columns. This saves time compared to recomputing X'X from scratch.

Efficiently add a new predictor to an estimated multiple linear regression model

1 Answers1