1

Apologies for the ambiguous description as I can't remember exactly it's called or the exact details. When I think about OLS, I typically think about regressing $y$ on top of the predictors simultaneously.

The other approach that I am alluding to is, IIRC, where you do some kind of partial regression on a subset of the predictors, and then you're left with partial residuals, and then your regress the partial residuals on the remaining predictors. I may be remembering this incorrectly, but I vaguely recall it was something like this.

What is the name of this strategy? I would like to read about it in more detail.

David
  • 705
  • Are you thinking about the FWL theorem (https://en.wikipedia.org/wiki/Frisch%E2%80%93Waugh%E2%80%93Lovell_theorem)? That sounds closest to the second paragraph in the question body. I am not familiar with a procedure that regresses residuals on other residuals as stated in the question title – wahid Nov 26 '23 at 03:44
  • @wahid hmm this seems somewhat related but I don't think this is what I recall reading about. let me see if I can find the post on here where I saw what I alluded to in the OP – David Nov 26 '23 at 04:24
  • I think I found one source: https://stats.stackexchange.com/questions/28474/how-can-adding-a-2nd-iv-make-the-1st-iv-significant whuber's answer – David Nov 26 '23 at 04:25
  • Way back when I was a student reading about regression algorithms, this approach was called Gram-Schmidt orthogonalization. There's a few extra little bits added on to actually calculate a regression. – Glen_b Nov 26 '23 at 07:10
  • I'm guessing you are referring to two stage least squares, where an initial model is estimated on the IVs to DVs, then those residuals are regressed on the response of interest. But I could be wrong there. – Shawn Hemelstrand Nov 26 '23 at 11:55
  • I have called it "sequential matching," borrowing terminology from Tukey & Mosteller. Gram-Schmidt is a general algorithm (for creating sequences of orthogonal bases in Hilbert spaces), as are the closely related QR decomposition and SVD. – whuber Nov 26 '23 at 16:09

0 Answers0