0

I know that the hat matrix $H = X(X^T X)^{-1} X^T$, and that $\hat{Y} = HY$. When we have some non-zero constants that we multiply each respective predictor by, which just multiplies every column in the data matrix $X$ by the respective constant, the hat matrix stays the same. And so $\hat{Y}$ is the same as well. How can I prove that the hat matrix does not change?

I was told to think of a $p\times p$ matrix $V$ where the diagonal entries are the constants, and to make the new data matrix $XV$, and calculate the hat matrix like this, but I am still confused how to start.

develarist
  • 3,917
  • 1
  • 21
  • 52
Quinky
  • 3

1 Answers1

1

You are on the right direction. If you replace $X$ by $XV$, and remember a few properties (specifically, you need $(AB)^{-1}=B^{-1}A^{-1}$, $(AB)^{T}=B^{T}A^{T}$ and $V=V^T$) you get:

$$\begin{align} H'&=XV(VXX^TV)^{-1}VX^T\\ &=XVV^{-1}(XX^T)^{-1}V^{-1}VX^T\\ &=X(XX^T)^{-1}X^T\\ &=H \end{align}$$

PedroSebe
  • 2,680
  • 9
  • 17
  • Thanks a lot for your help. Would changing X like this change RSS or $\widehat{\beta_{i}}$ ? Since RSS = $y^{T}[I-H]y$ , I'm assuming it wouldn't change since our H was unchanged, but i'm a bit more confused about our betas. Does that have to change for our y to remain unchanged? – Quinky Oct 02 '20 at 03:57
  • @Quinky, see e.g. https://stats.stackexchange.com/questions/348758/coefficient-of-determination-invariant-to-centering-and-rescaling-of-variables/348780#348780 https://stats.stackexchange.com/questions/208341/scaling-in-linear-regression/208344#208344 https://stats.stackexchange.com/questions/188715/how-do-regression-results-change-after-standardization-as-a-general-rule/188736#188736 for related posts – Christoph Hanck Oct 02 '20 at 06:43
  • @Quinky you are correct that the RSS won't change, since the hat matrix stays the same. The betas will change though: if you take the matrix expression of $\hat\beta$ and replace $X$ by $XV$, you will see that $\beta'=V^{-1}\beta$. That makes sense: if a predictor is scaled by a factor $v$, the only way we can keep $\hat y$ unchanged is by scaling that coefficient by $v^{-1}$. – PedroSebe Oct 02 '20 at 15:57