1

So today I've seen a lecture regarding multiple linear regression with response $y$, an intercept and two explanatory variables $x_1$ and $x_2$. That is, $$y_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\epsilon_i$$ We let $\hat{\beta}_0$, $\hat{\beta}_1$ and $\hat{\beta}_2$ be the least square estimators for $\beta_0$,$\beta_1$ and $\beta_2$.

So the lecturer said that we could reflect over the following: If the sample correlation of the two explanatory variables $x_1$ and $x_2$ is positive, i.e. $r_{x_1,x_2}>0$ then the correlation between $\hat{\beta}_1$ and $\hat{\beta}_2$ is negative and I just wonder how I can show it. Any suggestion would be helpful!

whuber
  • 322,774
Joey Adams
  • 181
  • Hint: apply a formula for the covariance between the $\hat\beta_i.$ It depends on the covariance matrix of the $x_i.$ https://stats.stackexchange.com/questions/68151 and https://stats.stackexchange.com/questions/135201 will get you started. – whuber Mar 29 '22 at 18:48
  • Hello whuber. Thank you for your reply. So the formula I need is $Cov(\hat{\beta})=\sigma^2(X'X)^{-1}$? – Joey Adams Mar 29 '22 at 18:59
  • 1
    Yes, that's the correct one for known $\sigma^2.$ Otherwise you would replace that by an estimate $\hat\sigma^2$--but evidently that little nicety won't change any of the signs of $\operatorname{Cov}(\hat\beta).$ Your problem is simplified by working with centered variables: that makes the covariance matrix $X^\prime X$ a $2\times 2$ matrix (with non-negative determinant) whose inverse has a simple, relevant formula. – whuber Mar 29 '22 at 19:07
  • But how can this be connected to the correlation between $\hat{\beta}_1$ and $\hat{\beta}_2$? I.e. how can this be negative? Sorry I might not understand the question. – Joey Adams Mar 29 '22 at 19:23
  • Hello I have the same question for a similar problem and want to understand this. What should be my design matrix? And is it just for $\hat{\beta}$? – Amir Hassan Apr 10 '22 at 12:16

0 Answers0