2

I've seen a lot of questions about why the sum of squares for the residual follows a chi-squared distribution. I have the same question about the sum of squares for the regression.

I have the following regression model: $y_i=\beta_0+\beta_1x_{1i}+...+\beta_kx_{ki}+\epsilon_i$

Then the sum of the squares for the regression is: $SS(Regression)=\sum(\hat{y_i}-\bar{y})^2$

Where $\hat{y_i}$ is the expected value for $y_i$ given $x_{1i},...,x_{ki}$ and $\bar{y}$ is the mean value for $y$

Now I do not see how SS(Regression) is the sum of squared normally distributed random variables. Really, I don't even see that there are any random variables in this sum! Any help is much appreciated.

[I found 1 question asking the same thing, but without any answers: Degrees of freedom in regression analysis ]

kjetil b halvorsen
  • 77,844
Alison
  • 21
  • 2
  • 1
    What are you assuming about $\epsilon_i$ if you don't see any random variables in the sum? (Bear in mind that both $\bar y$ and $\hat y_i$ are linear combinations of the $y_i,$ whence they are affine combinations of the $\epsilon_i.$) BTW, did you omit a crucial "not" after "now I do"? – whuber Sep 24 '19 at 11:57
  • Thank you for the note on the typo, I've corrected it. – Alison Sep 24 '19 at 13:38
  • Isn't it true that $\hat{y_i}=\beta_0+\beta_1x_{1i}+...+\beta_kx_{ki}$ in which case epsilon doesn't occur in the term and the mean obviously is just a constant? – Alison Sep 24 '19 at 13:41
  • 1
    No, that's not how the fitted values are defined. $\hat y_i = \hat\beta_0 + \hat\beta_1 x_{1i} + \cdots + \hat\beta_k x_{ki}$ and the estimated coefficients $\hat \beta_j$ are all linear combinations of the $y_i.$ – whuber Sep 24 '19 at 13:58
  • 2
    @Alison SSReg doesn't follow a chi-squared distribution. A suitably scaled SSReg would (under appropriate assumptions). – Glen_b Sep 24 '19 at 14:12
  • @Glen_b What would such assumptions be? – Alison Sep 24 '19 at 20:13
  • 2
    The usual assumptions that apply when doing normality-based tests on regression; including i.i.d normal errors – Glen_b Sep 24 '19 at 21:44
  • For simple linear regression, see https://stats.stackexchange.com/q/513284/119261. – StubbornAtom Apr 19 '22 at 19:27

0 Answers0