In linear regression, why are raw least squares residuals heteroskedastic?

Question

In my course notes on a regression course with regards to the detection of heteroskedasticity there's the following quote:

"Because the least-squares residuals have unequal variances even in the homoscedastic case, it is preferable to use the standardized residuals."

My intuition tells me that since the LS regression line necessarily goes through the center of the datacloud, it will be a better fit for points in the middle of the covariate space than on the tails, thus giving us larger variance on the extremes.

Despite this, this does not seem like it's necessary. And at the same time I wonder about why do we care for homoscedasticity on standardized or studentized residuals and not for the raw ones.

Answer 1

Assuming the usual linear model with constant variance $\sigma^2$. I will use notation (and some results) from Leverages and effect of leverage points. The linear model in matrix form is $$ Y= X\beta + \epsilon $$ where $\epsilon$ is a vector of $n$ iid error terms. Then the hat matrix is $H=X(X^TX)^{-1}X^T$, and its diagonal terms are the leverages $h_{ii}$. We can show that the variance of the residuals $e_i = y_i-\hat{y_i}$ is $\sigma^2 (1-h_{ii})$ (remember $0<h_{ii}<1$.)

So, under this model, to get constant-variance residuals we divide by $\sqrt{1-h_{ii}}$: the standardized residuals defined by $r_i=\frac{y_i-\hat{y}_i}{\sqrt{1-h_{ii}}}$ have constant variance. So for many uses in residuals analysis we prefer this standardized residuals, for instance in checking the assumption of constant variance.

EDIT

In a comment the OP writes:

As far as I know the formal assumption is not "homoscedasticity of standardized residuals", but only residuals by itself.

This confuses errors with residuals. The errors are the unobserved $\epsilon_i$ in the regression equation $y_i =\beta_0 +\sum_i \beta_i x_i +\epsilon_i$, while residuals is the observed difference between observation and model prediction. Homoskedastcity means that the the errors all have the same variance, not that the residuals have constant variance. If you want to use residuals to test/critizise the constant variance assumption, it is better to use a version of the residuals that do have constant variance (under the model.)

Answer 2

Suppose you have three $x$-values: $-1,0, +1.$

The corresponding dependent variables $Y_1,Y_2,Y_3$ are where the randomness is.

Now draw the picture. You can see why, if you move $Y_2$ up or down, the fitted line moves up or down. (By just $1/3$ as much as $Y_2$ moves.) But what happens if you move $Y_3$ up or down? The fitted line doesn't just move up or down; its slope also gets bigger or smaller. Or if you move $Y_1$ up or down, then the slope gets smaller or bigger, respectively. So the line has more tendency to stay close to the data point when the data point's $x$-value is far from the average $x$-value than when it is near the average $x$-value. Hence the observed residuals have a smaller variance when the $x$-value is far from the average $x$-value than when the $x$-value is close to the average $x$-value.

The fitted values are \begin{align} & \left(\widehat Y_1, \widehat Y_2, \widehat Y_3\right) \\[5pt] = {} & \left( \tfrac 2 3 Y_1+ \tfrac 1 3 Y_2, \,\,\, \tfrac 1 3 (Y_1+Y_2 + Y_3), \,\,\, \tfrac 1 3 Y_2 + \tfrac 2 3 Y_3 \right). \end{align} So the residuals are \begin{align} & \left( Y_1, Y_2, Y_3 \right) - \left(\widehat Y_1, \widehat Y_2, \widehat Y_3\right) \\[5pt] = {} & \left( \tfrac 1 3 Y_1 - \tfrac 1 3 Y_2, \,\,\, -\tfrac 2 3 Y_1+ \tfrac 2 3 Y_2 - \tfrac 2 3 Y_3, \,\,\, -\tfrac 1 3 Y_2 + \tfrac 1 3 Y_3 \right). \end{align} From this one can compute the variances of the residuals.