3

Using the simple linear regression model: $ {y_i}= {\beta_0} + {\beta_1}x_i + \epsilon_i$, where E[$\epsilon_i$]=0 and var[$\epsilon_i] = \sigma^2$...

If $ \hat{y_i}= \hat{\beta_0} + \hat{\beta_1}x_i$ and

$ \hat{y_i^*}= \hat{\beta_0} + \hat{\beta_1}x_i^*$

(where $\hat{\beta_0}$ and $\hat{\beta_1}$ are ordinary least squares estimates of $\beta_0$ and $\beta_1$)

how would I go about deriving $\text{cov}$$(\hat{y_i}, \hat{y_i}^*)$?

Here is what I have been thinking:

$\text{cov}$$(\hat{y_i}, \hat{y_i}^*) = cov(\hat{\beta_0} + \hat{\beta_1}x_i, \hat{\beta_0} + \hat{\beta_1}x_i^*) = \beta_1^2 cov(x_i, x_i^*) = \beta_1^2[E[(x_i - E(x_i))(x_i^* - E(x_i^*))]] $

If the work that I've done is correct, is there any more steps that I can take?

stats566
  • 45

1 Answers1

5

The trick here is to realize that $x_i$ and $x_i^*$ are in this case constants.

$cov(\hat{y}_i, \hat{y}_i^*) = cov(\hat{\beta_0} + \hat{\beta}_1 x_i, \hat{\beta_0} + \hat{\beta}_1 x^*_i) = cov(\hat{\beta_0},\hat{\beta_0}) + cov(\hat{\beta_0},\hat{\beta_1}x_i^*) + cov( \hat{\beta}_1x_i,\hat{\beta_0}) +cov(\hat{\beta}_1x_i,\hat{\beta_1}x_i^*) = var(\hat{\beta_0}) + x_i^*cov(\hat{\beta_0},\hat{\beta_1}) + x_icov( \hat{\beta}_1,\hat{\beta_0}) + x_ix_i^*var(\hat{\beta}_1) = var(\hat{\beta_0}) + cov(\hat{\beta_0},\hat{\beta_1})(x_i+x_i^*) + x_ix_i^*var(\hat{\beta}_1) $.

Now the rest should be easy as the $var(\hat{\beta}_0),var(\hat{\beta}_1),cov(\hat{\beta}_0, \hat{\beta}_1)$ are all well known. See here the exact variance/covariance values: How to derive variance-covariance matrix of coefficients in linear regression

Edit:

In response to your comment, it really is a matter of taste how far you take this from now on. I would proceed as follow:

$var(\hat{\beta_0}) + cov(\hat{\beta_0},\hat{\beta_1})(x_i+x_i^*) + x_ix_i^*var(\hat{\beta}_1) = \frac{\sigma^2 \sum x_i^2}{n\sum (x_i-\bar{X})^2} + (x_i+x_i^*)\frac{-\bar{X}\sigma^2}{\sum(x_i-\bar{X})^2}+ x_ix_i^*\frac{\sigma^2}{\sum(x_i - \bar{X})^2} = \frac{\sigma^2}{\sum(x_i-\bar{X})^2}(\frac{\sum x_i^2}{n}-\bar{X}(x_i+x_i^*)+x_ix_i^*)$

Community
  • 1
user1357015
  • 1,744
  • I apologize for starting a new thread, I will delete that one. Since: $var(\hat{\beta_0}) = \sigma^2(1/n + \bar{x}^2/S_{xx})$ and $var(\hat{\beta_1}) = \sigma^2/S_{xx}$ and $cov(\hat{\beta_0}, \hat{\beta_1}) = -\bar{x}(\sigma^2/S_{xx})$

    Would it follow that:

    $\text{cov}$$(\hat{y_i}, \hat{y_i}^) = \sigma^2(1/n + \bar{x}^2/S_{xx}) + (x_i^ + x_i)(-\bar{x}(\sigma^2/S_{xx})) + (x_ix_i^*)(\sigma^2/S_{xx}) = $

    $(\sigma^2)[(1/n + \bar{x}^2/S_{xx}) + (x_i^* + x_i)(-\bar{x}/S_{xx}) + (x_ix_i^*/S_{xx})]$?

    And if the steps I took are correct, is there any further simplification to be done?

     – stats566 Apr 24 '14 at 22:07
  • It's ok, we were all new here at one point :-). Take a look at what I did. Hope that helps! And if it does, please do marke it as complete! :-). – user1357015 Apr 24 '14 at 22:39