2

I stumbled upon a (for me) odd formula for the sample covariance matrix. Is there anyone who can show me how to get to this form?:

Start with the simple regression $\iota = Xb + \epsilon$, a $1\times T$ vector, and by applying OLS we get $\hat{b} = (X'X)^{-1}X'\iota.$

Now the formula stated:

$$\hat{\Sigma} = \frac{1}{T}X'X - \bar{x}\bar{x}'.$$

They state that $\hat{\Sigma}$ is the sample covariance matrix. Does anyone know how to get to this last result?

The definition is normally $\hat{\Sigma} = \frac{1}{T}\sum \limits_{t=1}^{T}(x_t - \bar{x})(x_t - \bar{x})'$, but I can't seem to derive the form above.

Thanks in advance!

Vin
  • 21
  • What is sample covariance matrix? Do we know the definition? If so, can we reach to this form? – User1865345 Dec 14 '22 at 20:35
  • By writing out the definition of matrix multiplication, you will find the two formulas agree. Their relationship is identical to that between the two standard formulas in the univariate case, namely the mean product of residuals or the mean product minus the product of means. – whuber Dec 14 '22 at 21:01
  • 1
    it may be helpful to first establish the (independently useful) relation $\mathbf{X}^\top\mathbf{X} = \sum_{t=1}^T \mathbf{x}_t\mathbf{x}_t^\top$. – John Madden Dec 14 '22 at 21:20
  • 1
    Thanks a lot for the help! The useful relation that you sent together with the fact hat the other terms add up to $-2\bar{x}\bar{x}' + \bar{x}\bar{x}'$ gets the result – Vin Dec 14 '22 at 21:32
  • Another way to obtain this result is Gaussian elimination, as explained at https://stats.stackexchange.com/a/108862/919. – whuber Dec 14 '22 at 21:46

0 Answers0