0

Suppose a study with dependent variable $y$, and predictors $x_1$, $x_2$, ..., $x_k$.

Assume that I have access to the covariance matrix $$\hat\Sigma = \begin{bmatrix}Var(y) & cov(x_1,y) & ... & cov(x_k,y) \\ cov(y, x_1) & var(x_1) & ... & cov(x_1, x_k) \\ ... \\ cov(y,x_k) & cov(x_1,x_k) &... & var(x_k) \end{bmatrix}$$ and the mean vectors $\hat\mu = [\bar{y}, \bar{x_1}, ... \bar{x_k}]'$. Is it possible to get the error matrix of the regression coefficients?

It is easy to get the estimates of the regression coefficients (see responses given here or here for example). With some manipulation, we also get the sum of square residuals $\epsilon^T.\epsilon$ as well as the error variance $\hat\sigma^2$ fairly easily with

$$\hat\sigma^2 = Var(y) -\Big( \sum_{i=1}^k \beta_i^2\, var(x_i) +2 \sum_{1 \le i<j \le k} \beta_i \,\beta_j \,cov(x_i,x_j) \Big)$$ and $$\epsilon^T.\epsilon = \hat\sigma^2 \times (n-k) $$

To get $\widehat{var(\beta_i)}$, I have seen on many locations that the design matrix $\mathbf{X}$ can be used as

$$ \hat\sigma^2 (\mathbf{X}^T. \mathbf{X})^{-1}$$

(and it works well). But what if I do not have access to the data, only to the summary statistics that are the vector of means and the covariance matrix (and sample size of course)? Can it be done?

Denis Cousineau
  • 249
  • Since you have the mean vectors, you can easily recover $X^\prime X$ from $\hat\Sigma$ with the simple algebra of the covariance formula. Go on from there -- you have already described the hard part, which is obtaining $\hat\sigma^2.$ – whuber Dec 13 '23 at 23:13
  • Indeed, as the covariances $cov(x_i,x_j) = 1/(n-1) \sum (X_{k,i}-\hat\mu_i)(X_{k,j}-\hat\mu_j)$ can be expanded as $1/(n-1) \big(\sum X_{k,i} X_{k,j} - \sum X_{k,i} \hat\mu_j + \sum X_{k,j} \hat\mu_i + n \hat\mu_i \hat\mu_j \big) $ we end up finding that $(X'X){ij} = (n-1) \hat\Sigma{ij} + n \hat\mu_i \hat\mu_j $. Still, not easy to find the inverse from this expression... – Denis Cousineau Dec 14 '23 at 12:58
  • Re "not easy:" in what sense? Matrix inversion is a basic operation for which many algorithms exist. There are simple formulas for the entries in terms of determinants: look up "matrix adjugate." – whuber Dec 14 '23 at 15:08
  • 1
    Thanks for the link, that indeed makes things easier! – Denis Cousineau Dec 14 '23 at 15:55

0 Answers0