Linear Regression - Proof that coefficients estimated via OLS follow a normal distribution

Question

The aim of my question can be better illustrated by this quote extracted from the third chapter of Elements of Statistical Learning (link to book):

I'm trying to understand why, given that the error term follows a Gaussian distribution with mean 0 and constant and finite variance, do the coefficients also follow a Gaussian.

I understand how to derive the expected values of the beta estimate as well as its variance. I'm only having trouble proving its sampling distribution (the Normal).

Also (and please let me know if I should be asking this question in a new post)

how would the sampling distribution of the beta estimates be affected if the error term is not normally distributed - would they, for example, also tend to follow a normal distribution if our sample size is sufficiently large, under the CLT?

this is often inspected visually using a qq plot. Shapiro-Wilk can test it formally. — Estimate the estimators, Oct 09 '23 at 18:22
Does this answer your question? R: test normality of residuals of linear model - which residuals to use — Estimate the estimators, Oct 09 '23 at 18:22
basically because a linear function of a gaussian variable is also gaussian (we condition on distribution of X/assume X is deterministic). why don't you add the formula for the coefficients to your question, substituting in the definition of Y, ie 3.9. — seanv507, Oct 09 '23 at 18:45
you are better off reading another book on OLS. ESL basically just reviews the results for OLS. — seanv507, Oct 09 '23 at 18:46
$\varepsilon$ has a Normal distribution and linear combinations of Normal variables are Normal. This will be true even with just a single observation -- it has nothing to do with the CLT. BTW, all Gaussian distributions have finite variance. — whuber, Oct 09 '23 at 21:22
@seanv507 that is exactly my question! I do understand the intuition behind what you are saying but was hoping for a more theoretical approach. I believe I found it already looking at the post I've linked on my comment to the answer of Demetri Pananos — Frederico Portela, Oct 12 '23 at 11:02
@Xi'an I think a better duplicate, covering exactly the same excerpt from the same book (!) is Multivariate distribution of linear regression coefficients and unbiased variance estimator — Silverfish, Oct 28 '23 at 18:40

score 0 · Answer 1 · answered Oct 09 '23 at 21:18

Quoting almost verbatim from Wooldridge...

Under the assumptions of the classical linear model (namely, that the errors are gaussian with mean 0 and finite variance), each of the estimated coefficients can be written as

$$ \hat{\beta}_j = \beta_j + \sum_{i=1}^n w_{i, j}u_i$$

where $w_{i, j} = \hat{r}_{i,j}/SSR_j$ is the ratio between the ith residual from the regression of $x_j$ onto the other variances and the sum of swaured residuals from said regression. Because the $wz$ depend only on the independent variables, we can treat them as nonrandom and observe the estimated coefficients are linear combinations of iid gaussians. Since, by assumption, the $u$ were iid Gaussian, so to is the estimated coefficient.

Thank you I'll take a look at it! In the meantime I also found a post that mostly answers my question (this post question did not pop up in the 'possibly related' section when writing a new post). It talks about affine transformations of multivariate normal distributions — Frederico Portela, Oct 12 '23 at 10:58

Linear Regression - Proof that coefficients estimated via OLS follow a normal distribution

1 Answers1