11

Consider an ordinary least squares model, $$y = \beta X + \epsilon \qquad \epsilon\sim N(0, \sigma)$$

The Gauss-Markov theorem tells us that the ordinary least-squares (OLS) estimator is the minimum-variance linear unbiased estimator (BLUE) for the coefficients: $$ \beta \approx \hat\beta = (X^TX)^{-1}X^Ty $$

Does an unbiased, nonlinear estimator with lower variance, $\tilde\beta$, exist?

Based on my previous question.

user126350
  • 311
  • Not under normality. If the error is e.g. Laplace distributed, the mean absolute deviation estimator is more efficient. – hejseb Jul 04 '17 at 04:45
  • Can you suggest why no such estimator $\tilde\beta$ exists? – user126350 Jul 04 '17 at 04:49
  • 10
    If the underlying data generating process is i.i.d gaussian with a constant variance, and a linear mean model (basically the model matches the truth), in that case OLS is the minimum variance unbiased estimator, because it attains the Cramer Rao Lower Bound. Note that I didn't say minimum variance linear unbiased. Basically it is the best. http://www.econ.ohio-state.edu/dejong/note5.pdf. page 17. – Cagdas Ozgenc Jul 04 '17 at 06:24

2 Answers2

3

No. Linear regression is also BUE.

Source: https://www.ssc.wisc.edu/~bhansen/papers/gauss.pdf

ebuber
  • 31
  • I was not aware of this result... very interesting! (+1) – jbowman May 10 '23 at 01:24
  • While the question does specify $iid$ Gaussian errors, this answer would be improved by repeating that to clarify this additional requirement beyond the Gauss-Markov conditions. Without that, this answer seems to suggest that Gauss-Markov does not take it far enough. – Dave May 10 '23 at 10:09
2

The Gauss-Markov theorem gives the conditions where the OLS estimator is the BLUE, and those conditions do not include normality of the residuals. When we also include that normality assumption, then we can remove the "L" and wind up with the "Best Unbiased Estimator", not just the best linear unbiased estimator (section 2.1, example 1 of the Ohio State econometrics notes).

However, if we do not make the normality assumption, then we can wind up with nonlinear estimators of the coefficients that have lower variance than the OLS estimate but are unbiased. For example, consider heavy-tailed errors and the solution given by minimizing absolute loss (quantile regression at the median), as I do here.

Dave
  • 62,186