1

A comment to this question suggests that the OLS estimate of linear model parameters is unbiased, even when the error term is Cauchy. Given that Cauchy distributions lack an expected value, I am skeptical that the parameter estimates have an expected value (though I am on board with their expected value being the true parameter values if they do have an expected value).

Do the OLS parameter estimates even have an expected value when the error term is Cauchy? If so, is the bias equal to zero?

$$ y_i = \beta_0 + \beta_1x_{i, 1} + \dots + \beta_px_{i, p} +\epsilon_i\\ \epsilon_1,\dots,\epsilon_n\overset{iid}{\sim}t_1\\ \implies\\ \mathbb E\left[ (X^TX)^{-1}X^Ty \right] = \beta\\? $$

Dave
  • 62,186
  • 7
    In this case, it is meaningless (or incorrect) to try to use the "$E$" operator with respect to the random vector $(X^TX)^{-1}X^Ty$, because its "expectation" does not exist! The concept of "unbiasedness" or "bias" only makes sense if the expectation of an estimator exists. – Zhanxiong Apr 03 '23 at 19:43
  • 3
    One can use the median instead of the mean. – Xi'an Apr 03 '23 at 19:53
  • I can't find my notes on the subject, but I seem to recall the Lindeberg condition for the CLT to hold is along the lines of the residuals are bounded in probability. That is, lim∣∣max(ei)/∑ni=1(ei)2∣∣→0 .It's possible to conceive of some strong designs of the X that make this possible even for Cauchy errors. Of course, CLT of the regression parameter implies convergence in probability. To Zhanxiong's point, bias is a finite sample result and is, contradictorily not what we're dealing with here. – AdamO Apr 03 '23 at 20:04

0 Answers0