1

The OLS estimator $\hat{\beta}$ of $\beta$ is unbiased. Now, $\hat{\beta}$ is a function of the random sample (y,$x_1$,...$x_j$) and is therefore a random variable itself. When we compute $\mathbb{E}(\hat{\beta}|X)$ we are basically fixing $X$ to be the data we have and we take an expectation over the $y$ variable. Unbiasedness tells us that if we compute all possible values of $\hat{\beta}$ by runnning and OLS regression for every possible $y$, then we would find that $\mathbb{E}(\hat{\beta}-\beta|X)=0$ by averaging over the $\hat{\beta}$'s, that is, the average over all those $\hat{\beta}$'s is the true population parameter (conditional on $X$). However, in practice we have only one $y$ which will allow to compute an OLS estimate $\hat{\beta}$ which differs almost surely from $\beta$ - since $\hat{\beta}$ is random, equality would be an event of pure luck. But that being the case, why do we care so much that the coefficient is unbiased? Is it because it allows for an interpretation of the coefficients as the marginal effect on the mean of the sampled random dependent variable (or the marginal effect of $X$ on $y$ when $y$ is drawn from a typical random sample in the sense that the sample mean equals the population mean) ? I have also read coefficients being interpreted as the average marginal effect of a variable on the dependent variable. However, I would read it not as the average/mean marginal effect on the dep. variable but rather as the marginal effect on the mean of the dep.variable conditional on $X$. Which interpretation is correct?

kjetil b halvorsen
  • 77,844
Daniel Pinto
  • 633
  • 6
  • 16
  • 1
    Well, isn't it better (other things equal) to be right on average than wrong on average? It isn't that we care "so much" about it, it's just a nice property to have in general. That isn't to say we should never use biased estimators, oftentimes the "best" estimators are biased. – dsaxton Oct 10 '16 at 04:09
  • 3
    People pay lip service to unbiasedness, but (speaking more generally than your regression example), they demonstrably don't care about it all that much. Who uses an unbiased estimator of standard deviation for example? How many people unbias their parameter estimates when fitting lognormal distributions? etc etc. I'm a bit more explicit in my opinion that many, since I don't pay much lip service to unbiasedness at all -- I really don't care much at all about unbiasedness -- it's rarely of practical relevance in things I deal with; I'll often take a biased estimate if it will get me close...ctd – Glen_b Oct 10 '16 at 05:43
  • ctd... to what I want. Other things being equal, of course no bias is better than bias, but so often other things are far from equal. – Glen_b Oct 10 '16 at 05:45
  • @Glen_b: but in the context of regression coefficients it may be of importance, e.g. if you perform hypothesis tests on the coefficient or if you compute a confidence interval for the coefficent ? For hypothesis tests you test whether the ''true'' value is e.g. different from zero (I think it is good to work with an unbiased estimate in that case), similar for a confidence interval, you find a (random) interval that has a certain probability to contain the ''true'' value ? With biased estimates you have tests/intervals for the ''true'' value plus ''something'' ? –  Oct 10 '16 at 06:44
  • @fcop Hypothesis tests are case in point -- many people use likelihood ratio tests but most MLEs are biased. Similarly many confidence intervals are not centered on the parameter. Consider confidence intervals for the standard deviation of the error term in regression for example. Or if you really want to focus on regression coefficients, consider the case where you have a contaminating process that leads to occasional wild values that are not centered at the underlying uncontaminated population line we're interested in. A good estimator will clearly be biased for the ...ctd – Glen_b Oct 10 '16 at 07:14
  • (conditional) population mean coefficient for the actual data-generating process, and at the same time will be (somewhat) biased for the uncontaminated one ... but hopefully getting quite close to it relative to an "unbiased" one. – Glen_b Oct 10 '16 at 07:14
  • @Glen_bThank you for your answers! What about my last question: "I have also read coefficients being interpreted as the average marginal effect of a variable on the dependent variable. However, I would read it not as the average/mean marginal effect on the dep. variable but rather as the marginal effect on the mean of the dep.variable conditional on X. Which interpretation is correct?" – Daniel Pinto Oct 10 '16 at 22:31
  • @fcop (continued) Basically I see it as the marginal effect of x on y given than random sample (x,y) and, if we have a typical sample in the sense that our estimate of b coincides with what we would have if we would estimate b from infinitely many random samples (x,y) and take the average, this marginal effect coincides with the marginal effect of x on the population conditional mean (due to unbiasedness). Does this make any sense? – Daniel Pinto Oct 10 '16 at 22:31

0 Answers0