5

What causes exponential distribution to have biased and non-biased ML-estimator?

$f(x;\theta)=\theta \exp(-\theta x)$

has biased estimator.

$f(x;\theta)=\frac{1}{\beta} \exp(-x/\beta)$

has unbiased estimator.

But what causes this?

mavavilj
  • 4,109
  • 1
    If the variance is positive, one being unbiased implies the other is biased. There's an informal argument here – Glen_b Nov 21 '18 at 12:13

1 Answers1

3

The simple explanation is that

  1. The MLE of the transform $T(\theta)$ is the transform of the MLE $T(\hat{\theta})$, i.e., the MLE is equivariant by any one-to-one transform;
  2. Unbiasedness does not survive by non-linear transforms, i.e., if $\mathbb{E}_\theta[\hat{\theta}]=\theta$ then $\mathbb{E}_\theta[T(\hat{\theta})]\ne T(\theta)$

For exponential families, there exists one "mean parameterisation" for which the MLE is unbiased, namely if the density writes $$f(x|\theta)=h(x)\exp\{\theta\cdot S(x)-\tau(\theta)\}$$ then$$\mathbb{E}_\theta[S(X)]=\nabla\tau(\theta)$$ and the MLE $\hat{\theta}$ satisfies$$S(X)-\nabla\tau(\hat{\theta})=0$$which implies that $S(X)$ is the MLE of its expectation, $\nabla\tau(\theta)$, thus that$$\mathbb{E}_\theta[\widehat{\nabla\tau(\theta)}]=\nabla\tau(\theta)$$is unbiased.

Xi'an
  • 105,342
  • Okay, however, any way to tie this to the example I gave? I don't understand how making $\theta$ to $\frac{1}{\beta}$ would apply to what you're writing. – mavavilj Nov 21 '18 at 11:26
  • Try to fit the exponential distributions within the exponential families framework, i.e., identify $h$, $S$, and $\tau$. – Xi'an Nov 21 '18 at 11:53
  • @mavavilj Let $T(x) = \frac{1}{x}$. Property (1) then gives your maximum likelihood estimator for $\frac{1}{\theta}$. Property (2) says that estimator for $\frac{1}{\theta}$ will be biased. – Matthew Gunn Nov 21 '18 at 17:03