Is the MLE variance estimator for the normal distribution asymptotically normal?

Question

In textbooks, it is mentioned that the maximum likelihood estimators are asymptotically normal. I am having trouble with understanding how this can be true for the estimator of variance (for example, of a normal distribution).

Additionally, Cochran's theorem shows that the unbiased sample variance is chi-square distributed. My intuition is that this applies for finite samples and the chi-square distribution develops to a normal distribution as the sample size approaches infinity.

Can anybody offer an explanation on this matter?

What's left to answer here? Yes, the sample variance follows a chi-square distribution (as you said), and yes, this becomes asymptotically normal because a chi-squared distribution with large degrees of freedom is similar to a normal variable. If you're just looking for reassurance, rather than clarification on something you don't understand, you're correct and your reasoning is fine. — Closed Limelike Curves, Sep 03 '22 at 23:31
Relevant: https://stats.stackexchange.com/questions/475837/how-can-the-square-of-an-asymptotically-normal-variable-also-be-asympotically-no — COOLSerdash, Sep 04 '22 at 06:26
Thank you for your responses, you have helped validate my intuition on the matter. — Amav, Sep 05 '22 at 08:44
@Glen_b: What was hard for me to understand is how valid a normal distribution can be for the variance. For example, even though it is on an asymptotic level, this normal distribution can be used in Wald tests. Wouldn't this allow for negative variance values? Wouldn't the chi-square be more appropriate for testing? — Amav, Sep 05 '22 at 08:49
At finite sample sizes, it will of course be an approximation; while the normal will have some probability with values below 0 when the sample size is large enough that the approximation is good, the probability associated with variance < 0 will be very, very small. — Glen_b, Sep 05 '22 at 10:03
So, could the probability of variance < 0 be an intuitive check of whether the sample is large enough? What are the alternatives for assessing the sufficiency of sample size? — Amav, Sep 05 '22 at 10:59
What constitutes 'sufficiently accurate' depends on how far into the tail you need to look and how accurate you need it to be there. But since we can write down the density of the MLE exactly (and calculate its cdf on a computer any time we need it) we can just use that; we can avoid approximation. — Glen_b, Sep 05 '22 at 11:31

Glen_b · Accepted Answer · 2022-09-06T00:48:59.140

I think I understand now what your concern is.

Let's take $X_i\stackrel{_\text{iid}}{\sim}N(\mu,\sigma^2),\,i=1,2,...,n$

The MLE of $\sigma^2$, $s_n^2$ is $\frac{n-1}{n}$ times the usual unbiased sample estimate, $s_{n-1}^2$. Now we have that $(n-1)s^2_{n-1}/\sigma^2\sim \chi^2_{n-1}$, which is equivalent to $\Gamma(\frac{n-1}{2},2)$ in the shape-scale parameterization.

Consequently, $s^2_{n-1}$ is $\frac{\sigma^2}{n-1}$ times a $\Gamma(\frac{n-1}{2},2)$ and so $s^2_{n}$ is $\frac{\sigma^2}{n}$ times a $\Gamma(\frac{n-1}{2},2)$ and so $s^2_{n}\sim \Gamma(\frac{n-1}{2},\frac{2\sigma^2}{n})$ $-$ again, shape-scale.

This has mean $\frac{n-1}{n}\sigma^2$ and variance $\frac{2(n-1)}{n^2}\sigma^4$, or standard deviation $\frac{\sqrt{2(n-1)}}{n}\sigma^2$. Note, therefore, that the coefficient of variation is $\sqrt{\frac{2}{n-1}}$, so for example, when $n=33$, the mean is $4$ standard deviations above $0$ and when $n=99$ the mean is $7$ standard deviations above $0$.

As long as $n$ is not small, the probability that the normal approximation (the normal distribution with the same mean and variance) associates with negative values will be extremely small $-$ not of practical consequence; by $n=99$ that probability will be about $1.28\times 10^{-12}$, or a bit over $1$ part in a trillion ... and $99$ is quite some way short of the limiting case.

As we can see in the plot, by $n=99$ almost all of the density of the MLE is between $\sigma^2/2$ and $3\sigma^2/2$, and the normal approximation is very close to the true density. The negative half line is many standard deviations from the mean.

Thank you for your detailed response. Something that is not fully clear: by normal approximation, do you mean the gamma distribution that converges to normal with increasing sample size? If so, is the difference of the means of the distribution the bias of the MLE estimator? — Amav, Sep 05 '22 at 11:54
(Retyping my comment because the typo in the original is annoying): If the gamma is approximately normal, there must be a normal distribution that's close to it. The normal distribution with the same mean and variance is typically used when you want to use a normal approximation for something (indeed using those values usually the basis on which the variable was standardized to show convergence in the first place). When you said in comments earlier "how valid a normal distribution can be for the variance" ... what other normal distribution did you mean? — Glen_b, Sep 06 '22 at 04:38
"how valid a normal distribution can be for the variance" referred to my considerations regarding negative variance values. But with your replies, things are now clear. — Amav, Sep 06 '22 at 17:35

Is the MLE variance estimator for the normal distribution asymptotically normal?

1 Answers1