1

I have done extensive monte carlo simulations which have validated the following statements:

  1. Given a sample of N independent and identically distributed Rayleigh random variables $x_i$ with parameter $\sigma$, then $\widehat{\sigma^2}\approx \!\,\frac{1}{2N}\sum_{i=1}^N x_i^2$ is an unbiased maximum likelihood estimate.

  2. (Formally, from Siddiqui 4.10:) The confidence interval for that estimate comes from $\frac{N\overline{x^2}}{\chi^2 (2N)}$

Given these two statements, then is the median of the second distribution an unbiased MLE for $\sigma^2$?

I have checked some examples and found the median (50% quantile) given by (2) to match the proper MLE given in (1), but only to two significant figures. Is the difference between these two estimators significant? And if so, why does it exist? (E.g., is there a correction factor that would make them match exactly?)

feetwet
  • 1,108
  • 1
    What do you mean by "the confidence is distributed as"? – kjetil b halvorsen May 09 '17 at 19:46
  • 1
    There are now three related but different things in play: the confidence (which is a number between 0 and 100%), some kind of confidence limit or limits (the endpoints of a potentially infinite interval), and the width of a two-sided confidence interval. It would help to clarify which you're referring to by "confidence in that estimate." However, this probably makes little difference: your question appears to be purely one about whether the median of a $\chi^2(2N)$ variate equals $2N$. That's easily computed or looked up. – whuber May 26 '17 at 15:11
  • @whuber - I obviously need help with the terminology. As best I can tell #2 is the distribution from which one can draw confidence limits and intervals on the MLE in #1. In any case, as you note, the problem does appear to reduce to "whether the median of a $\chi^2(2N)$ variate equals $2N$." They are asymptotically equal, but I can't understand why they wouldn't be formally equal. And if they aren't formally equal, why and what the significance of that inequality is. – feetwet May 26 '17 at 16:21
  • @whuber - The titular question also seemed salient to me, even though I am particularly interested in the case I describe. I.e., is it generally true that the median of a confidence distribution on an estimate is itself an unbiased MLE? If not, are there necessary and/or sufficient conditions for that to be the case? – feetwet May 26 '17 at 16:29
  • The general answer is no: it depends on what you mean by "confidence distribution" and how it was calculated. Sufficient conditions are that this distribution is assumed to be symmetric, that what you mean by "confidence" is the width of a symmetric two-sided interval; that this interval is erected by adding and subtracting some multiple of a standard error to the estimated mean; and that this estimated mean is unbiased. Please visit the Wikipedia link I provided to learn what the median of a $\chi^2$ distribution is. – whuber May 26 '17 at 16:35
  • @whuber - Wikipedia only gives an approximation for the median, which is what I checked to note that it is asymptotically equal to the $\chi^2$ parameter. – feetwet May 26 '17 at 17:37
  • The approximation is good enough to show you the median does not equal $2N$. – whuber May 26 '17 at 20:15
  • @whuber - Not knowing where that approximation came from, that's not evident to me. In fact, for any N >> 1 the difference between N and the approximation of the median appears to approach 2/3, which in relative terms goes to zero. Is that an omitted error term in the approximation? I have no way to tell. Either way, it seems like some small $f(N)$ could correct for the difference between the median and $2N$. If I can find that function, then is the $\chi^2(N)$ median - $f(N)$ an unbiased MLE? – feetwet May 26 '17 at 21:32

1 Answers1

1

From an i.i.d. sample of Rayleigh r.v.'s we obtain the MLE/Method of Moments estimator

$$\hat \sigma^2_{MLE} = \frac{1}{2N}\sum_{i=1}^n x_i^2 \implies 2\hat \sigma^2_{MLE} = \frac{1}{n}\sum_{i=1}^n x_i^2$$

We see that the right hand side is the sample mean of the squared Rayleighs... and we can easily deduce by the change-of-variables formula that if $X$ follows a Rayleigh distribution with parameter $\sigma$, then the random variable $Z = X^2$ follows an Exponential distribution with scale parameter $\gamma = 2\sigma^2$.

A confidence interval for the mean of an Exponential distribution depends on the sum of the i.i.d exponentials, and the sum of i.i.d. $\sum^n Exp(\gamma) \sim Gamma(n, \gamma)$ (shape-scale paramaterization).

$$\frac{2n\bar Z}{\chi^2_{1-\frac{\alpha}{2},2n}} < \gamma < \frac{2n\bar Z}{\chi^2_{\frac{\alpha}{2},2n}} \implies \frac{2}{\chi^2_{1-\frac{\alpha}{2},2n}}\cdot Gamma(n,\gamma) < \gamma < \frac{2}{\chi^2_{\frac{\alpha}{2},2n}}\cdot Gamma(n,\gamma)$$

Since $\gamma /2 = \sigma^2$ we obtain

$$\left(\chi^2_{1-\frac{\alpha}{2},2n}\right)^{-1}\cdot Gamma(n,2\sigma^2) < \sigma^2 < \left(\chi^2_{\frac{\alpha}{2},2n}\right)^{-1}\cdot Gamma(n,2\sigma^2)$$

We have no closed form for the median $m$ of a Gamma distribution, but the Gamma becomes almost symmetric rather quickly, as the shape parameter increases in value (here equal to the sample size $n$). So the median will be almost equal to its mean, say $\mu$, which here is equal to

$$m_l \approx \mu_l = \frac{2n\sigma^2}{\chi^2_{1-\frac{\alpha}{2},2n}},\;\;\;m_u \approx \mu_u = \frac{2n\sigma^2}{\chi^2_{\frac{\alpha}{2},2n}}$$

for the lower and upper bound of the confidence interval respectively.

Now, as $n$ increases one can verify that both quantiles of the chi-squared distribution approach the degrees of freedom, here $2n$ (this happens because the chi-squared distribution becomes very concentrated around its mean value as the degrees of freedom increase),

$$n \uparrow \implies m_l, m_u \approx \sigma^2$$

So, using $\hat \sigma^2$ as the parameter instead of the unknown true value, we see why the median of this distribution is close to the MLE.

The approximate formula for the median of a $Gamma (k,\theta)$ distribution is

$$m \approx k\theta\cdot \frac{3k -0.8}{3k+0.2}$$ which in our case becomes

$$m_l \approx \mu_l\frac{3n -0.8}{3n+0.2},\;\;\; m_u \approx \mu_u\frac{3n -0.8}{3n+0.2}$$

Alecos Papadopoulos
  • 58,953