3

Given a sample vector $x$ of size $N$ from a normally distributed population. With frequentist methods the population mean is estimated as $\hat{\mu}=\frac{\Sigma{}x_i}{N}$, population sigma is estimated as $\hat{\sigma}=\sqrt{\frac{\Sigma{(x_i - \hat{\mu})^2}}{N - 1}}$.

Is there a Bayesian prior for the population mean and population sigma that will lead to the same estimates as the above frequentist formulas for any $x$?

winerd
  • 599
  • 2
    $p(\mu,\sigma) \propto \sigma^{-1}, ,, \mu \in \mathbb{R},, \sigma \in (0, \infty)$ – Cyan Mar 29 '13 at 12:25

1 Answers1

2

As pointed out by @Cyan in the comment above, the prior $p(\mu,\sigma) \propto \sigma^{-1}$ is a probability matching prior, in the following sense:

  1. Define $t=\frac{\mu-\bar{x}}{s/\sqrt{n}}$, where $\bar{x}$ and $s^2$ are the sample mean and variance (the same statistics you use as estimators).

  2. It follows that the posterior distribution of $t|(X_1,...,X_n)$ is student's t with $n-1$ degrees of freedom, precisely the distribution of $t$ as a pivotal quantity in the frequentist pardigm.

The implications are, for instance, that a HPD interval for $t$ coincides with the corresponding level confidence interval for $\mu$.

The joint posterior distribution for $(\mu, \sigma)$ is analytically intractable, but the posterior modes take the form:

$$ \mu = \bar{x} $$

$$ \sigma = \sqrt{\frac{n-1}{n+1}s^2} $$

Ferdinand.kraft
  • 136
  • Do you mean that using the prior $p(\mu,\sigma) \propto \sigma^{-1}$ the posterior MEANS of $\mu$ and $\sigma$ will be equal to the above $\hat{\mu}$ and $\hat{\sigma}$? – winerd Mar 29 '13 at 16:08
  • @winerd, I'm working on it, I believe this is the case indeed. The posterior is $p(\mu,\sigma|\boldsymbol{x}) \propto \sigma^{-n-1} e^{{-((n-1)s+n(\bar{x}-\mu)^2)/(2\sigma^2)}}$. I'll update the answer as soon as I', done :-) – Ferdinand.kraft Mar 29 '13 at 17:06
  • @winerd, Not for the standard deviation parameter. What is true is that the posterior mean of $\sigma^2$ is $\hat{\sigma}^2$. (Similarly, $\hat{\sigma}^2$ is unbiased for $\sigma^2$, but $\hat{\sigma}$ is not unbiased for $\sigma$.) – Cyan Mar 29 '13 at 18:53
  • @winerd, after struggling a few days with the posterior I must admit it is analytically intractable, an mathematica agrees with me... The best I could do is to find the posterior modes. – Ferdinand.kraft Apr 02 '13 at 02:59
  • @Ferdinand.kraft, that's not correct. Check out this and notice that the math still goes through if $\alpha_0 = \beta_0 = \lambda_0 = 0$. – Cyan Apr 02 '13 at 06:00
  • @Cyan What do you mean by $\alpha_0$, $\beta_0$ and $\lambda_0$? – winerd Apr 02 '13 at 08:12
  • @winerd, it refers to notation used in the linked derivation. (Maybe you didn't notice the link? It's http://en.wikipedia.org/wiki/Normal-gamma_distribution#Posterior_distribution_of_the_parameters ) – Cyan Apr 02 '13 at 14:41