4

I have quite a newbie doubt about Bayesian inference. Let's say that my prior data is composed by a Gaussian distribution (mean1, standard deviation1).

My likelihood is another Gaussian with mean2, std deviation2.

Now, my question is how can I get the posterior, please? The most clear tutorial I found is this, in eq(28), but I do not have clear whether the author is applying Monte Carlo there or not.

Ferdi
  • 5,179
galtor
  • 143

1 Answers1

5

Finding the posterior distribution does not involve Monte Carlo, but involves the Bayes rule. If the posterior turns out to be complicated (usually meaning that it is not a known distribution), then Monte Carlo methods are used to sample from the posterior. Thus, the first step is to always try and write down the posterior.

Notationally, your likelihood is $Y_i|\mu_1 \sim N(\mu_2, \sigma_2^2)$ assuming $\sigma_2^2 > 0$ is known. Let the prior on $\mu_2$ be $N(\mu_1, \sigma_1^2)$ where $\mu_1 \in \mathbb{R}, \sigma_1^2 > 0$ are fixed.

The posterior distribution is found by the Bayes Rule.

$$f(\mu_2|y) = \dfrac{f(\mu_2) \prod_{i=1}^{n}f(y_i|\mu_2)}{\prod_{i=1}^{n}f(y_i|\mu_2))}. $$

Since we condition on $y$, $f(y_i)$ is a constant.

\begin{align*} f(\mu_2|y) & \propto f(y|\mu_2)f(\mu_2)\\ &= \prod_{i=1}^{n} \left(\frac{1}{\sqrt{2\pi \sigma_2^2}} \exp \left\{-\dfrac{(y_i - \mu_2)^2}{2\sigma_2^2} \right\} \right) \frac{1}{\sqrt{2\pi \sigma_1^2}} \exp \left\{-\dfrac{(\mu_2 - \mu_1)^2}{2\sigma_1^2} \right\}\\ & \vdots \end{align*}

If you keep solving this like the link you shared, you will see that this takes the form of a Normal distribution in $\mu_2$ with the parameters as indicated in the link. You can find more details in the following links

https://www.youtube.com/watch?v=c-d05z0_5mw http://www.people.fas.harvard.edu/~plam/teaching/methods/conjugacy/conjugacy_print.pdf

Greenparker
  • 15,588
  • Thanks for the fast reply. OK, and what is the n that first appears on eq (16). Have you got a numerical example of this? Thx – galtor Mar 27 '16 at 15:33
  • If you expand the product, you will see the $n$ appear. It is just a matter of going through the steps of algebra. – Greenparker Mar 27 '16 at 15:35
  • And finally, please: if I want to make a prediction (sport prediction or poll prediction) what kind of posterior must I use: predictivie posterior or simply posterior? – galtor Mar 27 '16 at 15:37