6

I want to sample from a distribution with density $$ f(\mathbf x) = \int f(c) \prod_{i=1}^n f(x_i|c) dc $$ where $\mathbf x=(x_1,x_2,...,x_n)$. In my particular setup, is easy to sample according to the densities $f(c)$ and $f(x_i|c)$, but it is not obvious how to sample from the marginal given above.

Would the following work?

  1. Sample $c$ according to $f(c)$.
  2. Sample $x_i$ according to $f(x_i|c)$, for $i=1,2,...,n$.

Heuristic argument for why this should work:

  • Obviously $f(\mathbf x,c) = f(\mathbf x|c)f(c)$, so the $(\mathbf x,c)$ pairs generated by my procedure above are distributed according to $f(\mathbf x,c)$. Marginalization amounts to ignoring the $c$'s in any pair $(\mathbf x,c)$. So the $\mathbf x$'s produced in step 2 of my procedure, taken by themselves, are distributed according to $f(\mathbf x)$, as desired.

Is this sensible? Is there a short way to make it rigorous if so? Thanks!

mpiktas
  • 35,099
Mhc
  • 63
  • 3

1 Answers1

3

You are right!

The formulation you describe has a general name to it. The theorem is called de-Finetti's theorem for exchangeable sequences, and is the fundamental theorem behind the Bayesian philosophy and ideas. Specifically, $x_i$s are exchangeable, that is, any permutation of $x_i$s will have the same distribution as $f(x_1,\ldots,x_n)$.

In your strategy, I would be a bit more comfortable if you had drawn $c$ for every $i$. That is, generate $c$ from $f(c)$ and then generate $x_i$, and repeat this for $i=1\ldots n$. But I think if you generate enough $\mathbf {x}$s, this should not be a problem.

suncoolsu
  • 6,622