Monte Carlo Integration

Question

Let $I$ = the value of integration and $p$ = probability distribution.

The estimator is denoted as $\left\langle I \right >$ and is $$\left\langle I \right >=\frac{1}{N}\sum_{i=1}^{N}\frac{f(x_i)}{p(x_i)}$$

The expected value of this estimator is computed as follows:

$$E[\left\langle I \right >] = E[\frac{1}{N}\sum_{i=1}^{N}\frac{f(x_i)}{p(x_i)}]$$

$$\qquad \qquad \space \space \space \space = \frac{1}{N}E[\sum_{i=1}^{N}\frac{f(x_i)}{p(x_i)}]\cdots(1)$$

$$ \qquad \qquad\qquad\quad= \frac{1}{N}N\int\frac{f(x)}{p(x)}p(x)dx\cdots(2)$$

$$ = \int f(x)dx \space \space$$ $$ = I\qquad\qquad \space$$

I wonder why the two equation, $(1)$ and $(2)$, are equal??

Answer 1

In Monte Carlo integration, the samples $x_1, x_2, \ldots x_N$ are independent, identically-distributed random variables. This implies they all have the same expectation value. The derived quantities $f(x_i)/p(x_i)$ also all have the same expectation value. So the expectation of a sum of $N$ of these is the same as $N$ times the expectation of any one of them: $$E\left[ \sum_{i=1}^N \frac{f(x_i)}{p(x_i)} \right] = N \cdot E\left[\frac{f(x_1)}{p(x_1)}\right]$$ (I picked $x_1$ arbitrarily.)

Then, the expectation value of a continuous random variable is found by integrating over its probability measure: $$E\left[\frac{f(x_1)}{p(x_1)}\right] = \int_{\text{domain of }p(x)} \frac{f(x)}{p(x)} \, p(x) \, dx$$