1

I am learning mixture model. I understand what does it means, but I still do not understand some contexts. I really do not understand the following:

"The moments of a mixture random variables are linear combinations of the corresponding moments of the individual random variables." quote from" this site

Any help please?

Silver_80
  • 425

1 Answers1

2

Let $f_1$ and $f_2$ be two distributions and let $0<p<1$. You generate $X$, an observation from the mixture of $f_1$ and $f_2$, as follows: First generate $U\sim Unif(0,1)$. If $U\leq p$ then generate $Y\sim f_1$ and set $X=Y$. Otherwise, generate $Y\sim f_2$ and set $X=Y$.

It is therefore clear that for any set $A$, $$P(X\in A)$$

$$=P(X\in A|U\leq p)P(U\leq p)+P(X\in A|U>p)P(U>p)$$

$$=p\int_A f_1(x)dx+(1-p)\int_A f_2(X)dx$$

This is because given $U\leq p$, $X$ is coming from $f_1$.

Thus the density of $X$ will simply be $f_X(x)=pf_1(x)+(1-p)f_2(x)$.

So he $r$-th moment of $X$ will be $E(X^r)=\int_\mathbb R x^rf_X(x)dx=p\int_\mathbb Rx^rf_1(x)dx+(1-p)\int_\mathbb R x^rf_2(x)dx =pm_1(r)+(1-p)m_2(r)$

where $m_i(r)$ is the $r$-th moment under $f_i$.

Landon Carter
  • 1,565
  • Thank you so much for your help. Then, what does "linear" mean here. Also, do we have $1-p$ because we given $U >p$? – Silver_80 Oct 05 '17 at 06:31
  • 1
    Linear means the convex combination of the individual moments. In general if the mixture distribution has probabilities $p_1,p_2,...,p_k$ and the $r$-th moment for $f_i$ ($1\leq i\leq k$) is $m_i(r)$ then the r-th moment of mixture is $\sum_{i=1}^k p_im_i(r)$. This is a linear combination of the individual moments. And yes you have $1-p$ due to $P(U>p)$. – Landon Carter Oct 05 '17 at 06:42
  • Amazing answer and comment. Thank you so much for your help. – Silver_80 Oct 05 '17 at 06:42