6

Suppose $X_1,X_2,\ldots,X_n$ is a random sample from a Poisson Distribution with mean $\theta$. How can I find the conditional expectation $E \left( X_1+X_2+3X_3 |\sum_{i=1}^n X_i \right)$?

I know that $\sum X_i $ has a $poisson (n\theta$) distribution. Similarly the random variable $X_1+X_2+X_3$ has a $poisson (3\theta)$ distribution. I get confused with the required summations afterwards though.

Thank you.

JohnK
  • 20,366

2 Answers2

4

Define $S_n=\sum_{i=1}^n X_i$. By symmetry, $$ \mathrm{E}\left[ X_1 \mid S_n \right] = \mathrm{E}\left[ X_2 \mid S_n \right] = \dots = \mathrm{E}\left[ X_n \mid S_n \right] \quad \textrm{a.s.} \quad (*) $$ Hence, using $(*)$ and the linearity of the conditional expectation, we have $$ \mathrm{E}\left[ X_1 \mid S_n \right] = \frac{1}{n} \mathrm{E}\left[ X_1+\dots+X_n \mid S_n \right] = \frac{1}{n} \mathrm{E}\left[ S_n \mid S_n \right] = \frac{S_n}{n} \quad \textrm{a.s.} $$ The same reasoning leads to $$ \mathrm{E}\left[ X_1 +X_2 +3X_3\mid S_n \right] = 5\,\mathrm{E}\left[ X_1 \mid S_n \right] = \frac{5\,S_n}{n} \quad \textrm{a.s.} $$ Now, remember that $S_n\sim \mathrm{Poisson}(n\theta)$, and find the pmf of $5\,S_n/n$ (consider its support).

Zen
  • 24,121
  • Thank you. Can you please explain further what you have done in step 2? I do not understand why $E [X_1|S_n ] =S_n /n$ – JohnK Jan 19 '14 at 00:06
  • Can you use $(*)$ to prove that $\mathrm{E}\left[ X_1 \mid S_n \right] = \frac{1}{n} \mathrm{E}\left[ X_1+\dots+X_n \mid S_n \right]$? – Zen Jan 19 '14 at 00:09
  • Yep. It makes sense, thank you. I was asking about the $E[S_n|S_n]=S_n$. That is a cool identity. – JohnK Jan 19 '14 at 00:11
  • 1
    That's one of the properties of the conditional expectation. Informally, if you have the information $S_n$, then your best guess for $S_n$ is exactly $S_n$. – Zen Jan 19 '14 at 00:14
  • 2
    The important thing here is to understand that $\mathrm{E}[X]$ is a real number, while $\mathrm{E}[X\mid Y]$ is a random variable. Take alook at this question: http://stats.stackexchange.com/questions/38700/conditional-expectation-of-multivariate-distributions – Zen Jan 19 '14 at 00:15
  • Uh yeah ;). Enjoy your evening. – JohnK Jan 19 '14 at 00:15
  • I am working on a similar exercise and since your answer was helpful, I did not feel like starting another thread. What I wanted to ask you is whether it is right to say that $E[X_1 |\bar{X}=\bar{x}]=E[\bar{X}|\bar{x}]=\bar{x}$. I think it is but would you mind confirming that for me? Thank you. – JohnK Jan 28 '14 at 20:59
  • Again, symetry tells us $\mathrm{E}[X_1\mid \bar{X}_n] = \dots = \mathrm{E}[X_n\mid \bar{X}_n]$. Hence, $\mathrm{E}[X_1\mid \bar{X}_n]=(\mathrm{E}[X_1\mid \bar{X}_n]+\dots+\mathrm{E}[X_n\mid \bar{X}_n])/n=\mathrm{E}[\bar{X}_n\mid \bar{X}_n]=\bar{X}_n$. – Zen Jan 28 '14 at 23:37
  • Symetry here means two things: 1) IID $X_i$'s; 2) The symetry (with respect to the $X_i$'s) of the conditioning sigma-field (generated by $\bar{X}_n$). – Zen Jan 28 '14 at 23:41
  • By the way, it's a good exercise to make the symetry argument rigorous. – Zen Jan 28 '14 at 23:42
2

The OP has apparently found the way, so I am posting an answer.

I will denote $Z \equiv \sum_{i=1}^n X_i$. By linearity of the expected value we have $$E \left( X_1+X_2+3X_3 |Z \right)= E \left( X_1 |Z \right)+E \left( X_2 |Z \right)+3E \left(X_3 |Z \right)$$

Since the variables are i.i.d. they are also exchangeable,at least with respect to $Z$ (to which they have a symmetric relationship), so the three conditional expected values will be equal:

$$E \left( X_1+X_2+3X_3 |Z \right)= 5E \left( X_1 |Z \right)$$

Moreover, it is a known result that the conditional distribution of $X_1$ conditional on $Z=k$ is a Binomial,

$$X_1 | Z=k \sim Bin\left(k, \frac {E(X_1)}{E(Z)}\right) = Bin\left(k, 1/n\right)$$

and so

$$5E \left( X_1 |Z=k \right) = 5\frac kn$$

Our conditional expectation is viewed as a function of $Z$, is not conditioned just on $Z$ acquiring a specific value. Generalizing the last equation we obtain

$$5E \left( X_1 |Z \right) = \frac 5n Z= 5 \frac 1n \sum_{i=1}^n X_i$$

Note that

$$E \left( X_1 |Z \right) \rightarrow_p E(X_1) \;\;\text {as}\;\; n\rightarrow \infty$$

which should be intuitive.

Alecos Papadopoulos
  • 58,953
  • You might consider revising slightly the sentence beginning with "Since the variables are i.i.d....". The fact that they are i.i.d. does not imply that they are exchangeable conditional on an arbitrary $Z$. This $Z$ is special! – cardinal Jan 19 '14 at 01:38
  • @cardinal I thought it self-understood but I will point it out, sure. Thanks. – Alecos Papadopoulos Jan 19 '14 at 01:43
  • Thank you very much. It's helpful to have two ways to arrive at the result. – JohnK Jan 19 '14 at 10:04