2

Say that we have $X_i \sim \mathcal{N}(\mu_i, \sigma_i^2)$. Is there some formula to calculate analytically the expected value of the sum $S = \sum_i^n X_i^2$?. This is equivalent to computing $\mathbb{E}\left(||X||^2 \right)$ where $X \sim \mathcal{N}(\mu, \Lambda)$ and $\Lambda$ is a diagonal matrix with positive entries $\Lambda_{ii} = \sigma_i^2$.

From the above, we know that each $\frac{X_i^2}{\sigma_i^2} \sim \chi^2_1\left( \frac{\mu^2}{\sigma_i^2} \right)$, where $\chi^2_1(\lambda)$ is the non-central chi-squared distribution with 1 degree of freedom and non-centrality parameter $\lambda$.

I am interested in this problem following several answers to related problems (e.g. 1, 2), showing that the probability distribution of this sum, $S = \sum_i^n X_i^2$, or equivalently, of $||X||^2$ is very complicated, being related to the generalized chi squared distribution. But I'm wondering whether, as opposed to the distribution, the expected value (and hopefully some other moments too) have simpler expressions.

dherrera
  • 1,258
  • 8
  • 26
  • Is a $\chi^2$ - without non-centrality - even supported for non-integral degrees of freedom? I think not. Therefore there are some constraints to state before you can even deal with summing non-central, non-identical RVs. – AdamO Nov 06 '23 at 18:52
  • 1
    The initial question looks too simple: since $E[X_i^2]=\operatorname{Var}(X_i)+E[X_i]^2 = 1+1^2 = 2,$ then $E[S] = 2\sum_i w_i$ whether or not the $w_i$ are positive. Are you sure you have asked the question you intended? – whuber Nov 06 '23 at 19:30
  • why don't you formulate your question in a simpl yet general form? – Math-fun Nov 06 '23 at 20:16
  • 1
    @whuber good point, I see now OP asked about the sum's expectation. It was so easy, I thought they were asking about the general distribution... which is too hard :D – AdamO Nov 06 '23 at 20:20
  • @whuber answer is right. I got caught up in how complicated the actual distribution is, and completely missed the trivial formula for the expected value. But that was my intended question. However, despite seeing a lot of questions about the distribution of $||X||^2$ for $X\sim \mathcal{N}(\mu, \Sigma)$ in this site, I hadn't found one for $\mathbb{E}(||X||^2)$. I edited the question to be about the more general and useful case. If this is not a duplicate of another question, do you want to post your answer @whuber? – dherrera Nov 06 '23 at 20:37
  • If not, I'm happy to post your comment as an answer @whuber – dherrera Nov 06 '23 at 22:09
  • @AdamO - why not? A chi-squared distribution is just a gamma distribution with parameters presented differently. – Henry Nov 07 '23 at 00:44

1 Answers1

1

As pointed out in the comments, the answer is obtained using basic probability.

We have that $\mathbb{E}\left[ X_i^2 \right] = \mathrm{Var}(X_i) + \mathbb{E}\left[X_i \right]^2$. Also, we have that, for independent $Y_i$, $\mathbb{E}\left[ \sum_i Y_i \right] = \sum_i \mathbb{E} \left[ Y_i \right]$. Putting these two facts together, for $X \sim \mathcal{N}\left(\mu, \Lambda \right)$ where $\Lambda$ is a diagonal covariance matrix with elements $\Lambda_{ii} = \sigma_i^2$:

$$\mathbb{E}\left[ ||X||^2 \right] = \mathbb{E}\left[ \sum_i^n X_i^2 \right] = \sum_i^n (\sigma_i^2 + \mu_i^2).$$

Luis Mendo
  • 1,069
dherrera
  • 1,258
  • 8
  • 26