Expectation of $\left(X-M\right)^T\left(X-M\right)\left(X-M\right)^T\left(X-M\right)$

Question

If $X=[x_1,x_2,...,x_n]^T$ is an $n$-dimensional random variable and we have

$E\left\{X\right\} = M = \left[m_1,m_2,...,m_n\right]^T$

$Cov\left\{X\right\} = \Sigma = diag\left(\lambda_1,\lambda_2,...,\lambda_n\right)$

how can I express the following expectation in terms of $M$, $\Sigma$, and $n$ (and maybe raw $m_i$'s and $\lambda_i$'s)?

$E\left\{ \left(X-M\right)^T\left(X-M\right)\left(X-M\right)^T\left(X-M\right)\right\}$

Supposing $x_i$'s are i.i.d and have normal distribution would be acceptable, but are these assumptions necessary?

Update:

1. I know that $E\left\{ \left(X-M\right)^T\left(X-M\right)\right\} = \sum_{i=1}^n \left(\lambda_i\right)$ but don't think this would help in this case.

2. In the section 6.2.3 Cubic Forms 8.2.4 Quartic Forms of Matrix cookbook there is a formula for calculated quadratic expectations like this, but i don't want just a formula to solve it. I think there should be a simple question for this problem because the covariance matrix is diagonalized.

Answer 1

Because $\left(X-M\right)^T\left(X-M\right) = \sum_i{(X_i - m_i)^2}$,

$$\left(X-M\right)^T\left(X-M\right)\left(X-M\right)^T\left(X-M\right) = \sum_{i,j}{(X_i - m_i)^2(X_j - m_j)^2} \text{.}$$

There are two kinds of expectations to obtain here. Assuming the $X_i$ are independent and $i \ne j$,

$$\eqalign{ E \left[ (X_i - m_i)^2(X_j - m_j)^2 \right] &= E\left[(X_i - m_i)^2\right] E\left[(X_j - m_j)^2\right] \cr

&= \lambda_i \lambda_j . }$$

When $i = j$,

$$\eqalign{ E \left[ (X_i - m_i)^2(X_j - m_j)^2 \right] &= E\left[(X_i - m_i)^4\right] \cr

&= 3 \lambda_i^2 \text{ for Normal variates} \cr

&= \lambda_i \lambda_j + 2 \lambda_i^2 \text{.} }$$

Whence the expectation equals $$\eqalign{ &\sum_{i, j} {\lambda_i \lambda_j} + 2 \sum_{i} {\lambda_i^2} \cr = &(\sum_{i}{\lambda_i})^2 + 2 \sum_{i} {\lambda_i^2}. }$$

Note where the assumptions of independence and Normality come in. Minimally, we need to assume the squares of the residuals are mutually independent and we only need a formula for the central fourth moment; Normality is not necessasry.

Answer 2

I believe this depends on the kurtosis of $X$. If I am reading this correctly, and assuming the $X_i$ are independent, you are trying to find the expectation of $\sum_i (X_i - m_i)^4$. Because $X_i^4$ appears, you cannot find this expectation in terms of $M$ and $\Sigma$ without making further assumptions. (Even without the independence of the $X_i$, you will have $E[X_i^4]$ terms in your expectation.)

If you assume that the $X_i$ are normally distributed, you should find the expectation is equal to $3 \sum_i \lambda_i^2$.

Answer 3

If you lose iid and normality assumptions things can get ugly. In Anderson book you can find explicit formulas for expectations of type

$\sum_{s,r,t,u}E(X_s-m)(X_r-m)(X_t-m)(X_u-m)$

when $X=(x_1,...,x_n)$ is a sample from stationary process, with mean $m$. In general it is not possible to express such types of moments using only the second and first moments. If we have $cov(X_i,X_j)=0$, it does not guarantee that $cov(X_i^2,X_j^2)=0$ for example. It does only for normal variables, for which zero-correlation equals independence.