2

My slides say that the exponent of a multivariate normal distribution, $(\mathbf{X} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\mathbf{X} - \boldsymbol{\mu})$, follows a chi squared distribution with degrees-of-freedom $p$ (i.e., the length of the $\mathbf{X}$ vector)

My intuitive understanding of chi squared is that it is the sum of (squared) standard normals that are independent.

Therefore back to the first sentence, doesn't this m.variate normal distribution have to be independent (i.e., $X_1, X_2.... X_p$ independent of each other) for this to make sense? The slides did not specify independence.

Thus, what I think is that for the correlation matrix, it will be zero for all off-diagonals. Am I right?

Ben
  • 124,856
jojorabbit
  • 153

1 Answers1

2

The quadratic form you are looking at is the squared-Mahalanobis distance. The independence in this computation comes implicitly from the fact that this quadratic form includes the inverse-variance matrix (also called the precision matrix) in the middle. To derive the distributional result at issue, consider a normal random vector $\mathbf{X} \sim \text{N}(\boldsymbol{\mu}, \boldsymbol{\Sigma})$ and suppose we let $\boldsymbol{\Delta} = \sqrt{\boldsymbol{\Sigma}}$ be the principal square root of the variance matrix, so that $\boldsymbol{\Sigma} = \boldsymbol{\Delta} \boldsymbol{\Delta}$. (Note also that $\boldsymbol{\Delta}$ is symmetric, so you can transpose it without changing the result.) Then we can get a standardised normal random vector by taking:

$$\mathbf{Z} \equiv \boldsymbol{\Delta}^{-1} (\mathbf{X} - \boldsymbol{\mu}) \sim \text{N}(\mathbf{0}, \mathbf{I}).$$

This allows us to rewrite the quadratic form at issue as:

$$\begin{align} (\mathbf{X} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\mathbf{X} - \boldsymbol{\mu}) &= (\mathbf{X} - \boldsymbol{\mu})^\text{T} (\boldsymbol{\Delta} \boldsymbol{\Delta}^\text{T})^{-1} (\mathbf{X} - \boldsymbol{\mu}) \\[12pt] &= (\mathbf{X} - \boldsymbol{\mu})^\text{T} (\boldsymbol{\Delta}^\text{T})^{-1} \boldsymbol{\Delta}^{-1} (\mathbf{X} - \boldsymbol{\mu}) \\[12pt] &= (\mathbf{X} - \boldsymbol{\mu})^\text{T} (\boldsymbol{\Delta}^{-1})^\text{T} \boldsymbol{\Delta}^{-1} (\mathbf{X} - \boldsymbol{\mu}) \\[12pt] &= [\boldsymbol{\Delta}^{-1} (\mathbf{X} - \boldsymbol{\mu})]^\text{T} [\boldsymbol{\Delta}^{-1} (\mathbf{X} - \boldsymbol{\mu})] \\[12pt] &= \mathbf{Z}^\text{T} \mathbf{Z} \\[6pt] &= \sum_{i=1}^p Z_i^2 \\[6pt] &\sim \text{ChiSq}(p). \\[6pt] \end{align}$$

Ben
  • 124,856
  • thank you so much for this answer, i'll need some time to digest it... but in short, does it mean that it is necessary for the individual Xs to be independent? since from what i gather from the defined Z, the correlation matrix is an identity matrix, therefore implying independence? – jojorabbit Sep 26 '22 at 16:47
  • 1
    No, it doesn't. The $X_i$ values are not independent but the $Z_i$ values are. – Ben Sep 26 '22 at 22:12