1

Suppose I have $\textbf{X}$ ~ $N(\textbf{0},\Sigma)$, and I'm considering two different linear combinations, $a^* X$ and $b^* X$, which we suppose are uncorrelated. I understand that linear combinations of variables in multivariate Gaussians are Gaussian, and that in a multivariate Gaussian, that uncorrelatedness implies independence.

But can I conclude from uncorrelatedness of $a^* X$ and $b^* X$ that they are independent? Do I know that $a^* X$ and $b^* X$ together form a multivariate Gaussian?

Note that uncorrelatedness of $a^* X$ and $b^* X$ does not imply a and b are orthogonal, as in 2D for a correlated Gaussian we could have a = <1,0> and $b = <\frac{-\sigma_{XY}}{\sigma_{XX}},+1>$, which are uncorrelated but not orthogonal.

Thanks.

user49404
  • 457

2 Answers2

4

Assume that $\mathbf{X}$ (an $n\times 1$ column vector) has a multivariate normal (MVN) distribution and that $\mathbf a$ and $\mathbf b$ are $1\times n$ row vectors. Then, $\mathbf{aX}$ and $\mathbf{bX}$ are univariate normal random variables. This follows from what many people take to be the defining property of MVN distributions:

$\mathbf{X}$ is said to enjoy a MVN distribution if and only if $\mathbf{aX}+c$ is a normal random variable for all choices of $\mathbf a \in \mathbb R^n$ and $c \in \mathbb R$.

Note that $\mathbf a = \mathbf 0$ is a permissible choice with the resulting constant $c$ being regarded as $\mathscr N(c,0)$, a (degenerate) normal random variable with mean $c$ and variance $0$. See the extensive discussion in the comments following this answer for more on this special case.

So, what about the normal random variables $\mathbf{aX}$ and $\mathbf{bX}$? First, to answer an issue raised by @Scott, note that $\mathbf X$ is the same random vector in both instances, and not two independent copies of $\mathbf X$ (which would be better denoted by different symbols such as $\mathbf X$ and $\mathbf Y$ or, better yet to emphasize their $X$ness, $\mathbf X^{(1)}$ and $\mathbf X^{(2)}$). So, with this put of the way, let us consider whether $\mathbf{aX}$ and $\mathbf{bX}$ are also MVN in addition to being just marginally normal as the above definition claims? Well, the defining property for MVN distributions (specialized to bivariate normal distributions in this instahce) says that $\displaystyle \left[\matrix{\mathbf{aX}\\\mathbf{bX} }\right]$ is MVN (more specifically bivariate normal) if and only if $$\alpha \mathbf{aX} + \beta \mathbf{bX} + \gamma = (\alpha \mathbf{a}){\mathbf X} + (\beta \mathbf{b})\mathbf{X} + \gamma = \big(\alpha \mathbf{a} + \beta \mathbf{b}\big)\mathbf{X} + \gamma$$ has a normal distribution, which it does from our assumption that $\mathbf X$ is MVN. What about Scott's feeling that we really ought to be looking at whether $\mathbf{aX}^{(1)}$ and $\mathbf{bX}^{(2)}$ (where $\mathbf X^{(1)}$ and $\mathbf X^{(2)}$ are two independent copies of $\mathbf X$) are MVN or not? Well, that case is trivial since $\mathbf{aX}^{(1)}$ and $\mathbf{bX}^{(2)}$ are independent normal random variables (and hence uncorrelated as the OP wants them to be), and that they are MVN follows directly from their independence and normality.

So, what the OP wants to know is true even without any specific assumptions about the mean vector being $\mathbf 0$ as the OP assumes or whether or not the covariance matrix $\Sigma$ of $\mathbf X$ is nonsingular or singular (note that I have carefully avoided the use of the word density -- a $n$-dimensional multivariate normal random variable does not have a $n$-variate density function), or whether or not $\mathbf a$ and $\mathbf b$ are such that $\mathbf{aX}$ and $\mathbf{bX}$ are uncorrelated (normal) random variables. When $\mathbf{aX}$ and $\mathbf{bX}$ are indeed uncorrelated, then since they are MVN, they are also independent.

Dilip Sarwate
  • 46,658
  • I think that there is some distinction to be made here between linear combinations of two i.i.d variables $\bf X$ (which we would normally say $\bf X$ and $\bf Y$ are i.i.d) and linear combinations of copies of the variable $\bf X$. Makes a definite difference in the "correlated" vs "uncorrelated" cases. Perhaps you would be kind enough to clarify? – Him Sep 22 '19 at 05:35
0

Do I know that a∗X and b∗X together form a multivariate Gaussian?

Yes. Form a matrix, $M$ as:

\begin{align} M &= \left[ \array{a\\b} \right] \end{align}

Since $X$ is Gaussian and since linear combinations of Gaussian are Gaussian, $MX$ is Gaussian. Since $MX$ is:

\begin{align} MX &= \left[ \array{a*X\\b*X} \right] \end{align}

we know that $aX$ and $bX$ are jointly Gaussian.

Bill
  • 7,884
  • "since linear combinations of Gaussian are Gaussian" this does not hold, generally. – Him Sep 20 '19 at 18:55
  • @Scott Yes, it does. If X is Gaussian, then M*X is Gaussian. – Bill Sep 20 '19 at 19:23
  • See, e.g. here – Him Sep 20 '19 at 19:44
  • @Scott Not relevant. All that says is that a linear combination of univariate but not multivariate normal need not be normal. The current case is linear combination of multivariate normals. – Bill Sep 21 '19 at 20:26
  • I think the issue between Bill and Scott is a matter of how one defines the MVN property, I have used a minimalistic definition in my own answer from which it is easy to show that $\mathbf {aX}$ and $\mathbf {bX}$ (as well as $\mathbf {cX}$ and $\mathbf {dX}$ and $\mathbf {eX}$ and $\cdots$) enjoy the MVN property, while Bill takes the more general view that $\mathbf X$ is MVN if and only if $\mathbf{AX}$ is MVN for all choices of $m\times n$ matrices $\mathbf A$ – Dilip Sarwate Sep 22 '19 at 16:28