9

Suppose a multivariate distribution over $\mathbb R^n$ has a singular covariance matrix. Can we conclude that it does not have a density function?

For example, it is the case for the multivariate normal distribution, but I am not sure if it is true for all other multivariate distributions.

This is, I think, a question of the existence of Radon-Nikodym derivative wrt the Lebesgue measure on $\mathbb R^n$ , but elementary probability theory may also have the answer.

Tim
  • 19,445

3 Answers3

7

A singular covariance matrix means that there exists a linear combination $Y = \sum_{i=1}^n a_i X_i$ of the $n$ random variables such that $E[Y] = a_0$ and $\operatorname{var}(Y) = 0$. Thus, all the probability mass lies in a hyperplane of $\mathbb R^n$ defined by $\sum_{i=1}^n a_i x_i = a_0$ and so the $n$ random variables cannot have a $n$-variate density function.

Dilip Sarwate
  • 46,658
3

Yes, but it will be a probability distribution over a lower dimensional subspace. You could argue that it is a probability distribution in R^N if you allow things like dirac delta functions. That's a subtle mathematical issue but physicists, for example, do it all the time.

Dave31415
  • 1,143
2

Although this is alluded to above, I want to make it clearer that whilst it may not have a meaningful density on $\mathbb{R}^{n}$ you can define the density on a Rank($\Sigma$)-dimensional subspace, where $\Sigma$ denotes the covariance matrix.

gung - Reinstate Monica
  • 145,122
James
  • 29
  • 2
    This is not generally true. Take, for instance, $(X,Y)$ for which $Y=0$ and $X$ has any nondegenerate non-continuous distribution. Although the rank of the covariance matrix is $1$, this distribution does not have a density on any one-dimensional submanifold of $\mathbb{R}^2$. – whuber Jun 03 '14 at 20:22