1

Suppose we have two independent standard normal variables $X$ and $Y$ and lets construct $Z$ such as $Z = \rho X + \sqrt{1-\rho^2 Y}$. Therefore, $Z$ is dependent on $X$ and $Y$. I want to find the distribution of $Z|X, Y$. However, when I try to generate the Multivariate Normal distribution from these three Gaussians, I obtain a singular covariance matrix. I read the Wikipedia article on the Multivariate Normal distribution and it mentions a degenerate case when the covariance matrix is singular, giving the following expression:

$f(\mathbf{x})= \left(\det\nolimits^*(2\pi\boldsymbol\Sigma)\right)^{-\frac{1}{2}}\, e^{ -\frac{1}{2}(\mathbf{x}-\boldsymbol\mu)^{{{\!\mathsf{T}}}} \boldsymbol\Sigma^+(\mathbf{x}-\boldsymbol\mu) }$

where $\boldsymbol\Sigma^+$ is the generalized inverse and $\text{det}*$ is the pseudo-determinant.

So, my question is: is it correct to use this distribution to find $Z|X,Y$? And in this case, can I still say that $Z|X,Y$ has continuous pdf of $\frac{f(x,y,z)}{f(x,y)}$? I lack some knowledge on this topic, so I would also be grateful for some references.

donut
  • 263
  • Thank you for your answer. How can I know if n normally distributed variables have an n-variate joint density? And in my case, does $X$, $Y$ and $Z$ have a joint distribution, even if it is not trivarate normal? – donut Nov 25 '20 at 11:58
  • Thank you. Can you suggest me some bibliography on the matter? – donut Nov 26 '20 at 01:34
  • @Dilip I do not know of any distribution of joint Normality that would rule out $(X,Y,Z)$ as being jointly Normal. If it did, we would have to put all kinds of edge case restrictions on standard results about Normal distributions (such as, "linear combinations of joint normal variables are jointly Normal except when they are of reduced rank"). – whuber Jul 20 '23 at 20:49
  • @whuber I miswrote. $X, Y, Z$ indeed are jointly normal but they don't enjoy a trivariate density function. Since comments are not editable, should I just delete the incorrect comments and replace them by new ones? – Dilip Sarwate Jul 20 '23 at 20:58
  • @Dilip Sure, that's fine. – whuber Jul 20 '23 at 21:31
  • Corrected version of previous comment: $, $ and $$ are jointly normal random variables, but they are not jointly continuous and they don't have a trivariate joint density $f_{X,Y,Z}(x,y,z)$. – Dilip Sarwate Jul 21 '23 at 14:49
  • Corrected version of previous comment: Random variables always have distributions if the meaning of distribution is restricted to the CDF; all random variables have CDFs. Two or more random variables also have joint CDFs. If the joint CDF has a joint partial derivative ---- $\frac{\partial^n F(x_1,\cdots, x_n)}{\partial x_1. \cdots, \partial x_n}$ exists everywhere, ---- they have a joint density too and are said to be jointly continuous. So, the OP's ,, have a joint CDF but not a joint pdf.. – Dilip Sarwate Jul 21 '23 at 15:00

1 Answers1

1

Since $X$ and $Y$ are independent, $(X, Y)$ have a joint bivariate normal distribution. Then you define $Z$ as a function of $X$ and $Y$ (I assume $\rho$ is a known constant). So the conditional distribution of $Z$ given that $X=x, Y=y$ is simply the constant $z=\rho x + \sqrt{1-\rho^2} y$.

So the joint distribution of $X, Y, Z$ is singular (see Multivariate normal with singular covariance, Can a multivariate distribution with a singular covariance matrix have a density function?, )

In your case the function defining $Z$ is a linear function, so a full description of the joint distribution can be given by the bivariate normal distribution of $X, Y$, and that linear function. If you replaced the linear function by some non-linear function, for example $f(x,y) = x + y^3$, then the distribution would still be singular (the probability mass would be concentrated on a two-dimensional surface in 3-space), but the $3ţimes 3$ covariance matrix would not be singular (that is, it would have an inverse), so the invertibility of the covariance matrix does not guarantee a non-singular distribution.

A simple example in R:

n <- 10000
set.seed(7*11*13)
X <- rnorm(n)
Y <- rnorm(n)
f <- function(x, y) x + y^3
Z <- f(X, Y)
dat <- cbind(X, Y, Z)

C <- cov(dat)
C


X  0.998418216 -0.005227852  1.012374
Y -0.005227852  0.984617994  2.955338
Z  1.012373644  2.955338277 16.487973
> svd(C)
$d
[1] 17.0937095  1.0001744  0.3771255


$u
            [,1]          [,2]       [,3]
[1,] -0.06169015  0.9460834489 -0.3179944
[2,] -0.18008324 -0.3239229950 -0.9287863
[3,] -0.98171501 -0.0000315037  0.1903566


$v
            [,1]          [,2]       [,3]
[1,] -0.06169015  0.9460834489 -0.3179944
[2,] -0.18008324 -0.3239229950 -0.9287863
[3,] -0.98171501 -0.0000315037  0.1903566

As all the singular values are positive, the covariance matrix is not singular.

kjetil b halvorsen
  • 77,844