Conditioning of join gaussian over a line

Question

I need to compute the conditional probability of bivariate normal distribution over a line. Let's suppose that X and Y both are normal distributions and that they are independent. Let's suppose that we want to find the probability that a point 'k' will be on the line L= aX+bY=c. To do so we need to compute the following:
$ p(k|aX+bY=c) = \frac {p_x(X_k)p_y(Y_k)}{p(aX+bY=c)} = \frac {p_x(X_k)p_y(Y_k)}{\int_{L} p_x(X)p_y(Y) dX dY}$

where k is point over the line L. For this to happen we will use a base for the line L define as a vector b(b1,b2) and a point of origin r(r1,r2). Therefore, the previous integral can be written as:
$ \frac {p_x(X_k)p_y(Y_k)}{\int_{t} p_x(r_1+b_1t)p_y(r_2+b_2t) dt} = \frac {p_x(r_1+b_1k)p_y(r_2+b_2k)}{\int_{t} p_x(r_1+b_1t)p_y(r_2+b_2t) dt}$

My goal now is to show that this last fraction is indeed a gaussian distribution and to get its mean and variance

Answer 1

Let $X$ and $Y$ be jointly normal random variables with means $\mu_X, \mu_Y$, and covariance matrix $\Sigma$. (We do not need that $X$ and $Y$ are independent, although it does simplify some calculations.)

$$ \begin{pmatrix}X\\Y \end{pmatrix} \sim N(\begin{pmatrix}\mu_X \\ \mu_Y \end{pmatrix} , \begin{pmatrix}\Sigma_{11} & \Sigma_{12}\\ \Sigma_{21} & \Sigma_{22}\end{pmatrix}) $$

Pre-multiply by a matrix $A$ to transform the second variable into the 'variables-part' of desired line equation, $aX + bY$. As $A$ is fixed and $X,Y$ are jointly normal, the result is still jointly normal.

$$ A\begin{pmatrix}X\\Y \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ a & b \end{pmatrix}\begin{pmatrix}X\\Y \end{pmatrix} = \begin{pmatrix}X\\ aX + bY \end{pmatrix} \sim N(A\begin{pmatrix}\mu_X \\ \mu_Y \end{pmatrix} , A\begin{pmatrix}\Sigma_{11} & \Sigma_{12}\\ \Sigma_{21} & \Sigma_{22}\end{pmatrix}A^T) $$

The mean of this multivariate distribution is

$$\begin{pmatrix} \mu_X \\ a\mu_X + b\mu_Y \end{pmatrix}$$

and, for simplicity, denote the resulting variance as

$$ A\begin{pmatrix}\Sigma_{11} & \Sigma_{12}\\ \Sigma_{21} & \Sigma_{22}\end{pmatrix}A^T = \begin{pmatrix}\Omega_{11} & \Omega_{12}\\ \Omega_{21} & \Omega_{22}\end{pmatrix} $$

Finally, follow one of the answers in this question, Deriving the conditional distributions of a multivariate normal distribution, to prove that the distribution of $X$ conditional on $aX + bY = c$ has a normal distribution with mean

$$ \mu = \mu_X + \Omega_{12}\Omega_{22}^{-1}(c - a\mu_X - b\mu_Y) $$

and variance

$$ \sigma^2 = \Omega_{11} - \Omega_{12}\Omega_{22}^{-1}\Omega_{21} $$

We now have the conditional distribution of $X$ given $aX + bY = c$. The corresponding value of $Y$ can be easily found from the same line equation, $Y = c/b - (a/b)X$.

For a point $k$ along the line $(r_1 + b_1k, r_2 + b_2k)$ The conditional distribution of $X$ can be re-expressed to give the conditional distribution of $K$, by re-arranging $x = r_1 + b_1k$ to $k = (x - r_1)/b_1$

Therefore $K | aX + bY = c$ has a normal distribution with mean and variance

$$ \mu_K = \frac{\mu_X + \Omega_{12}\Omega_{22}^{-1}(c - a\mu_X - b\mu_Y) - r_1}{b_1} $$

$$ \sigma^2_{K} = \frac{\Omega_{11} - \Omega_{12}\Omega_{22}^{-1}\Omega_{21}}{b_1^2} $$

Answer 2

A bivariate normal density can be likened to a piece of bologna (or did I mean to write baloney?) about which Americans often say "No matter how you slice it, it is still bologna". The meaning w.r.t. bivariate normal densities (regarded as a piece of bologna sitting on the $x$-$y$ plane) is that every cross-section of this piece (of lunch meat) has the shape of a univariate normal density. Slice it along the straight line $ax+by=c$ and the cross-section has the shape of a normal density. It is not exactly a univariate normal density in that the "area under the curve" in usually not $1$ as it must be for all valid univariate densities, but for any nonnegative function $g(x)$ with finite area $A$, $A^{-1}g(x)$ is a valid density function. So, as I said, the shape is of the correct form, but the area need not be, but the area is readily fixed.

So, what is the density of $aX+bY+c$ when $(X,Y)$ is bivariate normal? Easy-peasy -- it is a normal density with mean $$E[aX+bY+c] = a\mu_x +b\mu_Y +c$$ and variance $$\operatorname{var}(aX+bY+c) = a^2\operatorname{var}(X) + b^2\operatorname{var}(Y) +2ab\operatorname{cov}(X,Y).$$