Conditioning on a variable

Question

I am looking at some software code that performs conditioning on random variables. For example, one can have a set of random variables which have a multivariate normal distribution associated with them and then you can condition on a given variable to take on a certain value and then get the associated conditional distribution for the rest of the variables.

Now, I notice that distributions which are univariate do not have this method implemented. But for the sake of completeness, for example for a univariate Gaussian when we actually observe that the variable takes on a certain value, shouldn't the conditional distribution be a point mass with zero variance?

Answer 1

Let $X$ be a random variable with density function $f(x)$. You ask about the conditional distribution of $X$ given that $X=x$. And, intuitively, since there is no uncertainty left, we now simply know the value of $X$, the conditional distribution is a probability atom of 1 at $x$ (contradicting the comment by @Zen).

Some details. Since we cannot conditioning directly on an event of probability 1, take first a small region containing $x$ and denote it $dx$. Then $$ \DeclareMathOperator{\P}{\mathbb{P}} \P(X \in dx \mid X \in E) = \frac{\P(X \in dx \cap X \in E)}{\P(X \in E)} = \\ \frac{\P(X \in E \cap dx)}{\P(X \in E)} $$ and in the density case, shrinking $dx$ to $x$, in the limit we get the conditional density $$ \frac{f(x)}{\P(X \in E)} $$ Now, if we take $E = dx$, the quotient is 1, and in the limit shrinking $dx$ to $x$ we get a probability atom at $x$.