3

I'm an undergraduate student. I read about multivariate normal distribution in hogg and craig. And i wonder why the covariance is allowed to be positive SEMI-definite. I read this

Normal distribution with positive SEMI-definite covariance matrix

And I found this

Multivariate statistic

I don't understand it actually. it talks about affine subspace or something. We still need the probability distribution to integrate out to 1. 1 is a real number, so what is the relation of that "affine subspace" with $R^n$?? Can anybody explain it in simple way? I am totally curious about this. Any illustration will be appreciated, Thanks..

Community
  • 1
Aeroplane
  • 463

1 Answers1

3

Let's talk about it in two dimensions, $x, y$.

Suppose you have a standard normal distribution on $X$, while $Y$ is deterministic at 0. What is the joint distribution over $(X, Y)$? It is a multivariate normal with singular covariance. That's obvious because var($y$) = 0. It forms a nice distribution over a subspace of $(X,Y)$ though. Namely, $X$.

Now suppose we rotate it by 45 degrees. Now it still forms a nice distribution over a linear subspace of $(X,Y)$ but var($X$) and var($Y$) aren't zero. Neither are deterministic this time!

Now further suppose we translate it so that the nice subspace doesn't necessarily go through the origin. That's the fully general case where it's "nice" over an affine subspace of $(X,Y)$.

You can do the same thing in higher dimensions, where you have fewer degrees of freedom than the full space the distribution lives in.

imh
  • 66
  • Sorry, I'm not sure about what you mean by rotating the distribution 45 degrees. Would you please elaborate? And...is that ok to see $(X,Y)$ as multivariate normal whenever Y is actually a discrete random variable? – Aeroplane Nov 20 '14 at 02:59
  • 1
    Picture the joint distribution of a multivariate normal over $X,Y$ where both have mean 0. $X$ has variance 1 and $Y$ has variance $\sigma$. It's going to look like an ellipse. As $\sigma$ gets smaller and smaller, that ellipse gets narrower and narrower. In the limit, it's infinitely thin, and $Y$ is a deterministic variable. – imh Nov 21 '14 at 05:44
  • 1
    By rotate, I mean rotate that ellipse in the $X, Y$ plane. You could make two new variables $X' = (X + Y)/\sqrt{2}$ and $Y' = (X - Y)/\sqrt{2}$. This is a rotation, so we have a diagonal ellipse in the $X', Y'$ plane. In the limit $\sigma \to 0$ it becomes degenerate: $X' = Y' = X$, even though both are nondeterministic random variables.

    The translation I was talking about before was translating this ellipse. For an example of rotation and translation, $X'' = X + Y + 2$, $Y'' = X - 2Y + 50$.

    (and I meant variance $\sigma^2$ before, not that it matters. And by "nice", I meant nondegenerate.)

     – imh Nov 21 '14 at 05:54