0

I want to choose a random vector in high dimensions such that it all directions have the same uniform chance (i.e. isotropic in all directions). My current idea is the following algorithm:

  1. sample v from a high dim multivariate Gaussian
  2. then normalize that vector by dividing by its norm

I believe this is sufficient but don't know how to prove it. I know that high dimension stats can have weird behaviours (like most of the volume of a Gaussian is at the "edge") so I wanted to make sure I didn't miss anything trivial but important. Anyone know if this is correct? If it is how do I prove it?

What I have in mind in pytorch looks as follows:

v = MultivariateNormal(torch.zeros(10000), torch.eye(10000))
v = v/v.norm(10000)
Charlie Parker
  • 6,866
  • 3
    The title of the linked question says 3d, but the answers are applicable to arbitrary dimensions. The short answer is yes, this method works; this answer gives some numerical confirmation, and this one gives citations for a proof. The only pratical thing to worry about here as far as I know is to make sure that v.norm(2) isn't extremely small for numerical reasons; this is luckily very unlikely in high dimensions. – Danica Apr 01 '18 at 01:18
  • @Dougal what I fail to see is how that answer confirms that my solution is right? – Charlie Parker Apr 01 '18 at 01:21
  • 2
    A proof is given in the paper cited by the second link, but apparently it’s still not freely available despite being from 1959.... So check this comment instead for the idea. – Danica Apr 01 '18 at 01:30
  • @Dougal thanks! I see that my method is correct which is what I wanted the most. – Charlie Parker Apr 01 '18 at 01:32
  • This result does't require much proving: simply by looking at either the density function or the characteristic function of the isotropic multivariate Normal distribution you see it's a function of radius alone, whence it must be uniform on the sphere (because it doesn't depend on any spherical coordinate). – whuber Apr 01 '18 at 23:16
  • @whuber I think the reason its subtle for other people is cuz one can also uniformly sample the surface of a sphere which apparently gives the wrong answer. It concentrates at the poles I've been told. – Charlie Parker Apr 02 '18 at 00:29
  • 1
    @Pinocchio You must be referring to an incorrect method of sampling the sphere, then! Incidentally, your solution--although correct--uses unnecessary computation. By invoking a procedure to produce multivariate Normal samples you will be performing some initial matrix calculations and then subsequent transformations, all of which are unnecessary given that all you have to do is generate independent univariate Normal values. – whuber Apr 02 '18 at 14:41
  • @whuber ok fine I should use torch.normal (which I believe chooses a Gaussian individually for each point). But besides that they are both correct right? – Charlie Parker Apr 02 '18 at 15:31
  • @whuber why is the univariate normal thing correct? My intuition tells me the MultivariateGaussian has a higher chance of being isotropic because the mass of the MultivariateGaussian is concentrated at the edges of the sphere while the univariate Gaussian creates some sort of full ball where the points closest to the origin are most likely. Can you elaborate? – Charlie Parker Apr 02 '18 at 15:53
  • @Pinocchio A matrix where each entry is independently $N(0, 1)$ has exactly the same distribution as a matrix distributed as $N(\mathbf 0, \mathbf I)$. – Danica Apr 13 '18 at 19:16
  • @Dougal maybe Im confused about the volume being concentrated in the surface of the sphere but if thats the case how do I get a sphere that doesn't have its volume distributed in the surface but in the inside? thats what Im confused about I believe. – Charlie Parker Apr 13 '18 at 21:29
  • That’s called a ball, not a sphere. You can sample from that by sampling from a sphere and then scaling down to the new norm: sample from a Uniform(0,1) and take the $d$th root if you’re in $d$ dimensions. – Danica Apr 14 '18 at 09:00

0 Answers0