5

Suppose $x_i$'s are IID random samples from $d$-dimensional isotropic Gaussian centered at zero and normalized to have $E[\|x\|^2]=1$

Suppose we have $$a=x_1 \\ b=a+x_2 \\ c=b+x_3$$

As $d$ increases, the following is observed

$$\cos(a,c) \approx \cos(a,b)\cos(b,c)$$

Where $$\cos(x,y)=\frac{\langle x, y\rangle}{\|x\| \|y\|}$$

Why?

enter image description here

notebook

individual cosine distribution:

enter image description here

We can continue this pprocess with $d=c+x_4$, $e=d+x_5$, and this "three-way" cosine similarity seems to approach integer values among all triplets of vectors enter image description here

Yaroslav Bulatov
  • 6,199
  • 2
  • 28
  • 42
  • What do the distributions of the individual cosines look like? – jwimberley Sep 10 '22 at 00:42
  • 1
    added to ..... post – Yaroslav Bulatov Sep 10 '22 at 00:51
  • Thanks -- i should have asked, more specifically, do the individual cosine distributions narrow woth increasing d, and is there a sense in which the variance of the cosine ratio is narrower than the variance of the cosine distributions themselves? – jwimberley Sep 10 '22 at 01:52
  • they all seem to shrink at about the same rate – Yaroslav Bulatov Sep 10 '22 at 05:28
  • 1
    As $d$ increases, it becomes increasingly likely that any set of such Normal variates is nearly mutually orthogonal. Go on from there. BTW, your formula for the cosine is incorrect--the denominator should not be squared. Are you using the correct formula in your computations or the one you have posted? I think this result relies on using the wrong formula, which reduces (asymptotically) to the Pythagorean Theorem. – whuber Sep 10 '22 at 15:36
  • 1
    good catch, removing the square makes it easier to interpret -- cosines are just multiplicative in this setting $\cos(a,c)=\cos(a,b)\cos(b,c)$ – Yaroslav Bulatov Sep 10 '22 at 16:03
  • ... and that is because asymptotically the cosines are all $1.$ And because they all have Beta distributions, you can obtain more precise asymptotics if you like. See https://stats.stackexchange.com/a/85977/919. – whuber Sep 10 '22 at 16:58
  • They are all zero asymptotically, but this relationship requires them to go to zero at a specific rate – Yaroslav Bulatov Sep 10 '22 at 18:21
  • Right--I meant to write "0" instead of "1," corresponding to orthogonality. – whuber Sep 12 '22 at 15:29
  • This seems to be more general phenomenon for small perturbations. IE, if instead of "adding random noise" I do a random simple rotation with angle $\theta$, cosines are multiplicative for $\theta\le \frac{\pi}{4}$, whereas $\theta>\frac{\pi}{4}$, this relationship breaks down – Yaroslav Bulatov Sep 12 '22 at 16:41

1 Answers1

1

This was answered on Mathoverflow by Iosif Pinelis. Using Chebychev's inequality he showed that isotropic normal RV's $X,Y,Z$

\begin{equation} U:=X,\quad V:=X+Y,\quad W:=X+Y+Z, \end{equation}

\begin{equation} \cos(U,V)=\frac{U\cdot V}{\sqrt{U\cdot U}\sqrt{V\cdot V}}\sim_P\frac1{\sqrt2}, \end{equation} \begin{equation} \cos(V,W)=\frac{V\cdot W}{\sqrt{V\cdot V}\sqrt{W\cdot W}}\sim_P\frac2{\sqrt6}, \end{equation} \begin{equation} \cos(U,W)=\frac{U\cdot W}{\sqrt{U\cdot U}\sqrt{W\cdot W}}\sim_P\frac1{\sqrt3}. \end{equation}

\begin{equation} \cos(U,V)\cos(V,W)\sim_P \cos(U,W). \end{equation}

Yaroslav Bulatov
  • 6,199
  • 2
  • 28
  • 42