2

This is the reverse of my previous question Is PCA invariant to orthogonal transformations?

Let $A$ and $B$ be two $n$ x $p$ data matrices where $n$ is the number of samples and $p$ is the number of features.

Both $A$ and $B$ are centred (zero mean for each feature).

The following are the eigenvalue decompositions of their covariance matrices:

$$ \frac{1}{n-1} A^T A = V_A L_A V^T_A $$

$$ \frac{1}{n-1} B^T B = V_B L_B V^T_B $$

Now suppose that $A$ and $B$ have the same principal components, that is:

$$ A V_A = B V_B $$

Does it follow that $B$ is a rotation of $A$ (or vice-versa), that is:

$$ B = AQ \ ? $$

Community
  • 1
turdus-merula
  • 1,611
  • 19
  • 22
  • I post this question (and the answer below - which I hope is correct) in case it is helpful to someone. It is more or less a theoretical exercise, but it may have a practical application. – turdus-merula Oct 16 '16 at 15:49
  • 2
    Since $AV_A=BV_B$ is equivalent to $B=A(V_AV^{-1}_B)$ and $Q=V_AV^{-1}_B$ is orthogonal, the answer is evident. – whuber Oct 16 '16 at 17:56
  • @whuber indeed, it is, but it wasn't for me at first :) I answered my own question in case someone else needs this. Do you have any thoughts about the second part of my answer? – turdus-merula Oct 16 '16 at 18:17

1 Answers1

2

In this equation:

$$ A V_A = B V_B $$

we right-multiply by $ V_B^T $:

$$ A V_A V_B^T = B V_B V_B^T $$

Both $V_A$ and $V_B$ are orthogonal matrices, therefore:

$$ A V_A V_B^T = B $$

Because $ Q = V_A V_B^T $ is a product of two orthogonal matrices, it is therefore an orthogonal matrix. This means that:

$$ AQ = B $$

Note: if two datasets $A$ and $B$ have the same principal components, it could also be that $ B = A T^T $, where $T$ is a translation matrix (which is not orthogonal). However, since data centering is a prerequisite of PCA, $T$ gets ignored.

Also given this post, we can say that: two centred matrices $A$ and $B$ of size $n$ x $p$ are related by an orthogonal transform $ B = AQ $ if and only if their principal components are the same.

Community
  • 1
turdus-merula
  • 1,611
  • 19
  • 22
  • I did not find the trivial "theorem" in the end nowhere else on the Internet so I thought it would be helpful to write it down somewhere. – turdus-merula Oct 16 '16 at 15:51