1

In the case where I have an orthogonal $m \times n$ matrix $X$, and the covariance matrix of $X$, let's call it $C$, is such that $C_{ii} = 1$ for all $i$ such that $1 \le i \le n$ and $C_{ij}=0$ otherwise, how would I pick the direction of maximal variance for PCA, given that each direction has equal variance? Could I just pick any direction in my matrix $X$?

amoeba
  • 104,745
Dilip Thiagarajan
  • 135
  • What are about off-diagonal elements of C? Are they all 0? all equal? different? – ttnphns Sep 10 '16 at 07:57
  • 1
    PCA "hunts" after maximal variance directions within overall variance: http://stats.stackexchange.com/a/22571/3277. It doesn't matter in this respect whether Cii are all equal or not. – ttnphns Sep 10 '16 at 08:00
  • Off-diagonal elements of $C$ would all be 0. I did see that question - my understanding from that was each principal component explains an equal part of the variance in this case; is this correct? – Dilip Thiagarajan Sep 10 '16 at 19:39
  • 2
    Off-diagonal elements of C would all be 0 OK, then your data is spherical; go to read here: http://stats.stackexchange.com/q/92791/3277 – ttnphns Sep 11 '16 at 09:32

1 Answers1

1

It becomes arbitrary because you can't rank the eigenvalues: they are all equal. It becomes obvious thinking about the eigendecomposition formulation instead of SVD.

PCA in this scenario makes no sense though: the variables are already principal components. If you apply PCA on this data the PCs will correspond to the original variables, with possible changes in sign introduced by the method.

Firebug
  • 19,076
  • 6
  • 77
  • 139