4

When describing the condition number of a correlation matrix I have seen is described as the ratio of the singular values (from singluar value decomposition).

I have also seen it described as the ratio of the eigenvalues (from PCA analysis).

Can both be correct (under say different norms)? If so which one would be considered standard?

Baz
  • 1,763
  • 3
  • 16
  • 27
  • 1
    For a square positive definite matrix, such as a correlation matrix, singular values are exactly the same thing as eigenvalues! Singular values can be seen as a "generalization" of eigenvalues for non-square matrices. Or do you mean singular values of the data matrix (as opposed to the correlation matrix)? – amoeba Nov 20 '14 at 13:25

1 Answers1

5

In the context in which you are using the condition number, neither number is interpretable in itself, but is always to be compared to a threshold $t$. So long as one uses the threshold consistently, it doesn't much matter which one of the criterion is used.

Edit:

@whuber comments that in many contexts, the actual value, too, can be interpreted. One interpretation that is related to the the computation of covariance matrices (and their inverses) is from computer science.

The following interpretation of the condition number is often used as a rule of thumb in numerical programing. If you define the condition number as:

$$k=\left\lceil\frac{1}{2}\log\frac{\lambda_1}{\lambda_p}\right\rceil$$

where $\lambda_p$ ($\lambda_1$) is the smallest (largest) eigenvalue of the original matrix and $\log$ is the base 10 logarithm, then $k$ can be interpreted as the number of digits of accuracy one may lose on top of what would be lost to the numerical method due to loss of precision from arithmetic methods.

Community
  • 1
user603
  • 22,585
  • 3
  • 83
  • 149
  • 2
    I think half of this answer may be right: you want to know whether you have calculated a condition number or its square when you are comparing it to somebody's recommended threshold. But condition numbers do have absolute meaning in geometry and analysis as well as in statistics. – whuber Feb 02 '14 at 19:34
  • But for a correlation matrix, singular values are exactly the same as the eigenvalues! So it is not two criteria, it is one criterion. Or do you think that OP was referring to the singular values of the data matrix (not correlation matrix)? – amoeba Nov 20 '14 at 13:29
  • 1
    For the reason you point out, I assumed the OP meant the SVD of the [centered] data matrix (or else, the question wouldn't make much sense IMO). – user603 Nov 20 '14 at 14:32