Singular value and eigen-decomposition of a square symmetric matrix should be identical, but differ in sign

Question

As far as I know, singular value decomposition (SVD) and eigendecomposition give the same result for symmetric square matrices. But when I check the results in R, that's not what I see. Please see below R code (I set the random seed for reproducibility purposes):

set.seed(111)
X = matrix(rnorm(50), nrow=5, ncol=10)
prinden = svd(X %*% t(X))
prineden = eigen(X %*% t(X))

> prinden$u
       [,1]       [,2]        [,3]       [,4]        [,5]
[1,] -0.3812420  0.1714440  0.89298930 -0.1352677  0.09764363
[2,] -0.3086450 -0.1785482  0.05935694  0.9181008 -0.16256214
[3,]  0.2125363  0.8719446 -0.06555398  0.3011303  0.31553776
[4,] -0.6109540  0.4041138 -0.30752420 -0.2079534 -0.57062372
[5,] -0.5839147 -0.1230124 -0.31650973 -0.0697814  0.73407341

> prineden$vectors
       [,1]       [,2]        [,3]       [,4]        [,5]
[1,] -0.3812420 -0.1714440  0.89298930 -0.1352677  0.09764363
[2,] -0.3086450  0.1785482  0.05935694  0.9181008 -0.16256214
[3,]  0.2125363 -0.8719446 -0.06555398  0.3011303  0.31553776
[4,] -0.6109540 -0.4041138 -0.30752420 -0.2079534 -0.57062372
[5,] -0.5839147  0.1230124 -0.31650973 -0.0697814  0.73407341

The 2nd columns of prinden$u and prineden$vectors are negative of each other, while other columns are the same. How come is this possible? What am I missing?

Sign does not matter. It depends on the specific implementation of the function. If you negate the second column the results are identical. So these are actually two equivalent decomposition outputs. — ttnphns, May 24 '15 at 04:55
This is very closely related to http://stats.stackexchange.com/questions/88880. — amoeba, May 24 '15 at 07:02

score 2 · Answer 1 · answered May 24 '15 at 13:01

2

Also, if your square matrix is indefinite (and thus has negative eigenvalues), the SVD won't capture that since it always returns nonnegative singular values.

answered May 24 '15 at 13:01

Brian Borchers

5,263

Has QUIT--Anony-Mousse · Accepted Answer · 2015-05-25T00:24:53.850

1

The decomposition is not unique.

First of all, any reflection is a proper decomposition, too.

Secondly, if you have identical eigenvalues, any rotation within these axes is a proper decomposition, too.

In particular, any identity ("eye") + rotation matrix is a decomposition of any other identity ("eye") or rotation matrix (because any vector is a unit vector of them).

Such solutions do arise in practise when you use e.g. power iterations to find only the first few eigenvectors (numerically, not analytically, obviously. Reality isn't exact.)

edited May 25 '15 at 00:24

answered May 24 '15 at 10:09

Has QUIT--Anony-Mousse

42,358

I don't think "eye matrix" is a mathematical term. You are probably referring to the eye() function in Matlab, but what it returns is called an identity matrix. – amoeba May 24 '15 at 10:38
eye matrix may be less ambiguous. Identity can happen on so many levels. Personally I would have preferred the term identity, but have the impression some people find eye more understandable for stats users. – Has QUIT--Anony-Mousse May 25 '15 at 00:24

Singular value and eigen-decomposition of a square symmetric matrix should be identical, but differ in sign

2 Answers2