0

We are learning about Principal Component analysis in our class, and I having trouble understanding how to compute the principal component given a matrix. For example, here is the matrix we were given. We were asked to compute the SVD of the matrix, but how is the SVD of the matrix used to compute the first principal component? This is where I am confused.

enter image description here

Harry Lofi
  • 303

1 Answers1

1

Preamble

So, as you know singular value decomposition (SVD) decomposed a matrix $X$ (with $i$ rows and $j$ columns) into three matrices:

$X = U\Sigma T^T$,

where,

$U$ is an $i$ x $i$ matrix whose columns are the left singular vectors of $X$,

$\Sigma$ is an $i$ x $j$ matrix containing the singular values (in descending order),

and, $V^T$ is an $j$ x $j$ orthogonal matrix whose columns are the right singular vectors of $X$.

Answer

We were asked to compute the SVD of the matrix, but how is the SVD of the matrix used to compute the first principal component? This is where I am confused.

After you compute the SVD$^1$ of $X$, you can identify the first principal component as the first column of $V$ (not $V^T$, but $V$).

Finally, working through examples using R helped me learn matrix algebra concepts, so see the R code below.

# Your data
X <- as.matrix(
   rbind(
     c(-2,-2,-2,0,0),
     c(-2,-2,-2,0,0),
     c(4,4,4,0,0),
     c(0,0,0,-1,-1),
     c(0,0,0,0,0)
   )
 )

Mean centering


X_centered <- scale(X, center = TRUE, scale = FALSE)


SVD


svd_result <- svd(X_centered)


Extracting the first principal component


first_principal_component <- svd_result$v[, 1]


print(first_principal_component)

$^1$ Remember before you compute the SVD, make sure your data is mean centered. Not only does doing this improve numerical stability. It also helps with interpretability, as it allows for principle components to be interpreted as variations from the average of the dataset.

Preston Botter
  • 2,611