2

I have read about similar questions. I have data which has 68 columns and about 800 samples. The 68. column is the output the rest 67 is the input variables. I want to reduce the size of my input variables to for example 30 or 20 variables.

I have read about PCA. I already ran the PCA in Matlab and gathered a 67 x 20 matrix containing PCA coefficients. I calculated eigenvalues for each Principal component (10 eigenvalues). As far as I understand I should order these eigenvalues and select the PCA's with higher eigenvalues as important. For example, I chose PC1, PC3, and PC9.

How can I use this information to select among the original 67 variables? I mean how can I use this PCA analysis results to reduce a 67*800 matrix to a 20*800 matrix and get the variables which have higher effect on the target variable?

Amr Tarek Elmzayen
  • 92
Ferda-Ozdemir-Sonmez
  • 123
  • There are methods for principal variables analysis which aim at selecting variables, as nearly uncorrelated as possible. You would need to search harder for references and implementations. – Nick Cox Feb 21 '19 at 13:22
  • 1
    PCA is about dimensionality reduction (and thus misapplied here), your use case is closer to a process called feature selection. – Joe S Feb 21 '19 at 13:35
  • PCA is a dimensionality reduction technique. However, I think what you need to do first is a feature selection technique to decide which predictors(independent variables) have higher effect on the target(dependent variable) so you will use in model building. So I wish if you could take a look at this answer to a similar question. Answer – Amr Tarek Elmzayen Feb 21 '19 at 12:20

1 Answers1

3

The eigenvalues that you get in Matlab with pca() are already in decreasing order, so you are guaranteed that the first n Principal Components will be the most important ones. You can see this yourself if you look at a vector latent which is a vector of eigenvalues. If then you would like to reduce your variable set to 20 variables, you can simply retain only 20 first PCs from the coeff matrix.

What you do is you project your original data set $X$ (which is 800 $\times$ 67) on a reduced PC basis:

Z = X * coeff(:,1:20)

to get the PC scores matrix $Z$ which is the representation of your original data in a 20-dimensional space instead of 67-dimensional. The matrix $Z$ is size 800 $\times$ 20.

kamilazdybal
  • 812
  • 10
  • 20
  • The OP asked how to select input variables that have the greatest effect on the target variable. PCA is not designed to do this, because it's unsupervised. Using the top principal components (as suggested) maximizes variance explained for the inputs. But, there's no reason these components must retain information about the output. It's possible that informative components have small variance, and are therefore discarded. – user20160 Feb 21 '19 at 14:22