2

I performed PCA on 33 items with 133 observations. Considering the criteria to take components with eigenvalues >1, 4 factors can be extracted. I then did varimax rotation of those. However, I noticed one variable load under different components. So in this case, how can I say which variables load under a particular component?

Or what could have gone wrong in my analysis?

New Image for PCA following the comments.enter image description here

enter image description here

ttnphns
  • 57,480
  • 49
  • 284
  • 501
Benu P Dahal
  • 21
  • Please study this https://stats.stackexchange.com/q/185216/3277. (Irrespective of whether you are doing PCA or FA.) – ttnphns Mar 28 '23 at 11:36

2 Answers2

2

As answered by Eoin, you are asking about PCA but in your analyses, it seems you are performing an exploratory (?) Factor Analysis (FA); for this reason, I added the factor-analyisis tag. Although there is a great deal of overlap between PCA and FA, they are not exactly the same.

Now, there is no reason for the loadings of a given variable to be a priori non-zero for only one latent dimension. Indeed, loadings of a variable can be zero everywhere or nonzero everywhere. After all, FA is nothing more than a model in which you assume the target covariance matrix $\Sigma$ can be factored

$$ \Sigma = L L^\top + \Psi, $$ where $L$ is the loadings matrix and $\Psi$ is the diagonal matrix of specific variances. $L,\Psi$ are unknown parameters that you can estimate through maximum likelihood, PCA. etc.

If you use maximum likelihood, some identifiability constraints have to be placed. But, the point is that, unless you place suitable constraints on $L$, its estimate $\hat L$ will be a dense matrix.

Factor rotation methods, such as varimax or promax, are sometimes (but not always) helpful in this respect, since they tend to shrink low-valued loadings towards zero (i.e. the "holes" in the rotated loading matrix represent zeros) delivering thus more easily-interpretable results.

utobi
  • 11,726
1
  • The output you've shown is for exploratory factor analysis, not PCA. There are many tutorials out there that explain the relationship between PCA and factor analysis.
  • In both PCA and exploratory factor analysis, every variable will have some loading on every component. The varimax rotation you've used attempts to find a solution with a few large loadings and most of the rest as close to 0 as possible (I'm simplifying, but that's the general idea). Your software is only showing you loadings over a particular value (probably >0.35), but in reality there's a loading in every cell of your table.
Eoin
  • 8,997
  • I am sorry for the confusion the image created. Indeed, it was PCA but unfortunately, I uploaded a wrongimage. This helps. – Benu P Dahal Mar 28 '23 at 09:49