3

Is there any covariance matrix regularization suitable for factor analysis?

I have a data matrix where number of observations is smaller than the number of dimensions: $n<p$.

I am thinking of something like this paper which has been proposed for linear discriminant analysis.

amoeba
  • 104,745
pierre
  • 85
  • FA with complete multicollonearity (as with n<p situation) is impossible (pt 6) or at least theoretically flawed. The reason for that is there is not enough space for all the unique factors assumed to exist. – ttnphns Mar 02 '16 at 18:12
  • 1
    @ttnphns: Nevertheless, some methods of factor extraction (such as principal axis iterations) can work with $n<p$ covariance matrices, and if one considers factor analysis simply as a task of approximating covariance matrix $C$ with a low-rank $LL^\top$ term plus a diagonal $\Psi$ term, then this is still meaningful in $n<p$ case. No idea how useful it is in practice though. – amoeba Mar 03 '16 at 00:20
  • @amoeba, You are correct at that observation. However, many FA programs will just show the user with singular input data the door. And it is justified. – ttnphns Mar 03 '16 at 02:25

1 Answers1

3

I see that the question is old but maybe some one finds the answer still helpful. Recently, there were quite some papers published on this, here are some references, you might want to check out:

Jung, S., & Takane, Y. (2007). Regularized common factor analysis. New Trends in Psychometrics, 1(1), 1–10.

Jung, S., & Lee, S. (2011). Exploratory factor analysis for small samples. Behavior Research Methods, 43(3), 701–9.

Jacobucci, R., Grimm, K. J., & McArdle, J. J. (2016). Regularized Structural Equation Modeling. Structural Equation Modeling, 23(4), 55–566.

Bai, J., & Liao, Y. (2016). Efficient estimation of approximate factor models via penalized maximum likelihood. Journal of Econometrics, 191(1), 1–18.

Hirose, K., & Yamamoto, M. (2015). Sparse estimation via nonconcave penalized likelihood in factor analysis model. Statistics and Computing, 25(5), 863–875.

Hirose, K., & Yamamoto, M. (2014). Estimation of an oblique structure via penalized likelihood factor analysis. Computational Statistics and Data Analysis, 79(Kaiser), 120–132.

Hirose, K., & Yamamoto, M. (2015). Sparse estimation via nonconcave penalized likelihood in factor analysis model. Statistics and Computing, 25(5), 863–875.

Trendafilov, N. T., & Adachi, K. (2015). Sparse Versus Simple Structure Loadings. Psychometrika, 80(3), 776–790.

Trendafilov, N. T., Fontanella, S., & Adachi, K. (2017). Sparse Exploratory Factor Analysis. Psychometrika, (2011).

Some of these techniques are already implemented in R packages (e.g., regSEM or fanc), so they are quite accessible.

Flo Schafzähler
  • 46