Sparse Versus Simple Structure Loadings

Abstract

The component loadings are interpreted by considering their magnitudes, which indicates how strongly each of the original variables relates to the corresponding principal component. The usual ad hoc practice in the interpretation process is to ignore the variables with small absolute loadings or set to zero loadings smaller than some threshold value. This, in fact, makes the component loadings sparse in an artificial and a subjective way. We propose a new alternative approach, which produces sparse loadings in an optimal way. The introduced approach is illustrated on two well-known data sets and compared to the existing rotation methods.

Regularized Structural Equation Modeling

“…Instead, loadings below some prespecified threshold are either omitted from display, or are replaced by truncated values. Needless to say, this is a nonoptimal solution to obtaining a sparse structure (Trendafilov & Adachi, 2014). …”

“…The drawback of these alternative cost functions is that they induce model misfit and a reduction in explained variance. In the context of latent variable models, sparse estimation techniques were first applied to PCA through the use of penalization to create sparse loadings (Zou et al, 2006) along with alternative methods of rotation (Trendafilov & Adachi, 2014). Similar methods have also been applied to EFA through the use of penalized maximum likelihood estimation (Choi, Zou, & Oehlert, 2010; Hirose & Yamamoto, 2014a; Ning & Georgiou, 2011).…”

Approximated Penalized Maximum Likelihood for Exploratory Factor Analysis: An Orthogonal Case

“…The reader is directed to Trendafilov (2014) for a review on sparse PCA. In particular, the LASSO-type penalty has been applied by Zou, Hastie, and Tibshirani (2006) and Trendafilov and Adachi (2015) to produce a sparse loading matrix.…”

Sparse principal component analysis subject to prespecified cardinality of loadings

“…Here, we must mention the differences of sparse PCA from the rotation which can be used for facilitating the interpretation of loadings in the standard PCA (Jolliffe 2002;Trendafilov and Adachi 2015). Here, the rotation refers to obtaining orthonormal T so that a number of the elements in AT are close to zero, by exploiting the property that FA in (1) equals FTT A and FT can be substituted for F in (1.2): AT can also be regarded as the loading matrix.…”