Regularized Structural Equation Modeling

Abstract

A new method is proposed that extends the use of regularization in both lasso and ridge regression to structural equation models. The method is termed regularized structural equation modeling (RegSEM). RegSEM penalizes specific parameters in structural equation models, with the goal of creating easier to understand and simpler models. Although regularization has gained wide adoption in regression, very little has transferred to models with latent variables. By adding penalties to specific parameters in a structural equation model, researchers have a high level of flexibility in reducing model complexity, overcoming poor fitting models, and the creation of models that are more likely to generalize to new samples. The proposed method was evaluated through a simulation study, two illustrative examples involving a measurement model, and one empirical example involving the structural part of the model to demonstrate RegSEM's utility. Keywordsfactor analysis; lasso; penalization; regularization; ridge; shrinkage; structural equation modeling The desire for model simplicity in exploratory factor analysis (EFA) has led to the longstanding goal of achieving simple structure. By simple structure, we are broadly referring to a pattern of factor loadings where each indicator is influenced by a minimal number of latent factors, preferably one. Multiple researchers have set different criteria for achieving simple structure, with the most notable being Thurstone (1935Thurstone ( , 1947. However, the desire for simplicity in model structure comes by many names, including the aforementioned simple structure (Thurstone, 1935), variable complexity (Browne, 2001, parsimony (Marsh & Hau, 1996;Raykov & Marcoulides, 1999), "sparse loadings" in the context of principal components analysis (PCA; Zou, Hastie, & Tibshirani, 2006), and finally, sparsistency, denoting that all parameters in a sparse model that are zero are correctly estimated as zero with probability tending to one (Lam & Fan, 2009). Across multiple fields and different terminology, the goal is roughly the same: to accurately and efficiently estimate a model that is parsimonious in allowing users to easily interpret the model's representation of reality.However, simpler is not always better. Overconstraining models in an attempt to make interpretation easier, can lead to unacceptable levels of misfit and biased parameter estimates Correspondence should be addressed to Ross Jacobucci, Department of Psychology, University of Southern California, SGM 501, 3620 South McClintock Avenue, Los Angeles, CA 90089, USA. jacobucc@usc.edu. HHS Public Access Author Manuscript Author ManuscriptAuthor ManuscriptAuthor Manuscript (Hsu, Troncoso Skidmore, Li, & Thompson, 2014;Muthén & Asparouhov, 2012). In a confirmatory factor analysis (CFA), biased parameter estimates manifest as inflated covariances between latent factors caused by not allowing cross-loadings or residual covariances. It is not that the goal of simple structure is a poor choice, but that blind adherence to this st...

Regularized Structural Equation Modeling with Stability Selection

“…Similar to what was found in previous studies (Jacobucci, Grimm & McArdle, 2016 we are targeting at (e.g. FPRs and FNRs as within the dash-dotted region in right panel of Figure 1).…”

Are We Measuring Teachers’ Attitudes towards Computers in Detail?: Adaptation of a Questionnaire into Turkish Culture

“…Standardized coefficients, indicating the relationships between the items and relevant factors, ranged from .28 to .64 and were significant at .01. In general, by taking a closer look at the modelfit indexes it can be concluded that the model perfectly fits with RMSEA = 0.050, χ 2 /df=1.69 values (Tabachnick & Fidell, 2001;Jacobucci, Grimm & McArdle, 2016).…”

Exploratory Mediation Analysis with Many Potential Mediators

“…The SEM and filter methods were implemented using the lavaan package (Rosseeel, 2012), and CMF was implemented using the accompanying cmfilter package. For XMed, the regsem package (Jacobucci et al, 2016) was used with cross-validation was to find the optimal penalty parameter, and any variables with nonzero α and β paths were considered mediators. HIMA was run according to its implementation in the R package HIMA (Zhang et al, 2016), again with a p-value of .1.…”