The effect of estimation method and sample size in multilevel structural equation modeling

Abstract

Multilevel structural equation modeling (multilevel SEM) has become an established method to analyze multilevel multivariate data. The first useful estimation method was the pseudobalanced method. This method is approximate because it assumes that all groups have the same size, and ignores unbalance when it exists. In addition, full information maximum likelihood (ML) estimation is now available, which is often combined with robust chi-squares and standard errors to accommodate unmodeled heterogeneity (MLR). In addition, diagonally weighted least squares (DWLS) methods have become available as estimation methods. This article compares the pseudobalanced estimation method, ML(R), and two DWLS methods by simulating a multilevel factor model with unbalanced data.The simulations included different sample sizes at the individual and group levels and different intraclass correlation (ICC). The within-group part of the model posed no problems. In the between part of the model, the different ICC sizes had no effect. There is a clear interaction effect between number of groups and estimation method. ML reaches unbiasedness fastest, then the two DWLS methods, then MLR, and then the pseudobalanced method (which needs more than 200 groups). We conclude that both ML(R) and DWLS are genuine improvements on the pseudobalanced approximation. With small sample sizes, the robust methods are not recommended.

Measuring physiological influence in dyads: A guide to designing, implementing, and analyzing dyadic physiological studies.

“…One can also use multilevel structural equation modeling (MSEM) to estimate the model. We note that MSEM requires many more data points than MLM (Hox, Maas, and Brinkhuis, 2010; see also Hox, 2013) and so an MSEM approach is more appropriate for large sample sizes. We next provide overviews of three topics that are important for building an analysis model: the structure of repeated measures dyadic data, centering of variables, and whether to use raw values or change scores.…”

Math self-concept, grades, and achievement test scores: Long-term reciprocal effects across five waves and three achievement tracks.

“…All models were conducted using the robust maximum likelihood estimator (MLR) which is robust against non-normality of the observed variables (Hox, Maas, & Brinkhuis, 2010;Muthén & Muthén, 1998. The Mplus option <type = complex> was used to accommodate the hierarchical nature of the study.…”

Sex differences in longitudinal pathways from parenting to delinquency

“…Because the dependent variable (i.e., total delinquency frequency), was a right skewed count variable, negative binomial models were estimated using maximum likelihood estimation with robust standard errors (Yuan and Bentler 1998;Hox et al 2010). All indirect effects were estimated in Mplus, which uses the product of coefficients method for testing mediation analyses.…”