Croon’s Bias-Corrected Factor Score Path Analysis for Small- to Moderate-Sample Multilevel Structural Equation Models

Abstract

Maximum likelihood estimation of multilevel structural equation model (MLSEM) parameters is a preferred approach to probe theories involving latent variables in multilevel settings. Although maximum likelihood has many desirable properties, a major limitation is that it often fails to converge and can incur significant bias when implemented in studies with a small to moderate multilevel sample (e.g., fewer than 100 organizations with 10 or less individuals/organization). To address similar limitations in single-level SEM, literature has developed Croon’s bias-corrected factor score path analysis estimator that converges more regularly than maximum likelihood and delivers less biased parameter estimates with small to moderate sample sizes. We derive extensions to this framework for MLSEMs and probe the degree to which the estimator retains these advantages with small to moderate multilevel samples. The estimator emerges as a useful alternative or complement to maximum likelihood because it often outperforms maximum likelihood in small to moderate multilevel samples in terms of convergence, bias, error variance, and power. The proposed estimator is implemented as a function in R using lavaan and is illustrated using a multilevel mediation example.

Specifying Location-Scale Models for Heterogeneous Variances as Multilevel SEMs

“…To contextualize this in the context of organizational research, perhaps the effect of leadership climate on task efficiency is mediated by the within-team variability of individual empowerment such that teams with a strong leadership climate are more homogeneous on individual empowerment, which leads to improved efficiency because team members work more cohesively. Whereas mediation would be a difficult leap for a heterogeneous variance model in a mixed effect framework and would likely require a multistage model whose estimates would require Croon’s correction (Croon, 2002) to be unbiased (Devlieger & Rosseel, 2017; Kelcey, Cox, & Dong, 2019), mediation is another straightforward extension if modeling the random scale effects in the multivariate multilevel SEM framework (Hoffman, 2019). McNeish and Hamaker (2020) noted such advantages of multilevel SEM over mixed effects models, noting that multilevel SEM allows any hypothesized structural relations to be modeled between random effects.…”

Optimal Design of Cluster- and Multisite-Randomized Studies Using Fallible Outcome Measures

“…We have highlighted the detrimental effects of measurement error in the outcome when planning multilevel experiments, extended the tools available to efficiently plan these experiments, and provided additional recommendations to improve evaluation planning but recognize the vast scope of multilevel experimental designs and the current extensions of the optimal sample allocation framework far exceed what is discussed here. For instance, subsequent work should consider how measurement error influences experimental design when probing moderation and mediation effects (e.g., Kelcey, Dong, Spybrook, & Cox, 2017; Kelcey, Dong, Spybrook, & Shen, 2017; Kelcey et al, 2019; Spybrook, Kelcey, & Dong, 2016). We also encourage future work to expand considerations of measurement error into other designs such as three- and four-level CRTs and multisite CRTs.…”

“…Under these conditions, one alternative strategy is structural equation modeling (SEM) with multiple indicators (e.g., Ledgerwood & Shrout, 2011; Li & Beretvas, 2013). This approach properly considers measurement error associated with latent variables and recent advancements continue to expand the utility of SEM in evaluations of varying size and scope (e.g., Kelcey et al, 2019).…”

“…For instance, a recent review of the design quality and capacity of multilevel experiments in education suggests that the vast majority overlooked the possibility of measurement error (Spybrook et al, 2016). Similar reviews in organizational research have suggested that upwards of 90% of the sampled studies disregarded the possibility of measurement error (Aguinis, Edwards, & Bradley, 2017; Kelcey, Cox, & Dong, 2019). The lack of attention to and planning for outcome measurement error in the design of multilevel experiments may be in part attributable to the lack of accessible approaches to account for such error in the design phase and the lack available user-friendly tools.…”

Specifying Location–Scale Models for Heterogeneous Variances as Multilevel SEMs

“…Despite the frequency of questions about cohesion and heterogeneity in the organizational literature, Lang et al (2018Lang et al ( , 2019 for a heterogeneous variance model in a mixed effect framework and would likely require a multistage model whose estimates would require Croon's correction (Croon, 2002) in order to be unbiased (Devlieger & Rosseel, 2017;Kelcey, Cox, & Dong, 2019), mediation is another straightforward extension if modeling the random scale effects in the multivariate multilevel SEM framework (Hoffman, 2019). McNeish and Hamaker (2020) note such advantages of multilevel SEM over mixed effects models, noting that multilevel SEM allows any hypothesized structural relations to be modeled between random effects.…”