Model fit evaluation in multilevel structural equation models

Abstract

Assessing goodness of model fit is one of the key questions in structural equation modeling (SEM). Goodness of fit is the extent to which the hypothesized model reproduces the multivariate structure underlying the set of variables. During the earlier development of multilevel structural equation models, the “standard” approach was to evaluate the goodness of fit for the entire model across all levels simultaneously. The model fit statistics produced by the standard approach have a potential problem in detecting lack of fit in the higher-level model for which the effective sample size is much smaller. Also when the standard approach results in poor model fit, it is not clear at which level the model does not fit well. This article reviews two alternative approaches that have been proposed to overcome the limitations of the standard approach. One is a two-step procedure which first produces estimates of saturated covariance matrices at each level and then performs single-level analysis at each level with the estimated covariance matrices as input (Yuan and Bentler, 2007). The other level-specific approach utilizes partially saturated models to obtain test statistics and fit indices for each level separately (Ryu and West, 2009). Simulation studies (e.g., Yuan and Bentler, 2007; Ryu and West, 2009) have consistently shown that both alternative approaches performed well in detecting lack of fit at any level, whereas the standard approach failed to detect lack of fit at the higher level. It is recommended that the alternative approaches are used to assess the model fit in multilevel structural equation model. Advantages and disadvantages of the two alternative approaches are discussed. The alternative approaches are demonstrated in an empirical example.

Fair treatment, fair play? The relationship between fair treatment perceptions, political trust and compliant and cooperative attitudes cross‐nationally

“…However, the overall fit measures could potentially fail to identify misfit on the between level because the much larger sample size of the within level dominates the overall fit. To thoroughly assess the model fit, we apply the partial saturated method developed by Ryu and West (; see also Ryu ) and manually calculate level‐specific fit indices. This procedure yields acceptable fit on the within (CFI = 0.92, RMSEA = 0.03) and between level (CFI = 0.97) (more information on this method can be found in Online Appendix ).…”

The Relations Among School Climate, Instructional Quality, and Achievement Motivation in Mathematics

“…As instructional quality and the motivational constructs were measured at both the student and the classroom level, we tested for measurement invariance at these two levels by conducting (a) single-level MGCFA, and (b) multilevel MGCFA. For the latter, the student (individual) level was saturated, assuming only correlations among all items of a scale (Ryu 2014). For the school climate constructs, however, only (b) applied, because they were measured by teacher ratings.…”

Modified Servqual Model and Effects of Customer Attitude and Technology on Customer Satisfaction in Banking Industry: Mediation, Moderation and Conditional Process Analysis

“…According to Balakrishnan et al (2016) and Meyers et al (2006), there are more than 20 fit indices and no agreement which to report for measuring the hypothesized model. According to Balakrishnan et al (2016), Ryu (2014), and Byrne (2009), these measures have been characterized into four indices categories such as: 1) absolute fit index, 2) Relative fit index, 3) Noncentrality based index and 4) Parsimonious fit index. Complete cut off results for the considered indices of this study have been reported in Table 5.…”