Masking misfit in confirmatory factor analysis by increasing unique variances: A cautionary note on the usefulness of cutoff values of fit indices.

Abstract

Fit indices are widely used in order to test the model fit for structural equation models. In a highly influential study, Hu and Bentler (1999) showed that certain cutoff values for these indices could be derived, which, over time, has led to the reification of these suggested thresholds as "golden rules" for establishing the fit or other aspects of structural equation models. The current study shows how differences in unique variances influence the value of the global chi-square model test and the most commonly used fit indices: Root-mean-square error of approximation, standardized root-mean-square residual, and the comparative fit index. Using data simulation, the authors illustrate how the value of the chi-square test, the root-mean-square error of approximation, and the standardized root-mean-square residual are decreased when unique variances are increased although model misspecification is present. For a broader understanding of the phenomenon, the authors used different sample sizes, number of observed variables per factor, and types of misspecification. A theoretical explanation is provided, and implications for the application of structural equation modeling are discussed.

Music-based Autism Diagnostics (MUSAD) – A newly developed diagnostic measure for adults with intellectual developmental disabilities suspected of autism

“…However, the WRMR is seen as an experimental fit index (Muthen, 2014), so we lent more weight to the CFI and RMSEA in the evaluation. The x 2 test was significant, indicating no exact model fit and the need to investigate potential reasons for misfit (Heene et al, 2011). Modification indices (MIs) were evaluated to offer possible explanations.…”

“…Missing data were dealt with using the full information maximum likelihood (FIML) method in Mplus. Model fit was evaluated based on a set of different recommendations (Beauducel & Wittmann, 2005;Heene, Hilbert, Draxler, Ziegler, & Bü hner, 2011;Hu & Bentler, 1999;Yu, 2002). Consequently, besides the global model test using the Chi square statistic, the root mean square errors of approximation (RMSEA), the weighted root mean square residual (WRMR), and the comparative fit index (CFI) were used to evaluate model fit.…”

An examination of the self-referent executive processing model of test anxiety: control, emotional regulation, self-handicapping, and examination performance

“…The measurement model was assessed using a variety of model fit indices including the root mean square error of approximation (RMSEA), standardised root mean square residual (SRMR), comparative fit index (CFI), and Tucker-Lewis index (TLI). Hu and Bentler (1999) suggest a model fit is indicated by RMSEA < 0.05, SRMR < 0.08, and CFI and TLI > 0.95; however, such cut-off values should not be applied too strictly when working with reallife data (Heene et al 2011). The measurement model, estimated using maximum likelihood in Mplus v.8 (Muthén and Muthén 2017), showed a largely good fit to the data, χ 2 (220) = 273.07, p < .001, CFI = 0.954, TLI = 0.942, RMSEA = 0.030, SRMR = 0.047 and no obvious source of misspecification, and so, latent bivariate correlations were examined (see Table 2).…”

Interaction Effects between Openness and Fluid Intelligence Predicting Scholastic Performance

“…All other models were evaluated based on the Comparative Fit Index (CFI), the Standardized Root Mean Square Residual (SRMR), and the Root Mean Square Error of Approximation (RMSEA) with a 90% confidence interval [70][71][72][73][74]. We deemed the fit to be acceptable with cut-offs of CFI ≥ 0.90, RMSEA ≤ 0.08, and SRMR ≤ 0.06 [75]. Models with lower BIC values are expected to be more parsimonious and better-fitting when compared with other nested models [76].…”