Dynamic fit index cutoffs for confirmatory factor analysis models.

Abstract

Model fit assessment is a central component of evaluating confirmatory factor analysis models and often the validity of psychological assessments. Fit indices remain popular and researchers often judge fit with fixed cutoffs derived by Hu and Bentler (1999). Despite their overwhelming popularity, methodological studies have cautioned against fixed cutoffs, noting that the meaning of fit indices varies based on a complex interaction of model characteristics like factor reliability, number of items, and number of factors. Criticism of fixed cutoffs stems primarily from the fact that they were derived from one specific confirmatory factor analysis model and lack generalizability. To address this, we propose a simulation-based method called dynamic fit index cutoffs such that derivation of cutoffs is adaptively tailored to the specific model and data characteristics being evaluated. Unlike previously proposed simulation-based techniques, our method removes existing barriers to implementation by providing an open-source, web-based Shiny software application that automates the entire process so that users neither need to manually write any software code nor be knowledgeable about foundations of Monte Carlo simulation. Additionally, we extend fit index cutoff derivations to include sets of cutoffs for multiple levels of misspecification. In doing so, fit indices can more closely resemble their originally intended purpose as effect sizes quantifying misfit rather than improperly functioning as ad hoc hypothesis tests. We also provide an approach specifically designed for the nuances of one-factor models, which have received surprisingly little attention in the literature despite frequent substantive interests in unidimensionality.

Because excellencism is more than good enough: On the need to distinguish the pursuit of excellence from the pursuit of perfection.

“…To address these issues, McNeish and Wolf (2021) have recently created simulation codes to calculate the optimal fit that a model can realistically attain based on its characteristics and sample size (https://dynamicfit.app/cfa/). Given the results of Sample 1, the following optimal criteria would correctly reject 95% of the misspecified models while incorrectly rejecting 5% of the correctly specified models: CFI ≥ .951, RMSEA ≤ .078, and SRMR ≤ .057.…”

An international validation of the Bolton Unistride Scale (BUSS) of tenacity

“…As our models do not reflect the same conditions by which those fit indices were derived, our use of arbitrary cut-offs may make precise assessment of model fit or mis-specification difficult. The use of dynamic fit indices for CFA [56] is a novel development which would improve our ability to discern good model fit. At present, however, dynamic fit indices for bifactor models cannot be obtained.…”

The Hitchhiker’s Guide to Longitudinal Models: A Primer on Model Selection for Repeated-Measures Methods

“…While we have mentioned previously that MEMs (and especially MLMs) are specialized latent variable models and could fall under the general SEM umbrella (Bauer, 2003;Curran, 2003), there are conventions that tend to differ between SEMs and MEMs due to their historical prevalence/development in different fields. In general, SEMs focus not only on relative fit (e.g., likelihood ratios, AIC/BIC comparisons) but also on measures of "absolute" fit (e.g., CFI/TLI/RMSEA) which assess the degree to which our imposed model structure reproduces the characteristics (means and covariances) of the observed (unstructured) data (Bollen et al, 2014;Bollen & Stine, 1992;Hu & Bentler, 1998;Jackson et al, 2009;McNeish & Wolf, 2021;Satorra & Bentler, 2001;Widaman & Thompson, 2003). Furthermore, SEMs are inherently a multivariate modeling framework (even when modeling a single construct; more on this later) and naturally extend to multiple outcomes.…”

“…observed characteristics of the sample data 8 (Hu & Bentler, 1998;Jackson et al, 2009;McNeish & Wolf, 2021).…”