An Empirical Evaluation of the Use of Fixed Cutoff Points in RMSEA Test Statistic in Structural Equation Models

Abstract

This article is an empirical evaluation of the choice of fixed cutoff points in assessing the root mean square error of approximation (RMSEA) test statistic as a measure of goodness-of-fit in Structural Equation Models. Using simulation data, the authors first examine whether there is any empirical evidence for the use of a universal cutoff, and then compare the practice of using the point estimate of the RMSEA alone versus that of using it jointly with its related confidence interval. The results of the study demonstrate that there is little empirical support for the use of .05 or any other value as universal cutoff values to determine adequate model fit, regardless of whether the point estimate is used alone or jointly with the confidence interval. The authors' analyses suggest that to achieve a certain level of power or Type I error rate, the choice of cutoff values depends on model specifications, degrees of freedom, and sample size. KeywordsRMSEA; SEM; goodness-of-fit; computer simulations Structural Equation Modeling (SEM) has been widely used in sociological, psychological, and social science research. One of the appealing attributes of SEM is that it allows for tests of theoretically derived models against empirical data. For researchers using SEM techniques, evaluation of the fit of a hypothesized model to sample data is crucial to the analysis. A key feature of SEM is the test of the null hypothesis of ∑ = ∑ = (θ), also known as the test of exact fit, where ∑ is the population covariance matrix, ∑(θ) is the covariance matrix implied by a specific model, and θ is a vector of free parameters defined by the model. The model test statistic T enables an asymptotic test of the null hypothesis of H 0 : ∑ = ∑ (θ). A significant T, often reported as the model chi-square, would suggest misspecification of the model. However, such a test of exact fit of the proposed model is generally unrealistic, as hardly any model using real data is without error (e.g., Browne and Cudeck 1993). A trivial misspecification, particularly with large sample sizes, can lead to rejection of the model even when it may otherwise adequately reproduce the population covariance matrix.

Examining the factor structures of the Five Facet Mindfulness Questionnaire and the Self-Compassion Scale.

“…There are rules of thumb when choosing cut-off points to say that a model fit is acceptable or poor, from the fit indices provided by structural equation modelling statistical software (Hu & Bentler, 1998, 1999Schermelleh-Engel, Moosbrugger, & Müller, 2003). Suggested cut-offs for specific fit indices vary and should be used with caution since indices are influenced by sample size, model parameters, and data normality (e.g., Chen, Curran, Bollen, Kirby, & Paxton, 2008;Marsh, 2004;Nye & Drasgow, 2011). It is recommended that researchers report several indices rather than relying on a single type, since different indices together provide complimentary information (e.g., Kline, 2005).…”

Specifying Turning Point in Piecewise Growth Curve Models: Challenges and Solutions

“…However, there has been controversy regarding the advocacy for the proposed fixed cutoff criteria. Applied researchers were warned against a complete reliance on fixed cutoff criteria in assessing model fit (see e.g., [23][24][25]). Simulation studies have been done in the context of confirmatory factor analysis (CFA) models, evaluating the performance of the fit indices in identifying misspecification in covariance structures (see e.g., [21,23,24]).…”

Is Sport Sponsorship Global? Evidence from the United States, the United Kingdom, and India

“…This study's RMSEA values were slightly over this estimate, as Hu and Bentler (1999) suggested RMSEA values between .05 and .08 to indicate a fair fit, but recent research cautioned about using precise cutoff points for RMSEA (cf. Chen, Curran, Bollen, Kirby, & Paxton, 2008), so the values were considered acceptable.…”