Model fit is a fallible indicator of model quality in quantitative psychopathology research: A reply to Bader and Moshagen.

Abstract

As evidenced by our exchange with Bader and Moshagen (2022), the degree to which model fit indices can and should be used for the purpose of model selection remains a contentious topic. Here, we make three core points. First, we discuss the common misconception about fit statistics' abilities to identify the "best model," arguing that mechanical application of model fit indices contributes to faulty inferences in the field of quantitative psychopathology. We illustrate the consequences of this practice through examples in the literature. Second, we highlight the parsimony-adjacent concept of fitting propensity, which is not accounted for by commonly used fit statistics. Finally, we present specific strategies to overcome interpretative bias and increase generalizability of study results and stress the importance of carefully balancing substantive and statistical criteria in model selection scenarios.

Developmental trajectories of mother–adolescent agreement on maternal autonomy support and their contributions to adolescents’ adjustment.

“…This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. robust support to our findings, while attending to measurement quality of scales, which impacts model fit (Greene et al, 2022). Future research should replicate these findings in father-adolescent dyads.…”

Validation and Psychometric Analysis of the German Translation of the Appraisal of Self-Care Agency Scale-Revised

“…Lastly, as Greene et al (2022) [ 72 ] reported, model fit and fit measures are highly dependent on the type of data and the factor analysis method. Therefore, the deviation of our item attribution from the original ASAS structure by Sousa et al (2011) and other previous publications must be interpreted in light of the methods and participants included in the respective studies.…”

“…Therefore, the deviation of our item attribution from the original ASAS structure by Sousa et al (2011) and other previous publications must be interpreted in light of the methods and participants included in the respective studies. As Greene et al (2022) [ 72 ] indicated, model-based fit measures should not be seen as ultimate but instead be interpreted with regards to content and theoretical frameworks. In our analysis, we therefore not only looked at the best model fit but also decided that our variable attribution seems more reasonable in terms of actual content of the identified factors.…”

A critical evaluation of the p-factor literature

“…Foremost, we encourage p-factor researchers to abandon the unrestricted bifactor model and follow Waller and Meehl 80 : "Efforts to falsify theories by subjecting them to risky tests are the most efficient means of gauging a theory's mettle" (p. 331). Accordingly, researchers should drastically de-emphasize or disregard goodness-of-fit in bifactor modeling, 73,79,112 as they do when evaluating exploratory models. 113 A riskier, more rigorous test of the appropriateness of a bifactor model calls for incorporating plausible constraints, such as Bayesian constraints according to a priori theories.…”

“…131 Typically associated with relatively high model fit compared with other models, but strong fit does not indicate that the model is an adequate or valid description of the data. 71,73,112 Higher-order…”