Is the Bifactor Model a Better Model or Is It Just Better at Modeling Implausible Responses? Application of Iteratively Reweighted Least Squares to the Rosenberg Self-Esteem Scale

Abstract

Although the structure of the Rosenberg Self-Esteem Scale (RSES; Rosenberg, 1965) has been exhaustively evaluated, questions regarding dimensionality and direction of wording effects continue to be debated. To shed new light on these issues, we ask: (1) for what percentage of individuals is a unidimensional model adequate, (2) what additional percentage of individuals can be modeled with multidimensional specifications, and (3) what percentage of individuals respond so inconsistently that they cannot be well modeled? To estimate these percentages, we applied iteratively reweighted least squares (IRLS; Yuan & Bentler, 2000) to examine the structure of the RSES in a large, publicly available dataset. A distance measure, ds, reflecting a distance between a response pattern and an estimated model, was used for case weighting. We found that a bifactor model provided the best overall model fit, with one general factor and two wording-related group factors. But, based on dr values, a distance measure based on individual residuals, we concluded that approximately 86% of cases were adequately modeled through a unidimensional structure, and only an additional 3% required a bifactor model. Roughly 11% of cases were judged as “unmodelable” due to their significant residuals in all models considered. Finally, analysis of ds revealed that some, but not all, of the superior fit of the bifactor model is owed to that model’s ability to better accommodate implausible and possibly invalid response patterns, and not necessarily because it better accounts for the effects of direction of wording.

Appropriate Use of Bifactor Analysis in Psychopathology Research: Appreciating Benefits and Limitations

“…The bifactor model's flexibility can also enable it to show superior global fit than alternatives, even when the other models were themselves used to simulate data (33,(38)(39)(40)(41)(42)(43)(44)(45)(46). For example, skewed item distributions and unmodeled cross-loadings or correlated residuals can all lead fit statistics to favor the bifactor model over a correlated-factors model (with no general factor), even if the correlatedfactors model more accurately describes the true structure (46,47).…”

“…For example, skewed item distributions and unmodeled cross-loadings or correlated residuals can all lead fit statistics to favor the bifactor model over a correlated-factors model (with no general factor), even if the correlatedfactors model more accurately describes the true structure (46,47). The bifactor model's flexibility can also result in good model-data fit even when used with very noisy data or nonsense response patterns (33,45). Thus, it is inappropriate to use global fit statistics to evaluate the bifactor model or favor it over alternative models (29,48).…”

Kidney Disease Quality of Life 36-Item Short Form Survey (KDQOL-36) Normative Values for the United States Dialysis Population and New Single Summary Score

“…This model evaluates how well a general factor, in addition to specific factors, account for the latent structure of a measure. 22 For this model, we included three specific factors matching those from the three-correlated-factors model, each with the same items loading on the Burdens of Kidney Disease, Symptoms and Problems of Kidney Disease, and Effects of Kidney Disease factors as described above, except the factors are left uncorrelated. In addition, each item loads directly onto a general factor (Supplemental Appendix 2 [Supplemental Figure 4]).…”

Riskier Tests of the Validity of Bifactor Models of Psychopathology

“…Some studies do find that the general factor in a bifactor model is defined by distress (Lahey et al, 2012;Lahey et al, 2015;Olino et al, 2014;Waldman et al, 2016), but others find that the general factor is defined by externalizing (Castellanos-Ryan et al, 2016), thought disorder (Caspi et al, 2014), or even autism spectrum disorders (Martel, Pan, et al, 2017;Noordhof et al, 2015). These cross-study discrepancies have given rise to a dizzying array of interpretations of the p-factor , but are also potentially consistent with the bifactor model's tendency to fit any data (Reise et al, 2016).…”

“…First, bifactor models are vulnerable to overfitting data, engendering their parameter estimates unstable (e.g., Bonifay, 2015;Reise, Kim, Mansolf, & Widaman, 2016). Second, various simulation studies have demonstrated that bifactor models are prone to fitting any possible data, including potentially invalid (Reise et al, 2016) or random response patterns (Bonifay & Cai, 2017). Third, numerous simulation studies have demonstrated that bifactor models invariably show preferential fit compared with higher-order and correlated factors models even when the population ("true") model does not follow a bifactor structure (Greene et al, in press;Maydeu-Olivares & Coffman, 2006;Morgan et al, 2015;Murray & Johnson, 2013).…”