When Are Multidimensional Data Unidimensional Enough for Structural Equation Modeling? An Evaluation of the DETECT Multidimensionality Index

Abstract

In structural equation modeling (SEM), researchers need to evaluate whether item response data, which are often multidimensional, can be modeled with a unidimensional measurement model without seriously biasing the parameter estimates. This issue is commonly addressed through testing the fit of a unidimensional model specification, a strategy previously determined to be problematic. As an alternative to the use of fit indexes, we considered the utility of a statistical tool that was expressly designed to assess the degree of departure from unidimensionality in a data set. Specifically, we evaluated the ability of the DETECT "essential unidimensionality" index to predict the bias in parameter estimates that results from misspecifying a unidimensional model when the data are multidimensional. We generated multidimensional data from bifactor structures that varied in general factor strength, number of group factors, and items per group factor; a unidimensional measurement model was then fit and parameter bias recorded. Although DETECT index values were generally predictive of parameter bias, in many cases, the degree of bias was small even though DETECT indicated significant multidimensionality. Thus we do not recommend the stand-alone use of DETECT benchmark values to either accept or reject a unidimensional measurement model. However, when DETECT was used in combination with additional indexes of general factor strength and group factor structure, parameter bias was highly predictable. Recommendations for judging the severity of potential model misspecifications in practice are provided.

Investigating diagnostic bias in autism spectrum conditions: An item response theory analysis of sex bias in the AQ‐10

“…Although unidimensional model fit well and PA‐PCA, MAP and scree plots generally supported unidimensionality, these methods do not necessarily indicate whether and to what degree parameter estimates were biased by any violations of the assumption (e.g. Bonifay, Reise, Scheines, & Meijer, ; Reise, Scheines, Widaman, & Haviland, ). Fortunately, IRT parameter estimates appear to be relatively robust to minor violations of unidimensionality (Kirisci, Hsu, & Yu, ).…”

Construct Validity of the Professional Quality of Life (ProQoL) Scale in a Sample of Child Protection Workers

“…The internal consistency of the final model was tested with omega reliability coefficients (omega, omegaS, omegaH, omegaHS; McDonald, ; Reise, Bonifay, & Haviland, ; Revelle & Zinbarg, ). Explained common variance (Reise, Scheines, Widaman, & Haviland, ) and percentage of uncontaminated correlations (Bonifay, Reise, Scheines, & Meijer, ; Reise et al., ) were also used to judge the dimensionality of the bifactor CFA model. According to Gorsuch (), the sample size ( n = 310) was adequate for performing factorial analyses given his rule of 10 participants per item.…”

Well‐being in Patients with Affective Disorders Compared to Nonclinical Participants: A Multi‐Model Evaluation of the Mental Health Continuum‐Short Form

“…">For the bifactor CFA and bifactor ESEM models, we calculated: (a) model‐based composite reliabilities for the total scale and each subscale using omega (ω) coefficients, an estimate of the proportion of observed total or subscale variance due to all sources of common variance (McDonald, ); (b) omega hierarchical (ω H ), an estimate of the proportion of observed total variance due to the GF after accounting for all specific factors; (c) omega hierarchical subscale (ω HS ), an estimate of the proportion of observed subscale variance due to its specific factor after accounting for the GF; (d) explained common variance for the general factor (ECVg), an estimate of the proportion of total common variance explained by the GF; (e) explained common variance for each subscale (ECVs), an estimate of the proportion of each subscale common variance explained by its specific factor; (f) percent of uncontaminated correlations (PUC), a measure related to the structure of the instrument, becoming larger when there are many items and many small group factors. PUC in conjunction with ECVg and ω H , hence called “general factor strength indices,” determine the parameter bias in considering a bifactor model as essentially unidimensional (Bonifay, Reise, Scheines, & Meijer, ; Reise, Scheines, Widaman, & Haviland, ; Rodriguez, Reise, & Haviland, , ). For example, when PUC > 0.8, ECVg values become less important in predicting parameter bias; if PUC < 0.8, then ECVg > 0.6 and ω H > .7 suggest that the instrument can be considered as unidimensional despite the presence of some multidimensionality (Reise et al, ). …”