Reliability Estimation in Multidimensional Scales: Comparing the Bias of Six Estimators in Measures With a Bifactor Structure

Abstract

In the context of multidimensional structures, with the presence of a common factor and multiple specific or group factors, estimates of reliability require specific estimators. The use of classical procedures such as the alpha coefficient or omega total that ignore structural complexity are not appropriate, since they can lead to strongly biased estimates. Through a simulation study, the bias of six estimators of reliability in multidimensional measures was evaluated and compared. The study is complemented by an empirical illustration that exemplifies the procedure. Results showed that the estimators with the lowest bias in the estimation of the total reliability parameter are omega total, the two versions of greatest lower bound (GLB) and the alpha coefficient, which in turn are also those that produce the highest overestimation of the reliability of the general factor. Nevertheless, the most appropriate estimators, in that they produce less biased estimates of the reliability parameter of the general factor, are omega limit and omega hierarchical.

Assessment of Oral Skills in Adolescents

“…Figure 1 shows that the factor weights were acceptable and the correlation between the factors was high, indicating that the instrument presented the property of essential unidimensionality, thus enabling the overall score, rather than only the specific factors, to be used as a measure of oral competence [ 79 , 80 ]. To test for essential unidimensionality, a bifactor model was applied, which showed an excellent goodness of fit: χ 2 (187) = 450.082, p < 0.001, CFI = 0.991, TLI = 0.989, RMSEA = 0.042 (0.037, 0.047) and SRMS = 0.042.…”

“…The factor structure of the instrument in the second sample showed an acceptable factor validity in that it correctly recovered the structure of five correlated factors with an acceptable goodness of fit. To determine whether the structure could be considered essentially unidimensional, alternative unidimensional and bifactor models were tested and the results indicated that the total test score could legitimately be used as a measure of self-perceived oral competence [ 79 , 80 ]. This finding is crucial in that it legitimizes the use of the overall TSOC score, rather than having to make multidimensional use of the instrument by subscale or dimension.…”

The Russian Aphasia Test: The first comprehensive, quantitative, standardized, and computerized aphasia language battery in Russian

“…Next, more work is required to characterize the internal structure of the RAT. A much larger sample of PWA is needed to properly identify the test’s underlying factors and evaluate the internal structure of the subtests using specific estimators [ 47 , 51 ].…”

Measuring the Organizational Change Maturity of Chilean Companies

“…The correlation between each sub-index and the SCM index (Eqs 1-4 vs. 5) and the correlation index proposed by Lord and Novic (1968), Equation 4.7.4 is also used. As stated in previous literature (see Zinbarg et al, 2005;Cho, 2016;Trizano-Hermosilla et al, 2021), the multidimensional structure of a test or index requires specific reliability coefficients. When the questions that make up the index have a multidimensional structure, Cronbach's coefficient can deliver a poor lower limit for reliability.…”