Critical Values for Yen’s Q₃: Identification of Local Dependence in the Rasch Model Using Residual Correlations

Abstract

The assumption of local independence is central to all IRT models. Violations can lead to inflated estimates of reliability and problems with construct validity. For the most widely used fit statistic Q3 there are currently no well-documented suggestions of the critical values which should be used to indicate local dependence, and for this reason a variety of arbitrary rules of thumb are used. In this study, we used an empirical data example and Monte Carlo simulation to investigate the different factors that can influence the null distribution of residual correlations, with the objective of proposing guidelines that researchers and practitioners can follow when making decisions about local dependence during scale development and validation. We propose that a parametric bootstrapping procedure should be implemented in each separate situation in order to obtain the critical value of local dependence applicable to the data set, and provide example critical values for a number of data structure situations. The results show that for the Q3 fit statistic no single critical value is appropriate for all situations, as the percentiles in the empirical null distribution are influenced by the number of items, the sample size, and the number of response categories.Furthermore, our results show that local dependence should be considered relative to the average observed residual correlation, rather than to a uniform value, as this results in more stable percentiles for the null distribution of an adjusted fit statistic.

Assessing vulnerability to risk of suicide and self-harm in prisoners: a Rasch analysis of the suicide concerns for offenders in the prison environment (SCOPE-2)

“…Fit residuals (z-standardised) are expected to fall within the range − 2.5 to + 2.5. A residual correlation (Q3) value of 0.2 above the average correlation was used to indicate response dependency [31]. A series of t-tests were used to assess unidimensionality, with an indication of multidimensionality (or non-unidimensionality) being apparent where the lower bound 95% CI percentage of significantly different t-tests is greater than 5% [18,32].…”

Rasch analysis of the Frost Multidimensional Perfectionism Scale

“…All items are required to be locally independent, which can be tested by examining the residual correlations between items. A residual correlation with a magnitude more than 0.20 compared to the mean of all residual correlations is a sign of local dependency (Christensen, Makransky, & Horton, ; Marais & Andrich, ). The items which are locally dependent are combined into a subtest (Wainer & Kiely, ), and the overall model fit is re‐tested.…”

The WHOQOL-BREF: a modern psychometric evaluation of its internal construct validity in people with multiple sclerosis

“…The residual of an item is the difference between the estimate of item difficulty given the model, and the observed item difficulty, standardised to a mean of zero and standard deviation of 1. Residual correlations of + 0.2 above the average correlation are considered an indicator of a breach of the local independence criteria [19] . That is, there is something else, other than the trait being measured (e.g.…”

“…Data from the WHOQOL-BREF were fit to the Rasch model in a series of analyses representing different grouping of items shown in Table 1, ideal values for fit statistics derived from Rasch analysis literature are provided within the table [19,32]. The requirement for local independence of items was found to be breached for all groups.…”