A comparison of three sets of criteria for determining the presence of differential item functioning using ordinal logistic regression

Abstract

Specific criteria chosen to determine whether items have DIF have an impact on the findings. Criteria based entirely on statistical significance may detect small differences that are clinically negligible.

Computerized adaptive test for patients with foot or ankle impairments produced valid and responsive measures of function

“…An item demonstrates DIF if patients having the same ability but coming from different groups do not have the same probability of selecting a given item response [45,46]. Each item was assessed for DIF by using DIFdetect software [47,48], which combines IRT item calibration estimated by the Graded Response Model [49] by using Parscale software [50] with multiple ordinal logistic regression models for each item and independent variable category by using Stata software [51].…”

Disease‐related differential item functioning in the work instability scale for rheumatoid arthritis: Converging results from three methods

“…In this study, both tests of significance and magnitude measures (where available) were used in combination to identify both types of DIF, in accordance with previous recommendations (24,25). Some debate exists regarding the optimal statistical criteria to confirm DIF by the various methods (26,27); therefore, to minimize potential Type I errors, we have applied conservative statistical thresholds and also examined relative magnitudes or "degrees" of DIF among scale items. (10) is a widely used approach that identifies uniform DIF based on analysis of 3-way (2 ϫ 2 ϫ ) contingency tables via crosstabulation of item response (column) by group (row) for every level of the "conditioning variable" (i.e., level of measured trait), where represents the number of possible scores for the measure (28).…”

Differential item functioning was negligible in an adaptive test of functional status for patients with knee impairments who spoke English or Hebrew

“…Each item was assessed for DIF using difwithpar [42][43][44][45][46][47], which combines IRT item calibration estimated by the GRM [34] using Parscale software [48] with multiple ordinal logistic regression models for each item and covariate using Stata software (v. 9.2, College Station, TX) [49].…”

“…For non-uniform DIF, we used Bonferroni adjustment for a (e.g., 0.05/ 18 = 0.0028) based on the number of items in the scale. Optimal criteria for selecting statistical cut-points for detecting either uniform or non-uniform DIF are unknown [44,45]. We based cut-points on previous experience [44-46, 50, 51], but acknowledge large data sets facilitate identification of items with statistically significant DIF [45] that may be of negligible clinical importance [45], so further study related to cut-point selection is recommended.…”

“…Optimal criteria for selecting statistical cut-points for detecting either uniform or non-uniform DIF are unknown [44,45]. We based cut-points on previous experience [44-46, 50, 51], but acknowledge large data sets facilitate identification of items with statistically significant DIF [45] that may be of negligible clinical importance [45], so further study related to cut-point selection is recommended. The process is sequential, i.e., starts with one covariate and progresses to subsequent covariates, and iterative, i.e., decisions are made at each step during the DIF detection process.…”