Differential item functioning via robust scaling

Abstract

This paper proposes a new method for assessing differential item functioning (DIF) in item response theory (IRT) models. The method does not require pre-specification of anchor items, which is its main virtue. It is developed in two main steps, first by showing how DIF can be re-formulated as a problem of outlier detection in IRT-based scaling, then tackling the latter using established methods from robust statistics. The proposal is a redescending M-estimator of IRT scaling parameters that is tuned to flag items with DIF at the desired asymptotic Type I Error rate during estimation. Theoretical results guarantee that the method performs reasonably well when fewer than one-half of the items on a test exhibit DIF. Data simulations show that the proposed method compares favorably to currently available approaches, and a real data example illustrates its application in a research context where pre-specification of anchor items is infeasible. The focus of the paper is the twoparameter logistic model in two independent groups, with extensions to other settings considered in the conclusion.

Monitoring early childhood development at the population level: The ECDI2030

“…This type of country-to-country item-level variation has been documented even with highly engineered educational surveys such as the Programme for International Student Assessment (e.g., Zwitser, Glaser, & Maris, 2017), and initial research points to similar concerns with assessments of ECD (e.g., Halpin et al, 2019;Waldman et al, 2021). Psychometric methodology for international comparisons remains an active area of research (e.g., von Davier & Bezirhan, 2022;Halpin, 2022) and can directly inform ongoing efforts in ECD assessment.…”

Linking Error in the 2PL Model

“…M-estimation theory was applied in the investigation of DIF and linking in [28][29][30]. The simultaneous treatment of standard errors and linking errors in IRT models relying on M-estimation was presented in [18,31] (see also [32]).…”

“…using a robust function that fulfills the property ρ(x)/|x| → ∞ for |x| → ∞ (see [29,30]). For example, if the median was used, ρ would be the sign function…”