A Review of Recent Developments in Differential Item Functioning

Abstract

In many practical settings, essentially the same differential item functioning (DIF) procedures have been in use since the late 1980s. Since then, examinee populations have become more heterogeneous, and tests have included more polytomously scored items. This paper summarizes and classifies new DIF methods and procedures that have appeared since the early 1990s and assesses their appropriateness for practical use. Widely used DIF methods are evaluated alongside these new methods for completeness, clarity, and comparability.

Differential Item Functioning Analysis Using Rasch Item Information Functions

“…DIF methods for analyzing dichotomous items (i.e., correct/incorrect responses) generally fall into two classes: observed score and latent score approaches (Mapuranga et al, 2008). Some of the most popular observed score methods are the standardization approach (Dorans & Kulick, 1986), the Mantel-Haenszel procedure (MH; Holland & Thayer, 1988), and the logistic regression techniques (Swaminathan & Rogers, 1990).…”

“…Despite the many DIF methods developed over the years (see Mapuranga et al, 2008), MH continues to be the most popular method for evaluating item bias and is often considered the "testing industry standard" (Roussos et al, 1999). Although MH is strictly not an IRT DIF method, Holland and Thayer (1988) showed that when a test fits the Rasch model, the total score becomes a sufficient statistic for θ and hence the use of the total test score, instead of θ , can be used for matching if data are scaled using the Rasch model.…”

Contributions to the Quantitative Assessment of Item, Test, and Score Fairness

“…Two reviews of DIF methods were conducted by ETS staff: Dorans and Potenza (1994), which was shortened and published as Potenza and Dorans (1995), and Mapuranga et al (2008), which then superseded Potenza and Dorans. In the last of these reviews, the criteria for classifying DIF methods were (a) definition of null DIF, (b) definition of the studied item score, (c) definition of the matching variable, and (d) the variable used to define groups.…”

Who's on First? Gender Differences in Performance on the SAT® Test on Critical Reading Items With Sports and Science Content

“…Perhaps procedures such as differential distractor functioning (DDF) could be implemented, as DDF examines more than just the key, taking into account DIF on the distractors as well (Green, Crone, & Folk, 1989). Other methods of DIF analysis have also been investigated and proposed: Mapuranga, Dorans, and Middleton (2008) explored a variety of methods indexed by procedural complexity, computational intensity, and cost. That said, from a practical perspective, none of these proposed methods is a perfect fix.…”