A general framework and an R package for the detection of dichotomous differential item functioning

Magis, David; Béland, Sébastien; Tuerlinckx, Francis; Boeck, Paul De

doi:10.3758/brm.42.3.847

Cited by 237 publications

(232 citation statements)

References 54 publications

Supporting

Mentioning

207

Contrasting

Unclassified

Order By: Relevance

“…IRT calculations were performed using the R package "ltm" [70], and DIF calculations used the R package "difR" [71]. [20] and superior male performance on standardized quantitative instruments [23,25], this provides evidence that there was not a substantial disparity between male and female academic ability in Sample 1.…”

Section: F Bonferroni Correctionmentioning

confidence: 99%

Gender fairness within the Force Concept Inventory

Traxler

Henderson

Stewart

et al. 2018

Phys. Rev. Phys. Educ. Res.

134

View full text Add to dashboard Cite

Research on the test structure of the Force Concept Inventory (FCI) has largely ignored gender, and research on FCI gender effects (often reported as "gender gaps") has seldom interrogated the structure of the test. These rarely crossed streams of research leave open the possibility that the FCI may not be structurally valid across genders, particularly since many reported results come from calculus-based courses where 75% or more of the students are men. We examine the FCI considering both psychometrics and gender disaggregation (while acknowledging this as a binary simplification), and find several problematic questions whose removal decreases the apparent gender gap. We analyze three samples (total N pre ¼ 5391, N post ¼ 5769) looking for gender asymmetries using classical test theory, item response theory, and differential item functioning. The combination of these methods highlights six items that appear substantially unfair to women and two items biased in favor of women. No single physical concept or prior experience unifies these questions, but they are broadly consistent with problematic items identified in previous research. Removing all significantly gender-unfair items halves the gender gap in the main sample in this study. We recommend that instructors using the FCI report the reduced-instrument score as well as the 30-item score, and that credit or other benefits to students not be assigned using the biased items.

show abstract

Section: F Bonferroni Correctionmentioning

confidence: 99%

Gender fairness within the Force Concept Inventory

Traxler

Henderson

Stewart

et al. 2018

Phys. Rev. Phys. Educ. Res.

134

View full text Add to dashboard Cite

show abstract

“…The MantelHaenszel odds ratio (α MH ), follows a chisquare distribution with one degree of freedom under the null hypothesis of no association between item responses and group membership, conditionally upon the total score level. If the value of this indicator exceeds the threshold value for a chosen level of significance, then the item will be identified as displaying DIF (Magis, Beland, Tuerlinckx, & De Boeck, 2010 …”

Section: Mantel-haenszel (Mh) Methodsmentioning

confidence: 99%

“…They are robust, parametric methods, but assumptions and difficult to meet. DIF analyses can be conducted using IBM SPSS Statistics for instance (Zumbo, 1999), or software packages designed especially for DIF analyses like: "difR" (Magis, Beland, Tuerlinckx, & De Boeck, 2010), lordif" (Choi, Gibbons, and Crane, 2011), "deltaPlotR" (Magis & Facon, 2014). …”

Section: Dif Methods and Techniquesmentioning

confidence: 99%

See 1 more Smart Citation

Untitled

2017

PRU

View full text Add to dashboard Cite

A psychological test should be developed to accurately estimate the measured construct, being insensitive to extraneous factors that could affect it. The way in which the various external factors jeopardize the item responses is known as test bias and minimizing these influences is a primary concern for test developers. A relatively recent category of psychometric techniques that aim to identify those items that function differently in a group of test-takers is called Differential Item Functioning (DIF). The purpose of this paper is to present, in a non-technical manner, the conceptual foundation of DIF, and the most common applied techniques. We will discuss methods based on logistic regression, those using Item Response Theory models as well as the Mantel-Haenszel method. We will then discuss the strengths and limitations of each method, conditions of application, and the main software packages used to conduct such analyses.

show abstract

“…In the past decades many procedures for identifying items with differential functioning have been envisioned and several of these methods have been implemented in R (R Core Team 2016): The package difR (Magis, Beland, Tuerlinckx, and De Boeck 2010) includes eleven different indices for dichotomous items; the package lordif (Choi, Gibbons, and Crane 2011) implements a modified logistic regression algorithm that allows for both dichotomous and polytomous items using estimated sum scores or item response theory (IRT) abilities as the matching criterion; packages like mirt (Chalmers 2012), ltm (Rizopoulos 2006), lme4 (Bates, Mächler, Bolker, and Walker 2015), and eRm (Mair, Hatzinger, and Maier 2016;Mair and Hatzinger 2007) allow to test DIF using the likelihood ratio test through comparisons of multiple models; while packages like psychomix (Frick, Strobl, Leisch, and Zeileis 2012;Frick, Strobl, and Zeileis 2015), mRm (Preinerstorfer 2016), and psychotree (Komboz, Strobl, and Zeileis 2017;Strobl, Kopf, and Zeileis 2015), enable modeling and testing for test invariance using mixture Rasch models or through trees. A recent IRT parametric framework for detecting DIF that allows for detection for both dichotomous and polytomous items with unidimensional or multidimensional IRT models, known as the differential functioning of items and tests (DFIT) framework (Raju, Van der Linden, and Fleer 1995), was yet to be implemented.…”

Section: Introductionmentioning

confidence: 99%

DFIT: An R Package for Raju's Differential Functioning of Items and Tests Framework

Cervantes¹

2017

J. Stat. Soft.

View full text Add to dashboard Cite

This paper presents DFIT, an R package that implements the differential functioning of items and tests framework as well as the Monte Carlo item parameter replication approach for producing cut-off points for differential item functioning indices. Furthermore, it illustrates how to use the package to calculate power for the NCDIF index, both post hoc, as has regularly been the case in differential item functioning empirical and simulation studies, as well as a priori given certain item parameters. The version reviewed here implements all DFIT indices and Raju's area measures for tests comprised of items modeled with the same parametric item response unidimensional model (1-, 2-, and 3-parameters, generalized partial credit model or graded response model), the Mantel-Haenszel statistic with an underlying dichotomous item response model, and the item parameter replication method for any of the estimated indices with dichotomous item response models.

show abstract

A general framework and an R package for the detection of dichotomous differential item functioning

Cited by 237 publications

References 54 publications

Gender fairness within the Force Concept Inventory

Gender fairness within the Force Concept Inventory

Untitled

DFIT: An R Package for Raju's Differential Functioning of Items and Tests Framework

Contact Info

Product

Resources

About