A Comparison of the Logistic Regression and Contingency Table Methods for Simultaneous Detection of Uniform and Nonuniform DIF

Abstract

In this study, we investigate the logistic regression (LR), Mantel‐Haenszel (MH), and Breslow‐Day (BD) procedures for the simultaneous detection of both uniform and nonuniform differential item functioning (DIF). A simulation study was used to assess and compare the Type I error rate and power of a combined decision rule (CDR), which assesses DIF using a combination of the decisions made with BD and MH to those of LR. The results revealed that while the Type I error rate of CDR was consistently below the nominal alpha level, the Type I error rate of LR was high for the conditions having unequal ability distributions. In addition, the power of CDR was consistently higher than that of LR across all forms of DIF.

Difference or delay? A comparison of Bayley-III Cognition item scores of young children with and without developmental disabilities

“…We used R (R Core Team, 2016) with the difR package (Magis, Beland, & Raiche, 2015). The CDR is based on a combination of the Mantel-Haenszel (MH) and Breslow-Day (BD) procedures, as MH is powerful for detecting uniform DIF and BD for detecting nonuniform DIF (Güler & Penfield, 2009). We used item purification of at least 100 iterations.…”

A Comparison of Mantel-Haenszel and Standardization Methods: Detecting Differential Item Functioning

“…In addition to the results of these studies, the results of the Guler & Penfield (2009) study also show that sample sizes 300 and 1,000 can be used for DIF detection using the Mantel-Haenszel method.…”

Checking Equity: Why Differential Item Functioning Analysis Should Be a Routine Part of Developing Conceptual Assessments

“…Closer guidance may be provided by simulation studies in which data sets are generated thousands of times. This approach allows the properties of different DIF detection methods to be compared with respect to their power and type I error rate (e.g., Swaminathan and Rogers, 1990 ; Narayanan and Swaminathan, 1996 ; Güler and Penfield, 2009 ; Kim and Oshima, 2013 ; Drabinová and Martinková, 2016 ). Studies like these have demonstrated that the Mantel-Haenszel method works particularly well for small sample sizes but, as expected from its formula, does not always detect nonuniform DIF ( Swaminathan and Rogers, 1990 ; Drabinová and Martinková, 2016 ).…”