Detection of Differential Item Functioning with Nonlinear Regression: A Non‐IRT Approach Accounting for Guessing

Abstract

In this article we present a general approach not relying on item response theory models (non‐IRT) to detect differential item functioning (DIF) in dichotomous items with presence of guessing. The proposed nonlinear regression (NLR) procedure for DIF detection is an extension of method based on logistic regression. As a non‐IRT approach, NLR can be seen as a proxy of detection based on the three‐parameter IRT model which is a standard tool in the study field. Hence, NLR fills a logical gap in DIF detection methodology and as such is important for educational purposes. Moreover, the advantages of the NLR procedure as well as comparison to other commonly used methods are demonstrated in a simulation study. A real data analysis is offered to demonstrate practical use of the method.

Validity of the listening module of international English language testing system: multiple sources of evidence

“…DIF exists when different groups of learners have different probability of successfully answering an item (Drabinova and Martinkova 2017;Ferne and Rupp 2007;Li and Wang 2015); therefore, if the test takers have less or more the same knowledge, then they should perform similarly on test items; DIF is needed for test validity and test fairness (Fidalgo et al 2014;Hou et al 2014;Pae 2004Pae , 2012Su and Wang 2005;Zumbo 2003Zumbo , 2007.…”

“…As for the significance of CDM, George and Robitzsch (2014) recommend the use of CDM as one of the recent statistical tools for detecting DIF and plenty of psychometric questions in relation to DIF can be addressed with use of CDM (Hou et al 2014). To date, only a few studies have been conducted on DIF assessment within the framework of CDM (Drabinova and Martinkova 2017;Li and Wang 2015;Hou et al 2014;Li 2008;Zhang 2006). However an extensive body of research has been done in the area of cognitive diagnosis of students' learning (Li and Wang 2015; de la Torre 2011; de la Torre and Douglas 2004;Junker and Sijtsma 2001), no study has so far been done on detecting the DIF of IELTS LCT with use of CDMs, so that some researchers (e.g., George and Robitzsch 2014) suggest the use of CDM for DIF detection.…”

Development and Validation of the Homeostasis Concept Inventory

“…We have also included structural analyses to test the unidimensionality of the instrument. The analyses were completed in R (R Core Team, 2016; Supplemental Material, R Code) with the libraries ggplot (Wickham, 2009), lme4 (Bates et al , 2015), lmerTest (Kuznetsova et al , 2016), psychometric (Fletcher, 2010), psych (Revelle, 2015), corrplot (Wei and Simko, 2010), ltm (Rizopoulos, 2006), mirt (Chalmers, 2012), WrightMap (Torres Irribarra and Freund, 2014), difNLR (Drabinova et al , 2016), difR (Magis et al , 2015), and ShinyItemAnalysis (Martinková et al , 2017a). Students’ names and identification numbers were removed from all data sets before statistical analyses.…”

Checking Equity: Why Differential Item Functioning Analysis Should Be a Routine Part of Developing Conceptual Assessments

“…An example of a situation in which a primary and secondary latent trait are both required for a test occurred recently in a biology admission test for medical school in the Czech Republic ( Drabinová and Martinková, 2016 ; see also Štuka et al , 2012 ). On one item about childhood illnesses, DIF analysis revealed that women performed better than men.…”

“…On one item about childhood illnesses, DIF analysis revealed that women performed better than men. Content experts reviewed the item, and concluded that the difference occurred because women in the Czech Republic spend more time with children than men and therefore have more experience with childhood illnesses ( Drabinová and Martinková, 2016 ). The faculty, however, still considered the item to be fair despite the gender difference, because medical experts need to be familiar with childhood illnesses.…”