Performances of MIMIC and Logistic Regression Procedures in Detecting DIF

Abstract

In this study, differential item functioning (DIF) detection performances of multiple indicators, multiple causes (MIMIC) and logistic regression (LR) methods for dichotomous data were investigated. Performances of these two methods were compared by calculating the Type I error rates and power for each simulation condition. Conditions covered in the study were: sample size (2000 and 4000 respondents), ability distribution of focal group [N(0, 1) and N(-0.5, 1)], and the percentage of items with DIF (10% and 20%). Ability distributions of the respondents in the reference group [N(0, 1)], ratio of focal group to reference group (1:1), test length (30 items), and variation in difficulty parameters between groups for the items that contain DIF (0.6) were the conditions that were held constant. When the two methods were compared according to their Type I error rates, it was concluded that the change in sample size was more effective for MIMIC method. On the other hand, the change in the percentage of items with DIF was more effective for LR. When the two methods were compared according to their power, the most effective variable for both methods was the sample size.

Estructura y análisis diferencial del ítem de acorde a sexo y edad del Test de Inteligencia General Matrices en estudiantes universitarios

“…Finalmente, para corroborar los resultados anteriores, se realizó el modelo de regresión logística binaria, que evidenció un tamaño de efecto DIF mínimo (de nivel A; Zumbo, 1999) en 14 reactivos (A8, A12, B3, B6, B7, B8, B9, B12, C2, C3, C4, C6, C10 y C11), que pueden interpretarse como un sesgo insignificante, determinando la falta equivalencia de todos los reactivos del Test Matrices de acorde a los covariables de edad y sexo (Friesen et al, 2019;Uğurlu & Burcu, 2020). Un aspecto por considerar es que el método MIMIC presenta como limitación la falta de detección de DIF no uniforme a diferencia del análisis de regresión logística (Gómez-Benito, Hidalgo & Padilla, 2009;Jamali, Ayatollahi & Jafari, 2016).…”

The Impact of Missing Data on the Performances of DIF Detection Methods

“…where Previous simulation studies investigating DIF with MIMIC method have shown that under most circumstances, MIMIC method performed as efficiently as or better than the other methods (SIBTEST, MH, LR etc.) with regard to type I error rate and power (e.g., Finch, 2005;Uğurlu & Atar, 2020;Woods, 2009). Missing data is a significant factor in the performances of statistical methods.…”

“…In recent years, there has been a growing amount of literature on the DIF detection with MIMIC, a CFAbased DIF detection method (Finch, 2005;Jin & Chen, 2020;Montoya & Jeon, 2020;Shih & Wang, 2009;Uğurlu & Atar, 2020;Woods, 2009). Missing data can affect any type of analysis including CFA (Harrington, 2009).…”

Gender-based Differential Item Functioning Analysis of the Medical Specialization Education Entrance Examination

“…A range of procedures has been proposed based on different theories for analyzing DIF, including Mantel-Haenzsel (MH) (Holland & Thayer, 1986), Logistic Regression (LR) (Swaminathan & Rogers, 1990), Restricted Factor Analysis (RFA) (Oort, 1992), Item Response Theory Log-Likelihood Ratio (IRT-LR) (Thissen et al, 1993), Multiple Indicator Multiple Causes (MIMIC) model (MacIntosh & Hashim, 2003;Muthen, 1988) and others (Camilli & Shepard, 1994). These procedures have been studied fairly extensively in terms of their ability to correctly identify DIF items (Finch, 2005;Gomez-Benito & Navas-Ara, 2000;Güler & Penfield, 2009;Uğurlu & Atar, 2020). Since these procedures' assumptions and approaches to modeling the data are distinct, they may identify different items as displaying DIF.…”

“…3 Finch and French (2007) and Uğurlu and Atar (2020) reported in their studies that the power of the LR method increased as the sample size increased. Hidalgo and Lopez-Pina (2004) stated in their simulation study that the LR method is more effective than the MH method when determining nonuniform DIF, and the MH method is more effective when determining uniform DIF, and the results support the study of Swaminathan and Rogers (1990).…”