Purification procedures used for the detection of gender DIF: Item bias in a foreign language test

Abstract

Differential item functioning (DIF) detection was handled based on “Mantel-Haenszel (MH)”, “Simultaneous item bias test (SIBTEST)”, “Lord's chi-square”, “Raju's area” methods when item purification was performed or item purification was not performed using real data in current study. After detecting gender-related DIF, expert opinions were taken for bias study. It is important to conduct the gender bias research in the English test when purification is performed and when purification is not performed, as there were DIF studies, but there were not completely similar bias studies in the literature. The sample of the research consists of 7389 students who took the “Transition from Primary to Secondary Education Exam (TPSEE, referred to as “TEOG” in Turkey)” administered in April 2017. When gender-related DIF analysis was performed with the four methods, the results were found to differ partially. DIF analysis results differed in the different conditions item purification was performed or not. Detection of DIF was indicative of possible bias. In the second stage of the study, the opinions of seven experts were taken for item 11, for which DIF was detected at least at B level based on MH, SIBTEST. As a result of expert opinion, it was found that there was no item bias according to gender in any item in the English test. It is recommended that similar bias studies can be conducted for test developers to be aware of the features that may lead to item bias and to construct unbiased items.