Detecting Differential Item Functioning Using SIBTEST, MH, LR and IRT Methods

Çepni, Zafer; Kelecioğlu, Hülya

doi:10.21031/epod.988879

Cited by 4 publications

(5 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on the results obtained, it can be seen that there is a very significant bias in the 2013/2014 NE item items. More than half of the NE questions contain DIF, which indicates that the NE questions tend to benefit the focal group [15,38], in this case, for groups of students in NTT (representing the eastern region). Furthermore, disadvantaged students in DIY (representing the western region of Indonesia).…”

Section: Discussionmentioning

confidence: 99%

“…This condition indicates that the NE questions contain a relative bias significant by location within [11]. Racial and ethnic group differences may reflect actual differences rather than measurement bias [38] know that ethnicity may influence learning culture and learning opportunities [38,39].…”

Section: Discussionmentioning

confidence: 99%

“…In addition, the analysis carried out by researchers is still shallow to reveal the items and materials that contain DIF. In addition to geographical factors, many exciting things can be done to understand the bias of the NE items, such as differences in the primary language, learning opportunities, socioeconomics, other factors that are more interesting to study [3,21,38].…”

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Bias Geographic Location of Math National Examination in Junior High School: Analysis of Differential Item Functioning (DIF)

Elvira¹,

Kartowagiran²,

Retnawati³

et al. 2023

ISI

View full text Add to dashboard Cite

Diversity is a hot issue discussed in the world of education. With its diversity, Indonesia has great potential to study such things as the geographical diversity equalization factors that affect the quality of education. Implementation of national examinations (NE) as a benchmark and standards from primary to secondary education have a different condition for each location, such as Daerah Istimewa Yogyakarta (DIY) representing the West and Nusa Tenggara Timur (NTT) representing the western region of Indonesia. The focus of this research is to find out how much difference the ability of junior high school students in Indonesia in terms of geographical location. This study uses five DIF detection methods for mathematics NE 2013/2014 school year to analyze students' different abilities. The analysis results show that, in general, the NE questions for the 2013/2014 academic year benefit the focal group / NTT on Algebra, Geometry, and Statistics/Probability material, although with lower ability compared to group reference /DIY. With the analysis carried out, policymakers can take corrective steps to focus more on fixing problems, facilities, and resources teacher power so that problem inequality from aspect geographical there is no future again in Indonesia.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Bias Geographic Location of Math National Examination in Junior High School: Analysis of Differential Item Functioning (DIF)

Elvira¹,

Kartowagiran²,

Retnawati³

et al. 2023

ISI

View full text Add to dashboard Cite

show abstract

“…Some of these methods include Mantel-Haenszel, SIBTEST, IRT methods, standardization, chi-square, Likelihood ratio test, Logistic Regression, b parameter indices, probability differences indices, IRT Likelihood Ratio Test (LRT), general IRT-LR, log-linear models, and Lord's chi-squared test (Wiberg, 2007). Since these DIF detection methods are based on different statistical bases, detecting DIF using different methods may lead to different results (Çepni & Kelecioğlu, 2021;Bakan Kalaycıoğlu, 2022). Regardless of which DIF detection method is used, there are two groups in DIF analyses, the "focal group and the reference group", and different functionalization between these groups is considered.…”

Section: Differential Item Functioning and Item Purificationmentioning

confidence: 99%

Purification procedures used for the detection of gender DIF: Item bias in a foreign language test

BÜYÜKKIDIK

2023

International Journal of Assessment Tools in Education

View full text Add to dashboard Cite

Differential item functioning (DIF) detection was handled based on “Mantel-Haenszel (MH)”, “Simultaneous item bias test (SIBTEST)”, “Lord's chi-square”, “Raju's area” methods when item purification was performed or item purification was not performed using real data in current study. After detecting gender-related DIF, expert opinions were taken for bias study. It is important to conduct the gender bias research in the English test when purification is performed and when purification is not performed, as there were DIF studies, but there were not completely similar bias studies in the literature. The sample of the research consists of 7389 students who took the “Transition from Primary to Secondary Education Exam (TPSEE, referred to as “TEOG” in Turkey)” administered in April 2017. When gender-related DIF analysis was performed with the four methods, the results were found to differ partially. DIF analysis results differed in the different conditions item purification was performed or not. Detection of DIF was indicative of possible bias. In the second stage of the study, the opinions of seven experts were taken for item 11, for which DIF was detected at least at B level based on MH, SIBTEST. As a result of expert opinion, it was found that there was no item bias according to gender in any item in the English test. It is recommended that similar bias studies can be conducted for test developers to be aware of the features that may lead to item bias and to construct unbiased items.

show abstract

“…Although various gender fairness DIF studies were carried out on educational tests (Akcan & Atalay Kabasakal, 2019;Bakan Kalaycıoğlu & Berberoğlu, 2011;Çelik & Özer Özkan, 2020;Çepni & Kelecioğlu, 2021;Grover & Ercikan, 2017;Khorramdel et al, 2020;Kıbrıslıoğlu Uysal & Atalay Kabasakal, 2017) and health-related tests (Crane et al, 2004;Edelen et al, 2006;Sunderland et al, 2010) the number of studies on medical education tests was quite limited (Clauser et al, 1996a;Hope et al, 2018;Swanson et al, 2002). Clauser et al (1996a) analyzed responses of medical students to pediatrics, surgery, obstetrics-gynecology, and medicine subtests using MH and LR methods.…”

Section: Introductionmentioning

confidence: 99%

Gender-based Differential Item Functioning Analysis of the Medical Specialization Education Entrance Examination

Kalaycıoğlu

2022

Eğitimde Ve Psikolojide Ölçme Ve Değerlendirme Dergisi

View full text Add to dashboard Cite

The Medical Specialization Education Entrance Examination is a national high-stake test for the placement of medical graduates in medical specialization training in Turkey. The purpose of this study is to determine whether the Medical Specialization Education Entrance Examination items display gender-related differential item functioning (DIF) by using Mantel-Haenszel and logistic regression methods. To determine the presence of item bias, content experts reviewed items. The analyzes were conducted on the answers of 11,530 physicians to the Basic Medical Sciences and Clinical Medical Sciences tests of the 2017 Medical Specialization Education Entrance Examination spring term. According to the Mantel-Haenszel method, there were eleven out of 234 items identified as showing B level gender-related DIF. While six of the items functioned in favor of male physicians, five of them were in favor of female physicians. Since the number of items in favor of each gender is close, DIF cancellation occurs. According to content areas, one histology and embryology, one internal medicine, and three gynecology and obstetrics items were in favor of female physicians, one physiology, two medical pharmacology, one pediatrics, and two surgical items were in favor of male physicians. To the experts' reviews, there are no biased items. The medical specialty preferences of the physicians and content area of the displaying differential item functioning items overlapped.

show abstract

Detecting Differential Item Functioning Using SIBTEST, MH, LR and IRT Methods

Abstract: This paper was derived from the first author's doctoral thesis titled Detecting differential item functioning using SIBTEST, Mantel-Haenszel, logistic regression and Item Response Theory methods.

Cited by 4 publications

References 31 publications

Bias Geographic Location of Math National Examination in Junior High School: Analysis of Differential Item Functioning (DIF)

Bias Geographic Location of Math National Examination in Junior High School: Analysis of Differential Item Functioning (DIF)

Purification procedures used for the detection of gender DIF: Item bias in a foreign language test

Gender-based Differential Item Functioning Analysis of the Medical Specialization Education Entrance Examination

Contact Info

Product

Resources

About