An Investigation of Ordering Test Items Differently Depending on Their Difficulty Level by Differential Item Functioning

Abstract

Position effects may influence examinees' test performances in several ways and trigger other psychometric issues, such as Differential Item Functioning (DIF) .This study aims to supply test forms in which items in the test are ordered differently, depending on their difficulty level (from easy to difficult or difficult to easy), to determine whether the items in the test form result in DIF and whether a consistency exists between the methods for detecting DIF. Research Methods: Methods of Mantel Haenszel (MH) and Logistic Regression (LR) have been taken into consideration to identify whether the items in the tests involve DIF.

An Investigation of Item Position Effects by Means of IRT-Based Differential Item Functioning Methods

“…There are also studies reporting that ordering items in a test from easy to difficult has an impact on the probability of giving correct responses to the items (Balta & Omur Sunbul, 2017;Çokluk et al, 2016). In the present study, some items flagged as DIF were evaluated within this scope.…”

“…There are numerous studies on IP effects on psychometric item characteristics in the related literature (Hambleton, 1968;Hambleton & Traub, 1974;Kelinke, 1980;Klosner & Gellman, 1973;Leary & Dorans, 1985;Lee, 2007;Newman et al, 1988;Perlini et al, 1998). However, there are fewer studies on whether using different forms or booklets in achievement exams leads to certain psychometric problems such as DIF, and in the majority of these studies, while some focus on item order effects by ordering items from easy to difficult, difficult to easy, or randomly based on item difficulty index (Balta & Omur Sunbul, 2017;Çokluk et al, 2016;Freedle & Kostin, 1991;Plake et al, 1988;Ryan & Chiu, 2001), others focus on IP effects (Avcu et al, 2018;Bulut, 2015;Erdem, 2015). Ryan and Chiu (2001) developed two forms consisting of 40-items which included topics they had addressed, namely algebra, trigonometry, geometry, and analytic geometry.…”

“…Hence, examining the positioning of the CONTACT: Sümeyra Soysal  sumeyrasoysal@hotmail.com  Necmettin Erbakan University, Faculty of Education, Department of Educational Sciences, 42090, Konya, Turkey ISSN-e: 2148-7456 /© IJATE 2021 same items in various ways across different booklets should be examined and investigated to see whether or not one book type is more advantageous for some groups of test takers which is important for the test development process. The positions of items in booklets or test forms created by item position manipulations may lead to differential item functioning (DIF) (Akayleh, 2018;Balta & Omur Sunbul, 2017;Debeer & Janssen, 2013;Erdem, 2015). The present examines whether item position effects lead to DIF in test items or not.…”

Değişen Madde Fonksiyonunun Belirlenmesinde R Kullanımı: Ortaöğretime Geçiş Sınavı Fen Alt Testi

“…In the literature, the effect of item order appears to be source for DIF (Avcu et. al., 2018;Balta and Ömür-Sünbül, 2017;Chiu, 2012;Chiu and Irwin, 2011;Ryan and Chiu, 2001). However, there is no research that examines DIF for item orders on large scale tests held in Turkey.…”

“…DIF is examined in the literature for many variables (gender, culture, school type, socio-economic level). Item position effect is one of these variables (Avcu, Tunç and Uluman, 2018;Balta and Ömür-Sünbül, 2017;Chiu, 2012;Chiu and Irwin, 2011;Ryan and Chiu, 2001). In order to increase reliability and prevent cheating, especially in achievement tests, different forms (booklet types) are prepared with different item order.…”