Investigating the Effect of Item Position on Person and Item Parameters: PISA 2015 Turkey Sample

Abstract

Different positions of items in booklets affect the probabilities of correct answers. This effect is called the item position effect in the literature, which causes variances in the item and person parameters. The aim of this study is to investigate the item position effect within the framework of explanatory item response theory. The analyses of this research were carried out on the PISA 2015 Turkey sample, and the item position effect was examined in the domains of reading and mathematics. In addition, the effect of the item position in different item formats (open response and multiple choice) was investigated. According to the results, the item position effect decreased the probability of answering the item correctly, and this effect was higher in reading than in mathematics. Furthermore, in the domain of mathematics, open response items were affected more than multiple-choice items by the item position. In the reading domain, open response and multiple choice items were affected similarly. The results of the analysis show that there were undesirable effects of the item position, and these effects should be taken into account.

Analyzing the Interaction of Item Position Effect and Student Characteristics within Explanatory IRT Models

“…A general item position effect has been investigated in many studies in the literature (Christiansen & Janssen, 2020;Debeer & Janssen, 2013;Hahne, 2008;Meyers et al,2009;Nagy et al, 2018;Weirich et al, 2017). Demirkol & Kelecioğlu (2022) found that there is a general item position effect in the PISA 2015 reading and mathematics data, and the probability of correct answers decreases when the items are located in later positions. But it is precious, especially for test developers and practitioners, to examine items that are more sensitive to item position effect (Albano, 2013, Bulut et al, 2017.…”

“…The item position effect differed among students, that is, the students were not exposed to the same level of item position effect (Christiansen & Janssen, 2020;Deeber & Janssen, 2013;Demirkol & Kelecioğlu, 2022). These results raised the question of what individual characteristics might be related to the item position effect, and the relationship between different student characteristics and the item position effect was investigated (Smouse & Munz, 1968;Nagy et al, 2018;Qian, 2014;Weirich et al, 2017;Wu et al, 2019).…”

Exploring the Multiverse of Analytical Decisions in Scaling Educational Large-Scale Assessment Data: A Specification Curve Analysis for PISA 2018 Mathematics Data

“…Items administered at later test positions will typically be more difficult than if they were earlier administered in the test [ 121 , 122 , 123 ]. Moreover, position effects often differ between persons and, hence, across countries in LSA studies [ 124 , 125 , 126 , 127 , 128 ].…”

Examining Position Effects on Students’ Ability and Test-Taking Speed in the TIMSS 2019 Problem-Solving and Inquiry Tasks: A Structural Equation Modeling Approach

“…Previous studies on position effects in large-scale assessments have mainly focused on its impact on item parameters, such as item difficulty, to address the concern of fairness (e.g., [5][6][7][8][9]). Several more recent studies have also examined how position effects could vary in different subject domains (e.g., [10,11]), for different item types (e.g., [11,12]), or given different student characteristics such as ability levels (e.g., [11,13]) or gender (e.g., [14]). Other studies have explored the relationship between position effect and test-taking effort (e.g., [15,16]) or the relationship between ability and speed, including potential applications of response time in measuring or predicting achievement [17][18][19][20].…”