The Position of Distractors in Multiple-Choice Test Items: The Strongest Precede the Weakest

Abstract

Middle bias has been reported for responses to multiple-choice test items used in educational assessment. It has been claimed that this response bias probably occurs because test developers tend to place correct responses among middle options, tests thus presenting a middle-biased distribution of answer keys. However, this response bias could be driven by strong distractors being more frequently located among middle options. In this study, the frequency of responses to a Chilean national examination used to rank students wanting to access higher education was used to categorize distractors based on attractiveness level. The distribution of different distractor types (best distractor, non-functioning distractors…) was analyzed across 110 tests of 80 five-option items administered to assess several disciplines in five consecutive years. Results showed that the strongest distractors were more frequently found among middle options, most commonly at option C. In contrast, the weakest distractors were more frequently found at the last option (E). This pattern did not substantially vary across disciplines or years. Supplementary analyses revealed that a similar position bias for distractors could be observed in tests administered in countries other than Chile. Thus, the location of different types of distractors might provide an alternative explanation for the middle bias reported in literature for tests’ responses. Implications for test developers, test takers, and researchers in the field are discussed.

How is The Implementation of Diagnostic Tests in Biology Learning in South Sumatra?

“…Making questions in the form of short answer is easier to do when compared to multiple choice questions (McDermott et al, 2014). The development of short-form questions does not require the creation of distracting answers as in multiple-choice questions (Lions et al, 2021). In addition, the short form questions also make it easier for teachers to correct student test results.…”

Los Efectos Posicionales en Ítems de Opción Múltiple: el Orden de los Distractores Afecta a la Dificultad

“…Esta distribución propia del "efecto anclaje" se ha encontrado recientemente en el estudio de Lions et al (2021), en el que los autores concluyen que los distractores se ordenan de mayor a menor plausibilidad. Una hipótesis posible en este caso sería que los examinados descartasen por defecto las últimas opciones, ya que se darían cuenta de que son generalmente menos atractivas, aumentando su probabilidad de acierto cuando el DME se encuentra al final (Lions et al, 2021).…”

“…Los distractores tienden a generarse en orden de plausibilidad porque son los más fácilmente accesibles, fenómeno que Attali y Bar-Hillel (2003) denominaron "efecto anclaje"(anchoring effect). Lions et al (2021) postulan que podría ser la posición de los distractores más plausibles, y no de la OC, lo que produzca el sesgo central de respuesta, de manera que la posición de los distractores pudiera ser un factor contaminante, que no se suele tener en cuenta en la investigación sobre este tema.…”

“…En relación con los distractores menos plausibles, Kiat et al (2018) encuentran que la posición de estos no influía en la dificultad del ítem en su test. Por otro lado, Lions et al (2021) postulan que al colocarse estas opciones al final pueda ocurrir que los estudiantes las descarten rápidamente, aumentando su probabilidad de acertar.…”

Position of Correct Option and Distractors Impacts Responses to Multiple-Choice Items: Evidence From a National Test

“…The main principle behind these suggestions is that the position of item options should never lead examinees to detect the correct response. The fact that not all test developers adhere to this principle and that position biases are observable in the distribution of answer keys and strong distractors is an issue of concern (Attali & Bar-Hillel, 2003;Lions et al, 2021Lions et al, , 2022Mentzer, 1982;Metfessel & Sax, 1958).…”