Guess Where: The Position of Correct Answers in Multiple‐Choice Test Items as a Psychometric Variable

Abstract

In this article, the authors show that test makers and test takers have a strong and systematic tendency for hiding correct answers—or, respectively, for seeking them—in middle positions. In single, isolated questions, both prefer middle positions to extreme ones in a ratio of up to 3 or 4 to 1. Because test makers routinely, deliberately, and excessively balance the answer key of operational tests, middle bias almost, though not quite, disappears in those keys. Examinees taking real tests also produce answer sequences that are more balanced than their single question tendencies but less balanced than the correct key. In a typical four‐choice test, about 55% of erroneous answers are in the two central positions. The authors show that this bias is large enough to have real psychometric consequences, as questions with middle correct answers are easier and less discriminating than questions with extreme correct answers, a fact of which some implications are explored.

Does Correct Answer Distribution Influence Student Choices When Writing Multiple Choice Examinations?

“…Having opportunities to practice retrieving information pertaining to course content when studying is an active study approach that has been suggested to promote learning and long-term retention more effectively than passive processes such as reading the textbook and course notes (Carnegie, 2015;Karpicke, Butler, & Roediger III, 2009;Orr & Foster, 2013;Roediger & Karpicke, 2006). That being said, correct answer position and the possible development of a pattern could become important when a student does not know the correct answer and has resorted to guessing (Attali & Bar-Hillel, 2003). Indeed, Zimmerman and Williams (2003) suggested that guessing can negatively impact the reliability of MCQ examination outcomes, especially if the exams are short and the number of answer choices is small (e.g., true/false questions with a 50% chance of guessing the correct answer).…”

“…Bresnock et al (1989) reported that students had better outcomes when writing four-choice MCQ exams if a higher proportion of the correct answers were represented by the letter A rather than the letter D. They postulated that students were more prone to recognize the correct answer if they saw it immediately rather than after reading through several distractors and possibly becoming confused. On the other hand, a pair of studies separated in time by almost 50 years and each controlled to remove knowledge as a basis for answer selection revealed that a higher proportion (70-80%) of study participants, chose central letters (B and C) rather than A or D, the letters at the beginning or the end of the list (Attali & Bar-Hillel, 2003;Berg & Rapaport, 1954). Experimental design (participants not provided with actual answer choice content, only answer letters, and the earlier study also used imaginary questions) assured that participants were purely guessing in order to pick their answer from lists of four options.…”

Investigating Factors That Influence Item Performance on ACS Exams

“…6 This leads examinees to often seek out correct answers in the middle of a set of choices, a tendency mirrored by some examiners by where they place correct answers. 7 Such strategies can be minimized by using key randomization instead of key balancing and number right scoring instead of formula scoring. 5 McLeod, Zhang, and Yu 8 found no inherent disadvantages to students when exam items or the order of the answer choices were randomized.…”

Number Preferences in Lotteries

“…Multiple studies have shown that people have a tendency to select choice options presented in the middle of a display and avoid the edges. This behavior has been observed with laboratory and field data, for both individual choice and strategic interaction (Christenfeld, 1995;Rubinstein, Tversky & Heller, 1997;Shaw, Bergen, Brown & Gallagher, 2000;Attali & Bar-Hillel, 2003;Raghubir & Valenzuela, 2006;Chandon, Hutchinson, Bradlow & Young, 2009;Atalay, Bodur & Rasolofoarison, 2012;Valenzuela, Raghubir & Mitakakis, 2013;Bar-Hillel, 2015). Closely related to our analyses for lottery games, Bar-Hillel and Zultan (2012) examine the distribution of gamblers' bets on a roulette table and observe that numbers in the center are more popular.…”