Positional Response Set Among High School Students on Multiple‐choice Tests

Abstract

Investigations of positional response set in examinees of multiple-choice items have shown conflicting results. This study attempted to clarify this controversy. Over 1000 subjects were assigned at random to one of nine forms of a specially prepared vocabulary test. The difference among forms related to the position of the correct answers and the most popular distractor as determined by prior administrations of the test. The forms of the test were constructed so that if a universal positional response set existed, it would manifest itself as a difference of means of scores obtained on selected items dispersed throughout each of the different forms. The results of the analysis of variance showed no significant evidence of interindividual positional response set.The existence of a positional response set has been clearly established in studies where the test content had little meaning. Rapaport and Berg (1954) found students exhibited a significant positional response set when instructed to imagine themselves in a test situation and to respond on an answer sheet to the questions invented. Metfessal (1955), using an answer sheet with nonsense syllables as responses, also found evidence of a positional set. However when subjects have been confronted with examination tasks of a meaningful nature the findings of various researchers have been in conflict. Gustav (1963) and McNamara and Weitzman (1945) found significant positional response tendencies; contrary evidence was revealed in studies by Marcus (1963) and Hopkins and Hopkins (1964). This research was an attempt at clarifying these contradictory results. METHODThis study was designed in an attempt to (a) avoid establishing a response set due to frequent reinforcement of a particular position, (b) create a meaningful testing situation at an appropriate difficulty level and yet (c) keep the instrument uncomplicated in order to reduce the influence of other test taking "rules of thumb" procedures.Three vocabulary instruments, each composed of 80 items, were developed for three ability groupings (high, middle and low) of high school juniors.The tests were administered to a pilot group of 500 students and the 20 items having the highest point biserial correlations with total score were identified and called the 20 "critical items." On each test the 20 critical items were randomized throughout.Each test consisted of nine different forms in order to manipulate the critical items in nine different ways. The forms were constructed as follows: Form 1 had the correct answers for the 20 critical items in the first position. Form 2 had the most popular distractor for the critical items in the first position. Form 3 had the correct answers in the second position. Form 4 had the most popular distractor in the 161

The Effects of Technical and Unfamiliar Options on Guessing on Multiple‐choice Test Items

“…The ratio of choices selected as correct to the total number of possible choices was similar across positions: 48 percent for A's, 46 percent for B's, 48 percent for C's, and 49 percent for D's. These findings agree with those reported by Wilbur (1970).…”

Position of Correct Option and Distractors Impacts Responses to Multiple-Choice Items: Evidence From a National Test

“…The origin of the inconsistencies above reviewed might lie in the fact that many studies have examined CR position effects without controlling for distractors' position. Empirical research has shown that a strong distractor's position and its distance from CR can affect item accuracy (Ambu-Saidi & Khamis, 2000;Cizek, 1994;Fagley, 1987;Friel & Johnstone, 1979;Kiat et al, 2018;Shin et al, 2020; but see also Wilbur, 1970). For instance, in a study in which the most attractive distractor's (MAD) relative position and distance from CR were experimentally manipulated throughout a Malaysian pilot test in statistics responded by 232 undergraduate students, MAD was found to be 1.67 times more likely to be selected when presented before CR than after it.…”