An experimental study of the effects on item-analysis data of changing item placement and test time limit

Mollenkopf, William G.

doi:10.1007/bf02289044

Cited by 70 publications

(51 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These reviews described a number of studies that found changes between two or more administrations of the same test when no experimental treatment intervened between the two administrations, and even when one administration immediately followed the other. The findings in the present study, as well as in earlier ones (Gordon, 1952;Mollenkopf, 1950;Whitcomb & Travers, 1957), are even -17-more striking because they demonstrate changes within the same test, even a very short one. TIle present findings also agree with previous results that indicate that response styles may be associated with secular trends.…”

Section: Discussionsupporting

confidence: 79%

Difficulty and Other Correlates of Criticalness Response Style at the Item Level1

Stricker¹

1964

ETS Research Bulletin Series

View full text Add to dashboard Cite

This study examined the.relationship between the difficulty of items on an achievement-type test of report writing and (a) the items' correlations with the test's criticalness response style score, and (b) the items' correlations with the test's content score. Item readability and item format variables were also analyzed. The major findings were that (a) the items' difficulty was related to their correlations with the content score and unrelated to the items' correlations with the response style score; (b) the items' location in the test was generally related to their correlations with the response style score, and unrelated to the other variables; and (c) the readability indexes were not related to the three basic item variables. DIFFICULTY AND OTHER CORRELATES OF CRITICALNESS RESPONSE STYLE AT THE ITEM LEVEL 1Largely as a result of Cronbach's (1946; reviews of the early literature on response sets--or response styles--on tests, the view is widely held that tests and test items that are difficult or ambiguous are most affected by those response styles, such as acquiescence, evasiveness, and extremeness, that are linked to the response format.Despite the prevalence of this view, the results of relevant studies are not altogether convincing, and are largely limited to two response styles--acquiescence, and the position of the chosen alternative on multiple-choice tests.One major group of relevant studies examined acquiescence on tests or items of varying difficulty or ambiguity. Only one of the studies (Gage, Leavitt, & Stone, 1957) concerns the difficulty issue. In a comparison of difficult and easy general information items, the reliability of the number of true responses was .68 for the 50 difficult items, and .09 for the 40 easy ones. The two kinds of items, however, did not represent exactly the same content areas.The other studies of this kind concern various facets of item ambiguity. Two of these studies directly concern ambiguity. In one study (Bass, 1955), the original and reversed version of the California F scale items that were rated most opposite in meaning (and, hence, assumed to be clear in meaning) appeared to measure content (i.e., the correlation between the two items was relatively high and in the content direction), while those rated less opposite in meaning (and, hence, assumed to be -2-relatively ambiguous in meaning) did not, evidently because of the operation of acquiescence on the latter items. Moreover, an analysis of variance found significantly (£ < .01) greater acquiescence to the pairs of items that were least opposite in meaning. The assumption in this study that the rated level of opposition measures ambiguity is questionable, for it is possible for a pair of items that are rated as similar in meaning to be very upambiguous. In a second study (l3a.nta, 1961), the number of agree responses on attitude scales increased and the number of extreme responses decreased with the ambiguity of the scales' referents. The referents were "President Eisenhower," "College Fraternities,"...

show abstract

Section: Discussionsupporting

confidence: 79%

Difficulty and Other Correlates of Criticalness Response Style at the Item Level1

Stricker¹

1964

ETS Research Bulletin Series

View full text Add to dashboard Cite

show abstract

“…In all cases, students admitted to cheating after they were caught with their Scantron 1 not matching the examination that they claimed to have taken. Though the findings from the present study disagree with other studies that found a random arrangement of test questions resulting in lower examination scores (Mollenkopf, 1950;Hambleton and Taub, 1974;Taub and Bell, 1975;Plake et al, 1982), it is in line with others (Munz and Smouse, 1968;Marso, 1970;Huck and Bowers, 1972;Kleinke, 1980;Plake, 1980;Plake et al, 1981;Klimko, 1984). Baldwin and Howard (1983) found that ''better'' students scored significantly lower on examinations that had random sequencing of multiple-choice questions than on examinations with the originally ordered format, while ''poorer'' students appeared not to be affected.…”

Section: Discussionsupporting

confidence: 56%

A case study on multiple‐choice testing in anatomical sciences

Golda

2011

Anatomical Sciences Ed

View full text Add to dashboard Cite

Objective testing techniques, such as multiple-choice examinations, are a widely accepted method of assessment in gross anatomy. In order to deter cheating on these types of examinations, instructors often design several versions of an examination to distribute. These versions usually involve the rearrangement of questions and their corresponding answer choices. This study will determine whether the distribution of different versions of an examination affects student performance in a lower division anatomical science course. Students who receive the original version of an examination may be at an advantage over those that receive a shuffled version of an examination because of the systematic tendencies that go into examination construction. This study concludes that the shuffling of questions and answer choices to produce multiple versions of an examination does not affect student performance.

show abstract

“…Educational researchers have for years studied the effect of question order, beginning with Mollenkopf [1] and others in the 1950s. Most early studies [2] focused on changing the order of the ease of the questions: "easy to hard" and "hard to easy."…”

Section: Introductionmentioning

confidence: 99%

The Effect of Question Order on Responses to Multiple-choice Questions

Gray¹,

Rebello²,

Zollman³

2002

2002 Physics Education Research Conference Proceedings

View full text Add to dashboard Cite

Educators and researchers often make the assumption that the order of test or survey questions is unimportant. Is this assumption valid? This study investigates how the order of two related FCI questions (#13 and 14) affects students' responses. This study also investigates the effect an unrelated FCI question (#23) has on answers to the above problems. Four versions of a survey were administered before and after instruction to 243 students taking an algebra-based physics class. Versions 1 and 2 of the survey included the related physics questions in opposite order. Versions 3 and 4 included the unrelated physics question and one of the above questions. Student responses for the four versions were compared for both the pre-and post-instruction surveys. IntroductionFor years physics education researchers have used surveys and other investigative tools to learn what physics students know and how they think. These researchers often assume that the order of questions does not affect student responses. For as long as teachers have been giving tests, students have been looking for strategies and "tricks" to improve their performance on exams. These methods include "test taking strategies" often taught in high school and some college orientation classes. A common strategy includes looking for the answer or helpful hints in other problems when answering an unfamiliar question.Thus, students' performance may be influenced by their test taking skills or ability to "learn" the physics as they take the test. There is no reason to believe that students forget this skill when they are taking physics education research surveys as opposed to graded exams. This is a serious problem for researchers who are attempting to understand how students think about a series of problems or situations.Educational researchers have for years studied the effect of question order, beginning with Mollenkopf [1] and others in the 1950s. Most early studies [2] focused on changing the order of the ease of the questions: "easy to hard" and "hard to easy." With the advent of computerized adaptive testing educational researchers have begun to look more closely at the effect of changing the context and order in which test items appear. [3] The purpose of this paper is to test the aforementioned assumption that the order of questions in a test does not affect student responses to individual questions on the test. The results of our study described here suggest that the order of questions does

show abstract

An experimental study of the effects on item-analysis data of changing item placement and test time limit

Cited by 70 publications

References 3 publications

Difficulty and Other Correlates of Criticalness Response Style at the Item Level1

Difficulty and Other Correlates of Criticalness Response Style at the Item Level1

A case study on multiple‐choice testing in anatomical sciences

The Effect of Question Order on Responses to Multiple-choice Questions

Contact Info

Product

Resources

About