Using Item Response Theory To Assess Changes in Student Performance Based on Changes in Question Wording

Schurmeier, Kimberly D.; Atwood, Charles H.; Shepler, Carrie G.; Lautenschlager, Gary J.

doi:10.1021/ed100422c

Cited by 31 publications

(18 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As for the distractors, Answer Choice 1 (only gravity acts) is attractive to students at the very lowest end of the total score range (<2). Answer Choice 3 (gravity and a force in the direction of motion act) preferentially attracts students at the low end of the range as well (2)(3)(4)(5)(6)(7)(8)(9)(10). The IRC for Answer Choice 5 (gravity, a force in the direction of motion, and a force pointing from the center of the circle outward) shows a broad peak in the range of total scores of 9-16, while the IRC for Answer Choice 4 (gravity, a force exerted by the channel toward the center of the circle, and a force in the direction of motion) shows a broad peak in the total score range of 14-23.…”

Section: Additional Fci Insights From Irc Analysismentioning

confidence: 99%

An item response curves analysis of the Force Concept Inventory

Morris

Harshman

Branum‐Martin

et al. 2012

American Journal of Physics

View full text Add to dashboard Cite

Several years ago, we introduced the idea of item response curves (IRC), a simplistic form of item response theory (IRT), to the physics education research community as a way to examine item performance on diagnostic instruments such as the Force Concept Inventory (FCI). We noted that a full-blown analysis using IRT would be a next logical step, which several authors have since taken. In this paper, we show that our simple approach not only yields similar conclusions in the analysis of the performance of items on the FCI to the more sophisticated and complex IRT analyses but also permits additional insights by characterizing both the correct and incorrect answer choices. Our IRC approach can be applied to a variety of multiple-choice assessments but, as applied to a carefully designed instrument such as the FCI, allows us to probe student understanding as a function of ability level through an examination of each answer choice. We imagine that physics teachers could use IRC analysis to identify prominent misconceptions and tailor their instruction to combat those misconceptions, fulfilling the FCI authors’ original intentions for its use. Furthermore, the IRC analysis can assist test designers to improve their assessments by identifying nonfunctioning distractors that can be replaced with distractors attractive to students at various ability levels.

show abstract

Section: Additional Fci Insights From Irc Analysismentioning

confidence: 99%

An item response curves analysis of the Force Concept Inventory

Morris

Harshman

Branum‐Martin

et al. 2012

American Journal of Physics

View full text Add to dashboard Cite

show abstract

“…[21] It has been shown that specific phrasing of questions and options is an important aspect. [22][23][24] Questions seemingly unambiguous to experts were understood differently, but still in a reasonable manner, by novices. [22] On the other hand, it is an (often implicit) objective of many courses that students are able to handle the terminology, models, and underlying assumptions.…”

Section: The Challenge Of Multiple-choice Question Developmentmentioning

confidence: 97%

Questions for Classroom Response Systems and Teaching Instrumental Element Analysis

Schwarz¹

2021

Chimia

View full text Add to dashboard Cite

Asking students questions is a central, although understudied and underappreciated, ingredient of teaching. Formative questioning provides many opportunities for teachers and students, e.g. to practice skills and receive feedback. Among other approaches, classroom response systems (CRSs), which run on the mobile electronic devices of students, facilitate such active engagement of students in the lecture hall. This paper presents an overview on questions for teaching with a focus on questions for CRSs and provides considerations and brief guidelines for the development of multiple-choice questions. Examples from a mid-sized analytical chemistry lecture illustrate additional challenges and different probes for potential misconceptions. Moreover, limitations of valid interpretation of students' responses are emphasized. This leads to a discussion of the value of incorporating prompts for justifications into questions.

show abstract

“…Instead, the responses may be a parroting back of the stem of the question. In fact, studies in the sciences have shown that even small alterations in context and question wording influence the content of student responses to constructed-response questions (Nehm, & Ha, 2011;Schurmeier, Atwood, Shepler, & Lautenschlager, 2010;Weston, Haudek, Prevost, Merrill, & Urban-Lurain, 2015).…”

Section: Constructed Response Itemsmentioning

confidence: 99%

Effects of Question Stem on Student Descriptions of Histograms

Kaplan¹,

Lyford

Jennings³

2018

SERJ

View full text Add to dashboard Cite

Assessment is necessary, but difficult, in statistics education. Multiple choice assessments are common, particularly for research purposes. Open-ended assessments may be more adept at revealing student understanding, but ensuring their validity can be difficult. The study presented here examines differences in student descriptions of histograms for different question prompts and scenarios in order to understand how best to ask such questions in research and teaching situations. The results show that different ways of asking students to describe histograms and different scenarios and shapes of histograms lead to systematic differences in student descriptions of histograms. The paper concludes with suggestions for phrasing questions, both for research and classroom assessment, and provides directions for future research based on our results. First published May 2018 at Statistics Education Research Journal Archives

show abstract

Using Item Response Theory To Assess Changes in Student Performance Based on Changes in Question Wording

Cited by 31 publications

References 15 publications

An item response curves analysis of the Force Concept Inventory

An item response curves analysis of the Force Concept Inventory

Questions for Classroom Response Systems and Teaching Instrumental Element Analysis

Effects of Question Stem on Student Descriptions of Histograms

Contact Info

Product

Resources

About