Imperfect models, imperfect conclusions: An exploratory study of multiple-choice tests and historical knowledge

Reich, Gabriel A.

doi:10.1016/j.jssr.2012.12.004

Cited by 15 publications

(7 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the contrary, the findings indicate that the results from MC items might be misleading; students use other skills than intended when answering the items. Furthermore, since using general knowledge and test-wiseness was one of the main strategies for providing correct answers, MC items are likely to heavily overestimate student knowledge in science (Reich, 2013). Students using other skills than intended could also be true for CR items.…”

Section: Discussionmentioning

confidence: 99%

Complement or Contamination: A Study of the Validity of Multiple-Choice Items when Assessing Reasoning Skills in Physics

2017

View full text Add to dashboard Cite

The purpose of this study is to investigate the validity of using multiple-choice (MC) items as a complement to constructed-response (CR) items when making decisions about student performance on reasoning tasks. CR items from a national test in physics have been reformulated into MC items and students' reasoning skills have been analyzed in two substudies. In the first study, 12 students answered the MC items and were asked to explain their answers orally. In the second study, 102 students from five randomly chosen schools answered the same items. Their answers were scored, and the frequency of correct answers was calculated for each of the items. The scores were then compared to a sample of student performance on the original CR items from the national test. Findings suggest that results from MC items might be misleading when making decisions about student performance on reasoning tasks, since students use other skills when answering the items than is intended. Results from MC items may also contribute to an overestimation of students' knowledge in science.

show abstract

Section: Discussionmentioning

confidence: 99%

Complement or Contamination: A Study of the Validity of Multiple-Choice Items when Assessing Reasoning Skills in Physics

2017

View full text Add to dashboard Cite

show abstract

“…Association, & National Council on Measurement in Education, 1999Education, , 2014Messick, 1995 (Reich, 2009(Reich, , 2013Smith, 2017Smith, , 2018박진동, 2011;김수미, 2014…”

Section: 주목하였다(American Educational Research Association American Psmentioning

confidence: 99%

Analysis of Students’ Response Processes in Proficiency Test of Korean History Items

Lee¹,

Hong²

2019

J. Curric. Eval.

View full text Add to dashboard Cite

“…Since the experience, knowledge and skill are not gifted to be good item developers, hands-on training for them is necessary (Chen, 2011;Downing, 2009). Hence, no matter how good are the people in the testing field, the quality of test items can still be questioned in terms of validity and reliability of the test items (Reich, 2013).…”

Section: Introductionmentioning

confidence: 99%

The Use of Rasch Measurement Model in English Testing

Mokshein

2019

View full text Add to dashboard Cite

This study aimed at determining the quality of the English Paper 1 (EP1) items of UPSR trial examination for six graders in terms of its reliability, validity and items characteristics. It also sought to determine the difficulty levels of 40 multiple-choice items consisting five constructs of vocabulary, language and social expression, grammar, cloze-comprehension and reading comprehension. A number of 525 primary schools students were randomly selected from 3876 students in Kuala Selangor, Malaysia. Using the Rasch measurement model, the validity evidences were shown through the results of Principle Component Analysis (PCA), fit statistics and item distractor analysis. The results from PCA analysis showed the absence of second dimension in the test, which met the assumption of modern testing theory. Fit statistics analyses have identified seven misfit items that are beyond the acceptable range (0.7 - 1.3 logit). Item distractor analysis has identified five problematic items whereby three of them are also misfit items. Summary statistics shows that the reliability indices of Cronbach’s Alpha were greater than 0.80 and separation indices were greater than 2. This study would benefit teachers in improving existing assessment practice by spreading out the importance of item analysis in schools, particularly in language testing.

show abstract

Imperfect models, imperfect conclusions: An exploratory study of multiple-choice tests and historical knowledge

Cited by 15 publications

References 39 publications

Complement or Contamination: A Study of the Validity of Multiple-Choice Items when Assessing Reasoning Skills in Physics

Complement or Contamination: A Study of the Validity of Multiple-Choice Items when Assessing Reasoning Skills in Physics

Analysis of Students’ Response Processes in Proficiency Test of Korean History Items

The Use of Rasch Measurement Model in English Testing

Contact Info

Product

Resources

About