Theoretical evaluation of partial credit scoring of the multiple-choice test item

Abstract

We consider the effect of chance success due to guessing on the error of measurement in multiple-choice tests. In these types of tests, test-takers may eliminate some of the answer options to increase the expected score when guessing at the cost of increased measurement error. We consider an arbitrary multiple-choice test taken by an arbitrary test-taker that is expected to know an arbitrary fraction of its keys and distractors. For this model, we introduce a mathematically optimal scoring system that minimizes the error variance due to guessing if test-takers are rational in their strategy to optimize their expected score. The resulting system is easy to explain to test-takers and to implement when scoring.

Theoretical evaluation of partial credit scoring of the multiple-response test item

“…In a previous contribution (Persson, 2021, hereonafter designated "Paper I"), we made a theoretical comparison of the ideal properties of a number of scoring functions for the multiple-choice test item; that is, the type of selected-response item where there is one and only one correct option ("key") among any number of incorrect ones ("distractors"). Here, we report a similar analysis but for the type of selected-response item where there is an arbitrary integer number k of keys among c options in total; following Ma (2004), we call this type of test item a "multiple-response" one.…”