Theoretical evaluation of partial credit scoring of the multiple-response test item

Abstract

We compute and compare statistics of five different scoring rules for the selected-response type of test items where the number of keys is an arbitrary integer and the test-takers are perfectly rational agents. We consider a hypothetical test of factual recognition, in which the underlying ability that we seek to measure is the fraction of the item options that the test-taker truly recognizes (and not only guesses correctly), assumed directly proportional the test-taker’s domain knowledge. From these comparisons, two of these scoring rules are singled out as superior to the others.