An axiomatization of multiple-choice test scoring

Abstract

Abstract. This note axiomatically justifies a simple scoring rule for multiplechoice tests. The rule permits choosing any number, k, of available options and grants 1/k-th of the maximum score if one of the chosen options is correct, and zero otherwise. This rule satisfies a few desirable properties: simplicity of implementation, non-negative scores, discouragement of random guessing, and rewards for partial answers. This is a novel rule that has not been discussed or empirically tested in the literature.JEL Classification: C44, A2, I20

Theoretical evaluation of partial credit scoring of the multiple-choice test item

“…We have compared and discussed the optimal scoring rule for multiplechoice items for which partial knowledge is expected to contribute to the test-taking strategy. Of the rules tested in our simulations, the modified Zapechelnyuk rule (Zapechelnyuk, 2015) is probably the best, although this choice is somewhat subjective. After that comes arguably the simple "no guessing" rule which does not award partial knowledge, but which penalizes guessing.…”

“…It has also been suggested (Coombs et al, 1956;Akeroyd, 1982;Bush, 2001;Bradbard et al, 2004;Zapechelnyuk, 2015;Otoyo and Bush, 2018;Frandsen and Schwartzbach, 2006;Dressel and Schmid, 1953) that partial marks should be awarded to test-takers who indicate correctly that they know some options are wrong, rather than hazard a guess on the right answer. The added complexity of choice does not seem to pose any significant problems for the test-takers (Ben-Simon et al, 1997).…”

“…The problem with scoring systems of partial knowledge is that there is arbitrariness in how to award partial credit. While some authors derive scoring systems from different axioms (Frandsen and Schwartzbach, 2006;Zapechelnyuk, 2015), such an approach ignores the question of the actual validity of the test and may like this lead to unintended results, no matter how simple or appealing the underlying axioms are. Corroborating tests and their scoring systems against real-world performance is a time-consuming and expensive endeavor.…”

Singular Choices for Multiple Choice

“…We show that Frandsen and Schwartzbach's singular choice is not as complex to adopt as it has been presented to be [17] and that it is more than a stepping stone towards other scoring strategies [9,18]. Our message is that it is actually simple, powerful, and scalable:…”

“…• Zapechelnyuk [18] presents a new axiomatization of multiple-choice test scoring, and points out that his axioms are not satisfied by the singular choice.…”