On the learning benefits of confidence-weighted testing

Abstract

Taking multiple-choice practice tests with competitive incorrect alternatives can enhance performance on related but different questions appearing on a later cued-recall test (Little et al., Psychol Sci 23:1337–1344, 2012). This benefit of multiple-choice testing, which does not occur when the practice test is a cued-recall test, appears attributable to participants attempting to retrieve not only why the correct alternative is correct but also why the other alternatives are incorrect. The present research was designed to examine whether a confidence-weighted multiple-choice format in which test-takers were allowed to indicate their relative confidence in the correctness of one alternative compared with the others (Bruno, J Econ Educ 20:5–22, 1989; Bruno, Item banking: Interactive testing and self-assessment: Volume 112 of NATO ASI Series, pp. 190–209, 1993) might increase the extent to which participants engaged in such productive retrievals. In two experiments, such confidence-weighted practice tests led to greater benefits in the ability of test-takers to answer new but related questions than did standard multiple-choice practice tests. These results point to ways to make multiple-choice testing a more powerful tool for learning.

Confidence-weighted Testing as an Impactful Education Intervention within a Pediatric Sepsis Quality Improvement Initiative

“…Confidence-weighted testing improves learning of the material and increases learners’ knowledge of educational content tied to both correct and incorrect answer choices for each question. 10 , 11 Additionally, it allows for individualized feedback to optimize the learner’s time and knowledge acquisition. For example, when a learner confidently answers a question correctly, no additional instruction or corrective feedback is provided during the module because it would not improve learning.…”

“…Although other studies have demonstrated the value of eLearning in pediatric sepsis, 7 this report aims to describe our experience applying confidence-weighted testing to enhance learning. 10 , 11 To the extent that our initiative was successful, the principles we used in developing and evaluating our intervention could be incorporated in future education components of other QI initiatives to optimize learning and identify specific areas of local misinformation that could inform subsequent improvement efforts.…”

Implementing Confidence Assessment in Low-Stakes, Formative Mathematics Assessments

“…However, with repeated use of CA over time, calibration tends to improve (Gardner-Medwin & Curtin, 2007). It has been found that CA, often implemented in a multiple-choice context, can encourage self-checking, self-explanation and higher-level reasoning (Gardner-Medwin & Curtin, 2007;Sparck et al, 2016), and improve test validity by reducing gender bias (Hassmén & Hunt, 1994). Consequently, it has been suggested (Foster, 2016(Foster, , 2017Foster et al, 2021) that CA might have considerable potential benefits for formative assessment in the school mathematics classroom.…”

School students’ confidence when answering diagnostic questions online

“…We cannot completely rule out this possibility; however, the fact that confidence scores ranged across all of the emojis is evidence that large numbers of students were willing to register low confidence. One way in which to collect confidence measures with higher validity could be to implement confidenceweighted assessment, in which the student's mark is increased when the student expresses high confidence and is correct but is decreased when the student expresses high confidence but is incorrect (see Dirkzwager, 2003;Foster, 2016Foster, , 2021Rosewell, 2011;Sparck et al, 2016). Such an approach is intended over time to incentivise truthful confidence ratings and is particularly easy to implement in a multiple-choice model, where students can be asked to allocate, say, 100 marks across the four options A-D.…”