Rubrics enhance accuracy and reduce cognitive load in self-assessment

Abstract

Rubrics, which are designed to inform learners about assessment criteria and quality levels concerning a task assignment, are a widely used means to support learners in self-assessing their task performance. Usually, rubrics enhance task performance. Surprisingly, however, relatively little is known about the underlying mechanisms via which rubrics exert their beneficial effects. Although it is frequently suggested that the effects of rubrics on task performance are due to the fact that they support learners in accurately self-assessing their level of performance, which, in turn, paves the way for effective regulation, the empirical evidence that supports this notion is relatively scarce. Tightly controlled experimental studies, which allow isolating the effects of rubrics on the accuracy of self-assessments, are scarce. The present study was designed to close this gap. Specifically, in an experiment with N = 93 high school students we tested whether providing learners with a rubric in self-assessing the quality of self-written scientific abstracts would enhance judgment accuracy. As main results, we found that the rubric group showed higher absolute accuracy and lower bias. Furthermore, the rubric reduced cognitive load in self-assessing. These findings support the notion that one mechanism via which rubrics unfold their efficacy is by enhancing judgment accuracy.

Assessing the quality of science teachers' lesson plans: Evaluation and application of a novel instrument

“…The RALP allows a stable (RQ 1.2a) and intersubjectively comprehensible (RQ 1.2b) scoring procedure. As stated above, we intend the RALP to be used by teacher educators for grading purposes and by TSTs as a learning and self‐assessment tool (Brookhart, 2018; Krebs et al, 2022; Panadero et al, 2018). For that purpose, we suggest providing at least one coded lesson plan in addition to the scoring rubric so that the target group gets an idea of how the rubric can be applied and which parts of a typical lesson they mainly need to focus on.…”

“…First, the assumption that the RALP applies to all sciences must be investigated. Moreover, bearing in mind that the RALP is not only intended to be used as a grading tool for summative assessment purposes but also for formative assessment (Panadero & Jonsson, 2020) and self‐assessment purposes (Krebs et al, 2022), the following research desiderata (i.e., open questions for further research) particularly relate to the interpretation vertex of the Assessment Triangle (Figure 2; Pellegrino et al, 2016): If measuring prospective teachers' PCK (e.g., using the PCK‐in biology inventory; Großschedl et al, 2019), is there evidence of convergent validity, indicating that the RALP does indeed measure PCK ( Validity evidence based on relations to other variables )? How do prospective teachers engage with the RALP when applying it to their lesson plans?…”

“…How do prospective teachers engage with the RALP when applying it to their lesson plans? Can they improve a lesson plan after self-assessment (Krebs et al, 2022;Scherer, 2017; Validity evidence based on response processes)? If the RALP is applied by a teacher educator, by a peer student (peer assessment), or by students themselves (self-assessment) and the authors of the lesson plans are requested to improve their lesson plan afterward (Ozogul et al, 2008): Which of these three types 2013) concluding remarks at the end of his critical analysis of the expediency of Kane's (2013) terminological differentiation, we assume there is only one argument, the validity argument, which is the term we will adhere to in this paper.…”

“…Rubrics can be used for research purposes on the one hand and for formative or summative assessment in teacher education on the other hand (Krebs et al, 2022; Panadero et al, 2018; Panadero & Jonsson, 2013). This would also account for lesson planning (Ozogul et al, 2008), which is a crucial aspect to teach in science teacher training (Drost & Levine, 2015; Karlström & Hamza, 2021).…”

“…This would also account for lesson planning (Ozogul et al, 2008), which is a crucial aspect to teach in science teacher training (Drost & Levine, 2015; Karlström & Hamza, 2021). Since lesson planning is a cognitively demanding task (König et al, 2021) and rubrics have been shown to decrease the subjective cognitive load in self‐assessment processes (Krebs et al, 2022), a scoring rubric on lesson planning is assumed to help prospective science teachers in better responding to the challenges of lesson planning. A distinction is commonly made between holistic and analytic scoring rubrics (Sadler, 2009; Tomas et al, 2019): Holistic rubrics consist of one single scale of combined criteria evaluated simultaneously, enabling an overall evaluation in a time‐consuming way.…”