The quality and effectiveness of descriptive rubrics

Abstract

This review synthesizes the findings of studies of the use of rubrics in education settings published from 2005 to 2013. The review included studies only if the rubrics involved met the definition of having coherent sets of criteria and performance level descriptions for those criteria. Compared to the results of a previous review by Jonsson and Svingby (Educational Research Review 2(2): [130][131][132][133][134][135][136][137][138][139][140][141][142][143][144] 2007), the frequency, scope, and rigor of studies of rubrics have increased in recent years. Rubrics yield information of sufficient quality if certain conditions are met, most notably having clear and focused criteria. Evidence regarding the effects of rubrics on performance is positive overall. Evidence of the effects of rubrics on self-regulation of learning is mixed, though positive associations between rubric use and motivation to learn were identified in some studies. IntroductionA rubric is a coherent set of criteria for students' work that includes descriptions of levels of performance quality on the criteria (Brookhart 2013b). Rubrics by this definition differ from rating scales, which have criteria but no performance level descriptions, although in common parlance these are often called "rubrics". Several important purposes claimed for rubrics, that they facilitate student self-assessment, facilitate teacher and peer feedback, and help students envision what to do to improve their work, only make sense if both criteria and performance level descriptions are present (Andrade 2000; Jonsson and Svingby 2007). The purpose of this article is to document what is known about (1) the quality of assessment information from rubrics and (2) the effects of rubric use on student learning and motivation to learn, using the method of a literature review. Rubric use in educational settingsRubrics enjoy wide use in primary, secondary, and post-secondary education. They arose as part of a response to research in the 1980s suggesting that students were better at repeating facts and concepts than applying them, and the consequent interest in performance assessment (Lane and Tierney 2008) and standards-based reform (Brookhart 2013a). Performance assessment can reflect students' abilities to solve real-world problems, analyze and synthesize information, and apply their knowledge and skills only if they have carefully designed scoring procedures with clear criteria. Rubrics with criteria and performance level descriptions are deemed an effective vehicle for organizing and communicating criteria and performance expectations and for use in scoring and/or providing feedback on student work (Lane and Tierney 2008).Beginning in the 1990s, two other developments have bolstered the use of rubrics: increasing emphasis on formative assessment in education at all levels (Andrade and Cizek 2010) and increasing emphasis on assessment and evaluation in the accreditation process in post-secondary education (Gerretson and Golson 2005; Kerby and Romine 2010). Formative assessment an...

The Reliability and Validity of Peer Review of Writing in High SchoolAP English Classes

“…Rubrics are widely used in K–12 writing instruction and for high‐stakes writing assessments. Overall, rubrics have been shown to increase student achievement and motivation in writing, but only when the rubrics convey clear and specific descriptions of quality levels for task‐specific criteria (Andrade, Du, & Wang, ; Brookhart & Chen, ).…”

Supporting sustained adoption of education innovations: The Designing for Sustained Adoption Assessment Instrument

“…It is typical for rubric reliability to be established using consensus agreement. According to the literature, 70 % agreement is a typical threshold used for acceptable reliability; however, it should be noted that agreement depends heavily on the number of levels to a rubric (Brookhart and Chen 2014;Jonsson and Svingby 2007). The next step was to test the instrument with different populations to ensure that members of the intended audience for the DSAAI were consistent in interpreting the assessment instrument and scoring the propagation plans of different educational innovations, including their own.…”

Learning to See New Things: Using Criteria to Support Pre-service Teachers' Discernment in the Context of Teachers' Relational Work

“…Furthermore, criteria belong to a given community of practice, which means that the meaning attached to them is not easily transferred to other contexts. In other words, in order to understand the criteria, you also have to have some familiarity with the practice to which they belong.This understanding of criteria is also reflected in findings from empirical research, where several research reviews suggest that explicit criteria (in the form of scoring rubrics) may have the potential to promote student learning by clarifying expectations, but not without a thorough implementation (Jonsson and Svingby, 2007;Reddy and Andrade, 2010;Panadero and Jönsson, 2013;Brookhart and Chen, 2015;Brookhart, 2018). In particular, there is a distinction between school settings, which typically require more comprehensive implementations, and the highereducation context, where students are often able to use criteria productively even with very limited efforts to implement them (Panadero and Jönsson, 2013;Jonsson and Panadero, 2017;Brookhart, 2018).…”