Teachers' use of rubrics to score non‐traditional tasks: factors related to discrepancies in scoring

Meier, Sherry L.; Rich, Beverly S.; Cady, JoAnn

doi:10.1080/09695940600563512

Cited by 34 publications

(18 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Criteria are manageable parts, properties or attributes of the content or competence to be evaluated. They unravel the meaning of the competence to be measured (East, 2009;Meier, Rich, & Cady, 2006;Smith & Tillema, 2007). For example the competence 'problem solving' might consist of the criteria 'analysing a problem', 'inferencing divergently' and 'developing surprising/original solutions'.…”

Section: Criteria and Standards For Quality Assessmentmentioning

confidence: 99%

Criteria and standards of generic competences at bachelor degree level: A review study

Strijbos

Engels

Struyven

2015

Educational Research Review

View full text Add to dashboard Cite

Section: Criteria and Standards For Quality Assessmentmentioning

confidence: 99%

Criteria and standards of generic competences at bachelor degree level: A review study

Strijbos

Engels

Struyven

2015

Educational Research Review

View full text Add to dashboard Cite

“…Other ways of investigating reliability Two of the studies did additional investigations of the question of how consistent scoring is when rubrics are used. One study (Meier, Rich, and Cady 2006) is described in this section. The other (Cho, Schunn, and Wilson 2006) is described in the section on Validity later.…”

Section: Reliabilitymentioning

confidence: 99%

“…These studies are described in some detail because they expand arguments for reliability and validity beyond simply reaching a statistical threshold level. Meier, Rich, and Cady (2006) studied a sample of five eighth-grade teachers who selected mathematics performance tasks to use in their classroom. Reliability was investigated by charting the amount of difference between the teachers' and researchers' scores.…”

Section: Reliabilitymentioning

confidence: 99%

The quality and effectiveness of descriptive rubrics

2014

View full text Add to dashboard Cite

This review synthesizes the findings of studies of the use of rubrics in education settings published from 2005 to 2013. The review included studies only if the rubrics involved met the definition of having coherent sets of criteria and performance level descriptions for those criteria. Compared to the results of a previous review by Jonsson and Svingby (Educational Research Review 2(2): [130][131][132][133][134][135][136][137][138][139][140][141][142][143][144] 2007), the frequency, scope, and rigor of studies of rubrics have increased in recent years. Rubrics yield information of sufficient quality if certain conditions are met, most notably having clear and focused criteria. Evidence regarding the effects of rubrics on performance is positive overall. Evidence of the effects of rubrics on self-regulation of learning is mixed, though positive associations between rubric use and motivation to learn were identified in some studies. IntroductionA rubric is a coherent set of criteria for students' work that includes descriptions of levels of performance quality on the criteria (Brookhart 2013b). Rubrics by this definition differ from rating scales, which have criteria but no performance level descriptions, although in common parlance these are often called "rubrics". Several important purposes claimed for rubrics, that they facilitate student self-assessment, facilitate teacher and peer feedback, and help students envision what to do to improve their work, only make sense if both criteria and performance level descriptions are present (Andrade 2000; Jonsson and Svingby 2007). The purpose of this article is to document what is known about (1) the quality of assessment information from rubrics and (2) the effects of rubric use on student learning and motivation to learn, using the method of a literature review. Rubric use in educational settingsRubrics enjoy wide use in primary, secondary, and post-secondary education. They arose as part of a response to research in the 1980s suggesting that students were better at repeating facts and concepts than applying them, and the consequent interest in performance assessment (Lane and Tierney 2008) and standards-based reform (Brookhart 2013a). Performance assessment can reflect students' abilities to solve real-world problems, analyze and synthesize information, and apply their knowledge and skills only if they have carefully designed scoring procedures with clear criteria. Rubrics with criteria and performance level descriptions are deemed an effective vehicle for organizing and communicating criteria and performance expectations and for use in scoring and/or providing feedback on student work (Lane and Tierney 2008).Beginning in the 1990s, two other developments have bolstered the use of rubrics: increasing emphasis on formative assessment in education at all levels (Andrade and Cizek 2010) and increasing emphasis on assessment and evaluation in the accreditation process in post-secondary education (Gerretson and Golson 2005; Kerby and Romine 2010). Formative assessment an...

show abstract

“…Dennis, Newstead, and Wright 1996;Gipps and Murphy 1994;Harlen 2005;Meier, Rich, and Cady 2006;Pollitt and Murray 1996). It has also been shown that markers are susceptible to the "halo effect" of finding what they are predisposed to find based on prior knowledge of a student (Harlen 2004).…”

Section: Lower Levels Of Inter-rater Reliability Have Indeed Been Foumentioning

confidence: 99%

“…For instance, an American study compared the scores awarded by five middle school mathematics teachers, who had been trained to use a rubric for marking non-traditional mathematical tasks, with the scores given by an expert group composed of mathematics education researchers who used the same rubric (Meier, Rich, and Cady 2006). The results indicated that four of the five teachers experienced difficulty when required to give equal consideration to two separate criteria (the correctness of procedures and fullness of explanations), focusing on one at the expense of the other.…”

Section: Judgment Processes In Markingmentioning

confidence: 99%

Marking as judgment

Brooks

2012

Research Papers in Education

View full text Add to dashboard Cite

An aspect of assessment which has received little attention compared with perennial concerns, such as standards or reliability, is the role of judgment in marking. This paper explores marking as an act of judgment, paying particular attention to the nature of judgment and the processes involved. It brings together studies which have explored marking from a psychological perspective for the purpose of critical discussion of the light they shed on each other and on the practice of marking. Later stages speculate on recent developments in psychology and neuroscience which may cast further light on educational assessment.

show abstract

Teachers' use of rubrics to score non‐traditional tasks: factors related to discrepancies in scoring

Cited by 34 publications

References 7 publications

Criteria and standards of generic competences at bachelor degree level: A review study

Criteria and standards of generic competences at bachelor degree level: A review study

The quality and effectiveness of descriptive rubrics

Marking as judgment

Contact Info

Product

Resources

About