Assessing the reliability of self‐ and peer rating in student group work

Zhang, Bo; Johnston, Lucy; Kılıç, Gülşen Bağcı

doi:10.1080/02602930701293181

Cited by 45 publications

(38 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…More recently, Cho, Schunn, and Wilson (2006) found that the aggregate of at least four peer assessments were as reliable and valid as teacher assessments, whereas the reliability and validity of single peer assessments were much lower -presumably because students evaluate a subset of all teacher assessments and as a consequence develop different evaluative perspectives that are reflected in rating variability. Irrespective of the findings on the reliability of peer ratings versus teacher ratings (Cho et al, 2006;Falchikov & Goldfinch, 2000;Magin, 2001;Stefani, 1994;Topping, 2003;Zhang, Johnston, & Kilic, 2008), similarity in peer and teacher ratings provides no information as to whether the ratings affect students' subsequent performance. It is implicitly assumed that the high degree of similarity between peer and teacher ratings reflects rating fairness and that student responses to peer ratings will be similar to their responses to teacher ratings.…”

Section: Eed For Functional Developmentmentioning

confidence: 99%

Unravelling peer assessment: Methodological, functional, and conceptual developments

Strijbos

Sluijsmans

2010

Learning and Instruction

161

View full text Add to dashboard Cite

Peer assessment is an educational arrangement where students judge a peer's performance quantitatively and/or qualitatively and which stimulates students to reflect, discuss and collaborate. However, empirical evidence for peer assessment effects on learning is scarce, mostly based on student self-reports or involving comparison of peers' and teachers' ratings or anecdotal evidence from case studies. Systematic investigation of learning effects necessitates methodological, functional, and conceptual development in peer assessment research. This implies sound (quasi-)experimental studies, the definition of specific peer assessment mechanisms, and affiliations with other research domains. The articles in this special issue address these three needs and offer new directions for research.

show abstract

Section: Eed For Functional Developmentmentioning

confidence: 99%

Unravelling peer assessment: Methodological, functional, and conceptual developments

Strijbos

Sluijsmans

2010

Learning and Instruction

161

View full text Add to dashboard Cite

show abstract

“…The reliability and validity of peer grading have been researched primarily in the context of face-to-face higher education (Cheng & Warren, 1999;Cho et al, 2006;Falchikov & Goldfinch, 2000;Stefani, 1994;Zhang, Johnston, & Kilic, 2008). Reliability is usually measured by the consistency of scores given by multiple student graders, and validity is commonly calculated as the correlation coefficient between student-assigned scores and instructor-assigned scores, assuming that instructors can provide fair and accurate grading results.…”

Section: B Reliability and Validitymentioning

confidence: 99%

Peer Grading in a MOOC: Reliability, Validity, and Perceived Effects

Luo

Robinson

Park

2014

OLJ

View full text Add to dashboard Cite

Peer grading offers a scalable and sustainable way of providing assessment and feedback to a massive student population. However, currently there is little empirical evidence to support the credentials of peer grading as a learning assessment method in the MOOC context. To address this research need, this study examined 1,825 peer grading assignments collected from a Coursera MOOC with the purpose of investigating the reliability and validity of peer grading, as well as its perceived effects on students' MOOC learning experience. The empirical findings provide evidence that the aggregate of student graders can provide peer grading scores fairly consistent and highly similar to instructor grading scores. Student survey responses also indicate peer grading activities to be well received by a majority of MOOC students, who believe it was fair, useful, beneficial, and would recommend it to be included in future MOOC offerings. Based on the empirical results, this study concludes with a set of principles for designing and implementing peer grading activities in the MOOC context.

show abstract

“…Many stress the importance of creating an open dialogue with the students when explaining the scheme. Zhang and Johnston (2008) argue that student/teacher congruence is not necessary, since reliability of student ratings can be construed as consistent with each other. Concern has also been raised that the confidence in University assessment practices of 'external' communities, particularly those associated with employment of graduates for professional careers, may be negatively influenced but Langhan and Wheater (2003) dismiss this as of minor relevance if peer-assessment is integrated into a diverse portfolio of assessment, on the understanding that learning outcomes are being achieved.…”

Section: Potential Pitfallsmentioning

confidence: 99%

Multi-disciplinary Peer-mark Moderation of Group Work

Willmot

Pond

2012

IJHE

View full text Add to dashboard Cite

Self and peer assessment offers benefits for enhancing student learning. Peer moderation provides a convenient solution for awarding individual marks in group assignments. This paper provides a significant review of peer-mark moderation, and describes an award winning, web-based tool that was developed in the UK and is now spreading across the world as an open-source web application. It is available for use in any discipline. Qualitative research, at the home institution over several years, reinforces the evaluation of quantitative data extracted from the system and from an extensive user survey to confirm, update and strengthen the previous literature. The research also describes new insights into the thoughts of students, who appear to recognise the transparency that automated moderation offers. The statistics suggest few incidences of team-collusion when entering data, but indicate that peer-marking behaviour is influenced by group size, selection method and year of study. Students comment positively on the recognition of their levels of achievement within a team.

show abstract

Assessing the reliability of self‐ and peer rating in student group work

Cited by 45 publications

References 23 publications

Unravelling peer assessment: Methodological, functional, and conceptual developments

Unravelling peer assessment: Methodological, functional, and conceptual developments

Peer Grading in a MOOC: Reliability, Validity, and Perceived Effects

Multi-disciplinary Peer-mark Moderation of Group Work

Contact Info

Product

Resources

About