Peer and self assessment in massive online classes

Abstract

Peer and self-assessment offer an opportunity to scale both assessment and learning to global classrooms. This article reports our experiences with two iterations of the first large online class to use peer and selfassessment. In this class, peer grades correlated highly with staff-assigned grades. The second iteration had 42.9% of students' grades within 5% of the staff grade, and 65.5% within 10%. On average, students assessed their work 7% higher than staff did. Students also rated peers' work from their own country 3.6% higher than those from elsewhere. We performed three experiments to improve grading accuracy. We found that giving students feedback about their grading bias increased subsequent accuracy. We introduce short, customizable feedback snippets that cover common issues with assignments, providing students more qualitative peer feedback. Finally, we introduce a data-driven approach that highlights high-variance items for improvement. We find that rubrics that use a parallel sentence structure, unambiguous wording, and well-specified dimensions have lower variance. After revising rubrics, median grading error decreased from 12.4% to 9.9%.

Similarity and difference in fee-paying and no-fee learner expectations, interaction and reaction to learning in a massive open online course

“…Also important is how MOOC learning activities align with those being considered for delivering feedback at scale such as peer-and self-assessment (e.g. Kulkarni et al, 2013) or feedback generators (Whitelock, Gilbert & Wills, 2013).…”

Learner Behaviour in a MOOC Practice-oriented Course: In Empirical Study Integrating TAM and TPB

“…One common approach to overcome these issues is the mechanism of peer assessment (Kulkarni et al, 2013;Suen, 2014). Peer assessment is nothing new, but executing it within cyberspace is and requires technological tools.…”

Technology enhanced peer learning and peer assessment

“…However, as argued by Suen (2014) there is a risk that unmoderated peer assessment may suffer from lack of credibility. In a MOOC course on human centered interaction design Kulkarni et al (2013) compared student assessments with teacher assessments and found that that even though the majority of students assessed their peers within 10% of the teacher assessments, students generally assessed their peers higher than the teachers (7%). Furthermore, they also assessed peers from their own country higher than peers from other countries.…”