How Accurate Is Peer Grading?

Abstract

Previously we showed that weekly, written, timed, and peer-graded practice exams help increase student performance on written exams and decrease failure rates in an introductory biology course. Here we analyze the accuracy of peer grading, based on a comparison of student scores to those assigned by a professional grader. When students graded practice exams by themselves, they were significantly easier graders than a professional; overall, students awarded ≈25% more points than the professional did. This difference represented ≈1.33 points on a 10-point exercise, or 0.27 points on each of the five 2-point questions posed. When students graded practice exams as a group of four, the same student-expert difference occurred. The student-professional gap was wider for questions that demanded higher-order versus lower-order cognitive skills. Thus, students not only have a harder time answering questions on the upper levels of Bloom's taxonomy, they have a harder time grading them. Our results suggest that peer grading may be accurate enough for low-risk assessments in introductory biology. Peer grading can help relieve the burden on instructional staff posed by grading written answers—making it possible to add practice opportunities that increase student performance on actual exams.

Questions for Assessing Higher-Order Cognitive Skills: It's Not Just Bloom’s

“…The data are equivocal. Some research has shown that students do not perform as well on exams with higher Bloom's levels (Knecht, 2001; Freeman and Parks, 2010). Additionally, Freeman and colleagues (2011), found a strong negative association between a weighted Bloom's index and predicted exam score.…”

Validity of peer grading using Calibrated Peer Review in a guided-inquiry, conceptual physics course

“…In general, research indicates that peer review provides a valid student evaluation (compared to review by an expert) and promotes learning [4,5,14,15,[17][18][19][20][21][22]. Harris describes using peer-assessed laboratory reports in large biology classes, and reports excellent correlation between peer and expert scoring [18].…”

“…Sadler and Good trained students with a scoring rubric and report a very high correlation between peer-assigned grades and teacherassigned grades, though they note some patterns of bias when students assigned grades [15]. Freeman reports on the use of peer assessment of written practice exams in a large introductory biology course, and finds that students were significantly easier graders than an expert, but that the student grading was "good enough to use" in their context (practice exams representing 11%-15% of the points in the course) [17]. Freeman also notes that students were less reliable peer graders of higher-order questions involving application, analysis, synthesis, and evaluation.…”

Teaching and learning programming and software engineering via interactive gaming

“…However, in general, the accuracy achieved by peer grading may not be satisfactory [9]. In the context of MOOCs, "resistance to peer grading", "concerns about privacy", "turning in blank assignments to get access to view the work of others", "the need for assessment to be part of a conversation" have been identified as issues with peer grading [14].…”