Selecting Effective Examples to Train Students for Peer Review of Open‐Ended Problem Solutions

Abstract

Background Students conducting peer review on open‐ended problem solutions require training. For that training, the selection of training examples is critical. Purpose This study explored how the characteristics of five example solutions used in training and their associated expert evaluations affected students' abilities to score peer team solutions on a model‐eliciting activity (MEA). Design/Method For this training, individual students reviewed a randomly selected example solution to an MEA. They were then asked to compare their review and an expert's review. Students were then assigned a peer team solution to review. An expert later rated the five training examples and 147 team solutions that had been peer reviewed. Differences between the scores assigned by the expert and a student to a training example and a peer team solution were used to compute the student's improvement as reviewer from training to peer review. ANOVA testing with Tukey's post hoc analysis identified differences in improvement based on the training example students saw during training. Results Statistically significant differences were found in the number of errors students made during peer review depending on the quality of the example they reviewed in training. Specifically, a low‐quality example and associated expert evaluation resulted in more accurate scoring during peer review. Conclusions While students typically ask to see exemplar solutions, this research suggests that there is likely greater value, for the purpose of training for peer reviewing, in having students see expert evaluations of lower‐quality examples.

Using social media affordances to support Ill-structured problem-solving skills: considering possibilities and challenges

“…When building learners' skills related to generating solutions, educators have used many methods: providing worked examples (Jonassen, 2011), using libraries of success and failure-based cases (Tawfik & Jonassen, 2013), suggesting and evaluating solutions through collaborative discussions (Ertmer & Koehler, 2014, using questioning and "reflective toss" to prompt deeper consideration (Hmelo-Silver & Barrows, 2006), requiring peer or expert review for evaluating solutions (Demiraslan Çevik et al, 2015;Verleger et al, 2016), and requiring learners to generate multiple solutions to a single problem (Ertmer et al, 2019).…”

A collaborative working model for enhancing the learning process of science & engineering students

“…In this case, a greater volume of feedback is available than when relying only on teacher feedback and a broader variety of opinions are provided on the work (Topping, 2009). However, the validity of peer assessment scores given by students requires supervision and a training process (Verleger et al, 2016). Automatic tools have been designed to assess the quality of reviews by students (Yadav and Gehringer, 2016;Ramachandran and Gehringer, 2012).…”

Applying professional methods of team feedback to improve teamwork in an interdisciplinary project-based upper-level design course

“…We therefore find it somewhat surprising that Harms and Roebuck have not been more widely or approvingly cited in the engineering education literature. Verleger et al [6] dismiss Harms and Roebuck in passing, grouping it with other examples and asserting that they "train students to give feedback using an overly-structured approach." However, the grounds for their objections, with regard to the BET/BEAR models, are not explained.…”