Is peer review an appropriate form of assessment in a MOOC? Student participation and performance in formative peer review

Abstract

Many aspects of traditional higher education must be reconceptualised for massive open online courses (MOOCs). Formative and summative assessment of qualitative work in particular requires novel approaches to cope with the numbers involved. Peer review has been proposed as one solution, and has been widely adopted by major MOOC providers but there is currently little evidence about whether it is appropriate in the MOOC context, or under what conditions. Here, we examine student participation, performance and opinions of a peer review task in a biomedical science MOOC. We evaluate data from approximately 200 student topic summaries and 300 qualitative peer reviews of those summaries, and compare these to student demographic data (gender, age, employment status, education, national language) and to performance in multiple choice tests (MCQs). We show that higher performance in the written topic summary correlated with both higher participation in the peer review task, and with writing higher quality peer reviews. Qualitative analysis of student comments revealed that student opinion on the usefulness of the peer review task was mixed: some strongly believed it benefitted their learning, while others did not find it useful or did not participate. We suggest instructional design strategies to improve student participation and increase learning gain from peer review in the MOOC context.

The Impact of Integrating a MOOC Platform on the Teaching of Computer Science Course: A Case Study

“…ST2 concluded that "MOOC courses will not terminate universities' learning, but it is one way to support us to gain more knowledge". This finding is consistent with Al-Atabi and DeBoer (2014) and Meek et al (2017) who think that hybrid learning offers more opportunity to support students' learning and achievement.…”

“…This researcher found that 90% of his students completed the activities related to his course, and they were very positive about the method of teaching the course. Meek et al (2017) concluded that when mixing a MOOC with a traditional one, their results showed that students' performance in the written topic summary correlated with higher participation of students in the integral peer review task. Similarly, Griffiths et al (2014, p. 4) reported that, "students in the hybrid sections did as well or slightly better than students in the traditional sections in terms of pass rates and learning assessments".…”

Developing a Strategic Approach to MOOCs

“…Agreed good practice and guidelines for peer review would help to make this more effective (O'Toole, 2013). For example, when designing peer review, there could be options for differentiated support or peer groups for students with different ability levels, and opportunities to improve peer review skills (Meek, Blakemore, & Marks, 2016).…”

Involvement of Learners’ Characteristics within the Allocation of Submissions in the context of Peer Assessment in MOOCs

“…A concern that is reflected in the proposal of assessment approaches that target more reliability and more validity, combined with the objective of ensuring effective feedback to learners [5].…”

“…The openness of MOOCs implies dealing with enrolled learners who are not true peers, because of their different backgrounds, abilities, objectives, and behavior patterns [5].…”