Not All Rubrics Are Equal: A Review of Rubrics for Evaluating the Quality of Open Educational Resources

Abstract

The rapid growth in Internet technologies has led to a proliferation in the number of Open Educational Resources (OER), making the evaluation of OER quality a pressing need. In response, a number of rubrics have been developed to help guide the evaluation of OER quality; these, however, have had little accompanying evaluation of their utility or usability. This article presents a systematic review of 14 existing quality rubrics developed for OER evaluation. These quality rubrics are described and compared in terms of content, development processes, and application contexts, as well as, the kind of support they provide for users. Results from this research reveal a great diversity between these rubrics, providing users with a wide variety of options. Moreover, the widespread lack of rating scales, scoring guides, empirical testing, and iterative revisions for many of these rubrics raises reliability and validity concerns.Finally, rubrics implement varying amounts of user support, affecting their overall usability and educational utility.

What Research Says About MOOCs – An Explorative Content Analysis

“…The research areas covered in these articles can Furthermore, the evaluation of MOOCs and quality assurance is a very prominent and relevant topic in the publications. Rather than developing new quality frameworks for MOOCs from scratch, it is recommended to build upon quality models and instruments that were developed to measure the quality of multimedia applications, learning objects, and open educational resources (see Yuan & Recker, 2015).…”

How Korean Language Arts Teachers Adopt and Adapt Open Educational Resources: A Study of Teachers’ and Students’ Perspectives

“…Although many of the previous studies were conducted to investigate the correlation of OER in the classroom, previous published studies have only focused on the contexts of the United States (e.g., Kelly, 2014;Pitt, 2015;Yuan & Recker, 2015), United Kingdom (e.g., Bell, 2011;Connolly, 2013), Canada (e.g., Ives & Pringle, 2013;McGreal, Anderson, & Conrad, 2015;McKerlich, Ives, & McGreal, 2013), India (e.g., Khanna & Basak, 2013;Kumar, 2009;Panda, 2011), and South Africa (e.g., Mtebe & Raisamo, 2014a, 2014bPrasad & Usagawa, 2014 While uncertainty still exists about how educators may use OER differently worldwide, this article reports on a study that addressed these constraints through a mixed-method approach with an ethnographic lens.…”

The Validity and Instructional Value of a Rubric for Evaluating Online Course Quality: An Empirical Study

“…Ten course quality rubrics were ultimately selected based on the following criteria: they (a) were used for evaluating the quality of online courses; (b) consisted of more than two dimensions, with accompanying definitions of the dimensions; and (c) were used in higher education settings. Building on the approach used in a prior review of the quality rubric literature (Yuan & Recker, 2015), we examined online course quality rubrics used by higher education institutions in terms of three aspects: (a) development process, (b) quality dimensions, and (c) and results of reliability and validity testing.…”

“…While quality rubrics are commonly used in many higher education institutions, few rubrics have been empirically tested in terms of their reliability or validity (Yuan & Recker, 2015). Moreover, an often-ignored aspect of course quality is its influence on online interactions and student outcomes; in other words, the instructional value of the rubric.…”