Evaluating and Comparing the Usability of Web-based Course Management Systems

Abstract

Executive SummaryCourse Management Systems (CMS) are an increasingly important part of academic systems in higher education. When choosing a Course Management System for an educational institution, the usability of the system is the key to the effectiveness and efficiency of the online courses that are to be implemented. The goal of this paper is to report the results of a comparative usability study conducted in 2008-2009 on two different course management systems: BlackBoard and Moodle. students enrolled in the Fall 2008 and Spring 2009 sections of Introduction to EducationalTechnology participated in the study (72 and 63 respectively). At the beginning of each semester, participants were randomly divided into two groups to experience different CMSs at different times.It can be concluded from this study that in almost every module or function comparison that was made, Moodle was favored by course participants over Blackboard with the exception of the Discussion Board module where scores were not significantly different. At the end of the study, the researchers concluded that use of Moodle in online courses can be a suitable alternative to the current CMS system (BlackBoard). In fact, now that the pilot has showed that Moodle is as effective as BlackBoard, the researchers have already shared their experiences with other faculty members and expanded their investigations by involving numerous other online courses, instructors, and students, because the product showed significant potential for further examination.This study adds to the growing body of studies that are carried out to see if an open source CMS (Moodle) warrants consideration as an alternative to the institution's current course management system. In addition to comparing the students' feedback quantitatively, this study also tried to explain in detail what specific component / function of each CMS students found useful or better than in the other. Rather than focusing only on student satisfaction scores, this study further investigated what aspect of each module for each CMS course participants particularly liked or disliked.

Revisiting Instructor Misbehaviors: A Revised Typology and Development of a Measure

“…As Jones and Healing (2010) noted, students arrive at college "already schooled in a variety of practices related to learning and technology" (p. 344). To supplement student learning, many instructors have integrated technology into their lectures including PowerPoint presentations (Berk, 2011), student response systems/clicker technology (Powell, Straub, Rodriguez, & VanHorn, 2011), and online course management systems such as Blackboard or Desire2Learn (Unal & Unal, 2011). While this use of technology is intended to aid students in their learning, instructors who misuse technology run the risk of losing their credibility (Schrodt & Turman, 2005).…”

Collaboration, connectedness, and community: An examination of the factors influencing student persistence in virtual communities

“…The survey instrument consisted of 64 questions that were designed to measure the factors of collaborative learning (CLS or Collaborative Learning Scale) (So & Brush, 2008), campus connectedness (CCS or campus connectedness scale) (Summers, et al, 2005), sense of community (SCS or Sense of Community Scale) (Chavis, Lee & Acosta, 2008), affective organizational commitment (AOC or Affective Organizational Commitment Scale) (Meyer, Allen, & Smith, 1993), turnover intention (TRN or Turn Over Intention) (Kelloway, Gottlieb, & Barham, 1999), and system usability (SUS or System Usability Scale) (Brooke, 1996;Bangor, Kortum, & Miller, 2008;Unal & Unal, 2011). The instrument was tailored to the terminology used for communities and academia, and questions used a 5-point Likert-type scale with the following possible responses: strongly disagree, disagree, neutral, agree, and strongly agree.…”

A Hybrid Approach for Selecting a Course Management System: A Case Study

“…They conclude that institutions entering a VLE or looking to change their current CMS could use the features discussed in the paper to evaluate and compare the CMS proposals under consideration. Unal and Unal (2011) discuss usability of two widely used course management systems, Moodle and Blackboard, from students' perspective and conclude that Moodle is favored by students over Blackboard for most features. The study in our paper, however, goes a step further and discusses CMS functionality from all stakeholders' perspectives.…”