A comparative study of two usability evaluation methods using a web-based e-learning application

Abstract

Usability evaluation of e-learning applications is a maturing area, which addresses interfaces, usability and interaction from human-computer interaction (HCI) and pedagogy and learning from education. The selection of usability evaluation methods (UEMs) to determine usability problems is influenced by time, cost, efficiency, effectiveness, and ease of application. Heuristic evaluation (HE) involves evaluation by experts with expertise in the domain area and/or HCI. This comparative evaluation study investigates the extent to which HE identifies usability problems in a web-based learning application and compares the results with those of survey evaluations among end-users (learners). Severity rating was conducted on a consolidated set of usability problems and further comparison of findings was done on the major and minor problems. The results of HE correspond closely with those of the survey. However the four expert evaluators identified more problems than the 61 learners and identified 91% of the learners' problems, when major problems only were considered. HE by a competent and balanced set of experts showed itself to be an appropriate, efficient and effective UEM for e-learning applications.

Empirical validation of a usability inspection method for model-driven Web development

“…The Co-discovery Learning method was found to be slightly more effective than the others. − Ssemugabi and De Villiers [2007] reported a case study whose aim was to investigate the extent to which Heuristic Evaluation identifies usability problems in a Web-based learning application by comparing the results with those of Survey Evaluations among end-users. The Heuristic Evaluation performed by four expert evaluators proved to be an appropriate and effective usability evaluation method for e-learning applications.…”

“…− HE covers a broader range of usability aspects than other inspection methods such as, for instance, Cognitive Walkthroughs, whose usability definition is more focused on ease of navigation. − HE has provided useful results when used to conduct Web usability evaluations [Sutcliffe 2002;Allen et al 2006;Ssemugabi and De Villiers 2007]. − HE has often been used for comparison with other inspection methods [Costabile and Matera 2001;Chattratichart and Brodie 2004;Conte et al 2009].…”

How Usable Are the Learning Management Systems? The Users Have their Say

“…Existing questionnaire methods relating to the usability of E-learning applications cover several areas. For example, the researchers in [22] divided their questioning into three categories: general interface usability criteria, website-specific criteria for educational websites, and learner-centred instructional design that is grounded in learning theory and aims to provide effective learning. Additionally, the author in [6] used a questionnaire approach to assess ten areas: general system performance, software installation, manuals and online help, online tutorials, multimedia quality, information presentation, navigation, terminology and error messages, learnability, and overall system evaluation.…”

Does Student Crowdsourcing of Practice Questions and Animations Lead to Good Quality Materials?

“…Academic groups used forms of collaborative crowdsourced learning to study the effectiveness of student's assessment and peer instruction [7][10] [13][14] [16] . de Alfaro [7] created a crowdsourced grading tool, CrowdGrader, that allows students to grade and review their peer's homework submissions.…”

“…Ghauth concluded that with this recommendation system and rating system, 12.16% of users showed a significant improvement in understanding topics. Ssemugabi [16] compared two usability evaluation methods of web-based learning applications between 4 expert evaluators and 61 learners. Ssemugabi discovered that the expert evaluators identified 91% more problems with the web-based e-learning application than the learners.…”