Virtual Learning Environments (VLEs) are widely adopted in higher education to facilitate online methods of content delivery for the lecturers, to enable online submission for the students and to provide course management tools for the administration team. There are a variety of software solutions to choose from, modelled according to heterogeneous needs and functionalities. Despite the existence of clear organisational, pedagogical, and technological goals, the procurement and implementation of a VLE is a complex task with significant challenges. We present the case study of a university which went through the process of procuring a new VLE. We examine the usability of each VLE utilizing the System Usability Scale (SUS) and capture further feedback from the stakeholders by applying the Interactive Management (IM) methodology. The first part of the research focuses on the three VLEs remaining in contention during the final stages of the procurement process. The results of the usability evaluations are analysed, explained and compared. The second part of the study examines the selected and implemented VLE six months after its initiation. A usability test was carried out again on this VLE to examine changes since its launching. Additional feedback was collected from the stakeholders to support the fine-tuning process after the implementation. According to the evaluation of each user group, all the three VLEs performed below the average usability expectation. Generally, students evaluated the usability of the VLEs higher than the academics and administration staff. The usability scores of the students' evaluation from different courses and years show remarkable differences. The ranked and categorised feedback given by the stakeholders highlights the importance of planning, training and communication prior to and during the implementation process. Usability and learnability play important roles according to the feedback.
The System Usability Scale (SuS) survey is a widely respected tool for measuring usability. Generally, a SuS score is administered directly after a usability test to assess the usability and user experience of digital products. However, some researchers have used SuS as a survey as part of longitudinal ‘in the wild’ trials where SuS is often completed some period after the trial. The aim of this research was to determine if a participant’s memory of a product’s usability would change if a SuS survey was administered at different times after a test. Hence, we sought to understand if recalling the usability of a digital technology was affected by temporal bias or memory decay. This paper includes results and findings from two studies, study 1 involved evaluating a web application and study 2 involved evaluating a virtual learning environment. Collectively the two studies had 212 participants (n = 212). The findings conclude that there is no significant change of the user’s recollection of the usability of digital product as evidenced by an analysis of users who completed multiple SuS surveys over a short term period of 3 weeks or over an extended period of time of 6 months. RESEARCH HIGHLIGHTS The system usability scale is used to test for memory decay and temporal bias in judging the user experience of technologies at different time points. 212 participants took part in two studies ranging from 3 weeks to 6 months. There is no evidence that there is a temporal bias or memory decay when users complete a SuS survey at the two different time points of 3 weeks and 6 months.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.