Design implications from a usability study of GramStain‐Tutor^TM

Abstract

To examine interface issues in an educational software program, we recruited twenty-two users in health sciences departments for a usability study. Using the GramStain-Tutor TM (GST) CD-ROM, these users performed tasks in navigating the program and using various interactive features. Their actions and comments were recorded with a digital video recorder and an audio recorder. The main results of our study are: (a) the majority of the users did not use the multiple navigational options available in the program; (b) navigational patterns differed based on the training background and content knowledge of the users; (c) several visual cues critical to program use were not discovered by the majority of the users; and (d) icons representing specific functions were not intuitive from users' perspectives. We demonstrated how a usability study of a relatively small number of users can identify specific problems in interface design. We recommend these interface problems be addressed before conducting educational studies that examine how educational software programs affect student learning. Usability testingUsability testing is a collection of evaluation methods that is increasing in popularity. It is based on observing how users perform tasks, and obtaining feedback from users through formal tests or compilations of user comments (Nielson, 1993). These results lead to a set of recommendations that are fed back into the design process with the end goal of improving the product. Thus, usability is an iterative process of collecting response data or user feedback on a product, re-designing the product in response to that feedback, and re-testing the product to examine whether changes have enhanced the usability.In a typical usability session, the following issues are explored (Nielsen, 1993;Salvemini, 1998):• Can users easily learn a computer system supported by sufficient navigational control? • Is the system efficient to use or is usability hindered by an excessive number of icons and menu options? • Are system features easy to remember so that time to re-learn the system is minimized? • Does the system have low errors and efficient error recovery mechanisms? • Is the system pleasant to use?These questions can be answered by having users complete tasks while using the computer system. Tasks refer to a set of representative actions routinely performed in a particular working context. Often, users of different levels of content expertise and computer skills are asked to carry out these tasks. Using as few as 4-5 subjects who participate in a task-based study, 80% of usability problems can be identified (Virzi, 1992).Usability techniques include both qualitative and quantitative approaches, such as observations, think aloud protocols, questionnaires, interviews, logging actual use, and user feedback through debriefing (Nielson, 1993). Typically, a checklist, audio recorders and video recorders are used in observation sessions. Users are encouraged to articulate their thought processes while performing tasks. This "think-aloud...

Improving a Counselor Education Web Site Through Usability Testing: The Bibliotherapy Education Project

“…Perceptions of ease of use included the unanimously positive comments regarding the constant navigation bar, which appears on the left side of all screens on the Web site. This result is consistent with the findings in other usability studies that users prefer a constant navigational indicator on all pages of a Web site (Kim et al, 2001).…”

“…The usability study by Kim et al (2001) similarly found that "subtle visual cues are likely to go undetected when an interface demands that a user recognize embedded text [and] navigational features" (p. 603). These researchers recommend enlarging or otherwise enhancing the visibility of text, through distinct and consistent colors for certain types of navigation or content themes.…”

“…Participants were instructed to verbalize their thought processes as they addressed a list of tasks. This think aloud protocol is common to usability studies and enhances the observational information recorded (Kim et al, 2001;Krug, 2006). Participants were also told that they should approach the Web site as if they were working independently, meaning that the observers/note takers were available to clarify the nature of the tasks but would wait until the end of the session before answering questions concerning the BEP or Web site.…”

“…Kim, Brock, Orkand, and Astion (2001) described usability testing as an iterative process, in which user feedback influences modifications that are then retested. Krug (2006) pointed out that the initial usability test would uncover major usability issues such as a user not being able to find a link to important information or having difficulty finding the subsequent step in a process.…”

Development of a Tool for Evaluating Multimedia for Surgical Education

“…This recommendation is supported by previous work, which has spotlighted the fact that few evaluations of the specific elements of instructional design for surgical training multimedia-the user interface-have been conducted, see [8]. This has been characterized in the literature as an oversight that may mask the full educational benefits that multimedia could offer, where people adopt new learning methods without knowing their effectiveness, e.g., [9].…”