No abstract

No abstract

Usability assessment methods evolved from traditional human factors/ergonomics techniques beginning in the early 1980s. Following a brief historical introduction, we describe the four major categories of these methods: usability testing; usability inspections; surveys, interviews, and focus groups; and field methods. We describe the basic techniques of these methods, their strengths and weaknesses, their measurement characteristics, their validity and reliability, and how they are being applied to product development and assessment. We discuss the new challenges to usability assessment: (a) using the methods thoughtfully, (b) expanding the scope of assessment beyond usability and productivity to affect end user experience, (c) and emphasizing design solutions to problems in addition to simply finding the problems themselves.

The objective of this study was to investigate whether the aesthetic refinement of a software prototype is related to subjects' ratings of the usability of the prototype. We purchased a commercially available electronic device offering the functions of dictionary, calculator, and thesaurus (hereafter referred to as the Dictionary). We created four user interface prototypes of the Dictionary using line art, half-tone, gray scale, and color images of the product. The prototypes varied in the degree to which the displayed image resembled the physical look of the real product in terms of depth, tone, color, etc. All of the prototypes were interactive in that subjects could, with a mouse, make the prototypes operate like the Dictionary. Five groups of ten subjects each then made their ratings and performed tasks using either one of the prototypes or the Dictionary. The subjects rated ease of learning and use, forgiveness of mistakes, and aesthetics for the version they used, before and after performing tasks. Subjects who used a prototype also repeated a task using the real product and compared the two in terms of similarity of interaction and aesthetics. The results indicated that the aesthetic quality of prototypes within the range we varied did not bias users for or against the prototype's perceived usability. However, we learned that half-tone prototypes should be avoided when their coarse shadings decrease legibility. In addition, we found that making prototypes mimic the response time of a product or a design concept is important. When the real product produces slower response times, prototype performance may give an overly optimistic picture of the product's usability.

We asked five usability specialists to review the user interface to a phone-based, interactive voice response system. The experts were instructed to conduct their review independently in three one-hour sessions and to record each usability problem on a Problem Description Sheet along with the elapsed time from the beginning of the hour. Each expert then spent one hour reviewing their problem sheets and making a summary list of problems. Finally, the experts spent two hours together on a conference call discussing their impressions and coming to consensus on a prioritized list of problems and solutions. The results showed that when allocating expert time, it is more effective to have a greater number of experts spend fewer hours than to use fewer experts for more hours. The individual summaries included the majority of the severe problems, but left out many less severe problems and added new problems. The group report did not surface any new problems, but described the problems as being caused by more basic design flaws and proposed solutions that focused on the conceptual model on which the design was based.