This study investigated the effect of prototype fidelity on the information obtained from a usability test with potential users of a software product. Users engaged in a series of structured tasks using one of three prototypes which emulated a Microsoft® Windows 95™ based library search computer system and which differed only in their fidelity (low = paper; medium = screen shots, high = interactive Visual Basic prototype). Results indicated there were no significant differences as a function of prototype in the number and severity of problems encountered nor in the subjective evaluation of the product. More importantly, there was a high degree of commonality in the specific problems uncovered by users using the three prototypes. Other noteworthy results were that: (1) usability professionals (through informal heuristics analysis) and users differed in the incidence and commonality of problems identified, and (2) usability professionals did not agree in their ratings of problem severity.
This study examined the effect of concurrent and retrospective thinking-out-loud (TOL) on the frequency and value of user verbalizations during a software usability test. Three groups of users first learned to use an off-the-shelf database management package by means of a short tutorial and then engaged in six structured tasks. Users in the Concurrent condition thought-out-loud while performing the six tasks, whereas those in Retrospective-Immediate and Retrospective-Delayed conditions thought-out-loud while watching the videotape of their interaction with the software. Results indicated that there were no significant differences among the three conditions in performance or subjective evaluation of the software. More importantly, a verbal protocol analysis revealed that users in the Retrospective conditions spent more time making statements which had high value for designers than in the Concurrent condition. The value of verbalizations generated by the Retrospective conditions were not impacted by the 24 hour delay. The results were interpreted within context of differences in workload and in terms of the trade-off between increased value gained by using the retrospective paradigm versus increased cost of additional time to conduct the usability test.
The purpose of the study was to compare a team usability testing paradigm with that of the typical single user paradigm in terms of the quantity and quality of the user's verbalization (i.e. thinking out-loud) and performance. The study employed a three group design in which the type of usability paradigm (Single, Observer, Team) was manipulated. Users first learned to use an off-the-shelf database management package by means of a short tutorial and then engaged in six structured tasks. While engaging in the tasks, the users either thought-out-loud alone (Single condition), in the presence of an observer (Observer condition), or as participants of a team working on the tasks together (Team condition). Results indicated that there were no significant differences among the three conditions in terms of performance nor any extensive differences in their subjective evaluation of the software. However, users in the Team condition spent more total time verbalizing than those in the Single or Observer condition. More importantly, results of a verbal protocol analysis revealed that the Team spent more time making statements which had high value for designers than did the other two conditions (which did not differ from one another). When broken out by individual users in the Team, there were no significant differences between individual team members and users in the other two conditions in making high value comments. The results suggest that the Team paradigm may be more efficient in extracting high value information without any noticeable differences in performance or subjective impression of the software.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.