The evaluation of text editors: methodology and empirical results.

Abstract

This paper presents a methodology for evaluating text editors on several dimensions: the time it takes experts to perform basic editing tasks, the time experts spend making and correcting errors, the rate at which novices learn to perform basic editing tasks, and the functionality of editors over more complex tasks. Time, errors, and learning are measured experimentally; functionality is measured analytically; time is also calculated analytically. The methodology has thus far been used to evaluate nine diverse text editors, producing an initial database of performance results. The database is used to tell us not only about the editors but also about the users—the magnitude of individual differences and the factors affecting novice learning.

An overview of semantic search evaluation initiatives

“…Past studies on learnability have focused on initial learnability, referring to the initial performance with the system; whilst others have studied extended learnability, referring to the change in performance over time (Grossman et al, 2009). Additionally, Roberts and Moran (1983); Whiteside et al (1985); Davis et al (1989); Hearst et al (2002) showed that learnability and usability are congruent.…”

Creating a Framework for Improving the Learnability of a Complex System

“…They refer to several studies indicating that the concepts of learnability and usability are strongly related and even congruent. Roberts & Moran (1983), for example, found that procedural complexity underlies both the performance of experts and the learning of novices. Whiteside et al (1985) have also stated that the concepts of usability and learnability are congruent.…”

“…The tests were expected to reveal issues that are problematic for new users. Elliott et al (2002) and Roberts and Moran (1983), for example, have evaluated learnability with scenario-based tests in which users were observed while completing test tasks. Corresponding methods have been used by numerous other researchers for evaluating usability.…”

A review of scientific visualization software for the Macintosh computer

“…One would be a rigorous, quantitative evaluation-a strict comparison of the programs in terms of their performance over a baseline set of tasks. Roberts and Moran (1983) developed such a method for evaluating text editors in a series of representative tasks, and their technique was reviewed and improved by Borenstein (1985). Cohill, Gilfoil, and Pilitsis (1988) attempted to measure the utility of application software by rating packages on factors such as functionality, usability, performance, support, and documentation.…”