Using occurrence properties of defect report data to improve requirements

Abstract

Defect reports generated for faults found during testing provide a rich source of information regarding problematic phrases used in requirements documents. These reports indicate that faults often derive from instances of ambiguous, incorrect or otherwise deficient language. In this paper, we report on a method combining elements of linguistic theory and information retrieval to guide the discovery of problematic phrases throughout a requirements specification, using defect reports and correction requests generated during testing to seed our detection process. We found that phrases known from these materials to be problematic have occurrence properties in requirements documents that both allow the direction of resources to prioritize their correction, and generate insights characterizing more general locations of difficulty within the requirements. Our findings lead to some recommendations for more efficiently and effectively managing certain natural language issues in the creation and maintenance of requirements specifications.

A Case Study in Systematic Improvement of Language for Requirements

“…It is the only work to our knowledge that systematically addresses the psychological mechanics of communication and builds a method around the awareness of those mechanics such that the incidence of miscommunication is reduced by design. We have previously investigated the use of testing and defect data to indicate locations of linguistic deficiency in requirements [16], as well as analyzed the language of an accident reporting standard for deficiencies that threaten the integrity and comparability of resulting reports [6]. The activities reported on here replicate and extend the results of a formal experiment done previously using an earlier version of CLEAR [5].…”

“…High relative frequency indicates a high number of opportunities for misinterpretation. Similarly, high relative distribution indicates wide influence on multiple aspects of a system and thus the potential for misunderstandings to result in nonlocalized or uncontained faults that affect the system in pervasive ways [16]. In either case, we are interested in minimizing the risk of miscommunication of these phrases.…”

Examining the Relationships between Performance Requirements and “Not a Problem” Defect Reports

“…Wasson et al [21] use the failure reports created by testers to identify problematic phrases that occur in requirements documents. In our previous work [11], we demonstrate a process to improve performance requirements specifications from failure reports.…”

Improving Performance Requirements Specifications from Field Failure Reports

“…Wasson et al use the failure reports created by the testers to identify problematic phrases that are used in requirements documents [14]. In our procedure, the field failure reports from the users are used to specify more complete and specific performance requirements.…”