Empirical research on requirements quality: a systematic mapping study

Abstract

Research has repeatedly shown that high-quality requirements are essential for the success of development projects. While the term “quality” is pervasive in the field of requirements engineering and while the body of research on requirements quality is large, there is no meta-study of the field that overviews and compares the concrete quality attributes addressed by the community. To fill this knowledge gap, we conducted a systematic mapping study of the scientific literature. We retrieved 6905 articles from six academic databases, which we filtered down to 105 relevant primary studies. The primary studies use empirical research to explicitly define, improve, or evaluate requirements quality. We found that empirical research on requirements quality focuses on improvement techniques, with very few primary studies addressing evidence-based definitions and evaluations of quality attributes. Among the 12 quality attributes identified, the most prominent in the field are ambiguity, completeness, consistency, and correctness. We identified 111 sub-types of quality attributes such as “template conformance” for consistency or “passive voice” for ambiguity. Ambiguity has the largest share of these sub-types. The artefacts being studied are mostly referred to in the broadest sense as “requirements”, while little research targets quality attributes in specific types of requirements such as use cases or user stories. Our findings highlight the need to conduct more empirically grounded research defining requirements quality, using more varied research methods, and addressing a more diverse set of requirements types.

A Live Extensible Ontology of Quality Factors for Textual Requirements

“…An empirical-to-conceptual approach was chosen for the subsequent four iterations, as we aim to extract eligible objects for the four taxonomies from established literature. For this prototype, we selected the set of primary studies gathered in a recent systematic mapping study on empirical requirements quality research by Montgomery et al [3] as our data to extract from. This publication is the only secondary study to our knowledge which explicitly investigates requirements quality and, thus, serves as a reliable collection of peer-reviewed primary studies.…”

“…The set of references in [3] is heavily biased towards empirical work. To confirm that the ontology is also robust when considering non-empirical work, we conducted a final iteration considering publications that were excluded in the reference selection phase of [3].…”

“…This satisfies the need for detecting potential defects in textual requirements at an early stage, as the cost for addressing these defects increases the longer they stay undetected, putting the project success at risk when treated poorly [2]. The applicability of quality factors is corroborated by the plethora of existing tools which automate their detection [3], [4]. Among the popular requirements quality factors are passive voice [4], where the use of a verb in passive voice is associated with ambiguity of a requirement due to the omission of the subject within a sentence, and sentence length [5], where exceeding a specific threshold of words or characters in a sentence is associated with complexity due to the sentence becoming increasingly hard to comprehend.…”

“…While a certain degree of similarity between all three competing descriptions is apparent, the lack of consensus on the definition is bound to introduce ambiguity to the understanding of the quality factor. Furthermore, another challenge of requirements quality research is the proposal of shallow quality factors neglecting practical relevance due to insufficient or anecdotal evidence [3].…”

“…Next, we formalize this domain into an ontology where each element is represented by an individual taxonomy initially derived from literature. Using a set of 105 primary studies from the area of empirical research on requirements quality [3], we rigorously improve the structure of the ontology by applying established guidelines [8] and extracted eligible objects to populate the ontology. Finally, the refined structure of the ontology as well as all extracted objects are stored centrally in a repository and visualized through a connected web interface.…”