A systematic literature review of empirical research on quality requirements

Abstract

Quality requirements deal with how well a product should perform the intended functionality, such as start-up time and learnability. Researchers argue they are important and at the same time studies indicate there are deficiencies in practice. Our goal is to review the state of evidence for quality requirements. We want to understand the empirical research on quality requirements topics as well as evaluations of quality requirements solutions. We used a hybrid method for our systematic literature review. We defined a start set based on two literature reviews combined with a keyword-based search from selected publication venues. We snowballed based on the start set. We screened 530 papers and included 84 papers in our review. Case study method is the most common (43), followed by surveys (15) and tests (13). We found no replication studies. The two most commonly studied themes are (1) differentiating characteristics of quality requirements compared to other types of requirements, (2) the importance and prevalence of quality requirements. Quality models, QUPER, and the NFR method are evaluated in several studies, with positive indications. Goal modeling is the only modeling approach evaluated. However, all studies are small scale and long-term costs and impact are not studied. We conclude that more research is needed as empirical research on quality requirements is not increasing at the same rate as software engineering research in general. We see a gap between research and practice. The solutions proposed are usually evaluated in an academic context and surveys on quality requirements in industry indicate unsystematic handling of quality requirements.

The Business Model of Digital Platforms for the Sharing Economy: Intensive Case Study Methodology for Rover.com Pet Boarding Platform

“…Yin [29], Douce [31], and Olsson et al [32] and Garcia Portilla [33] proposed that the case study method is a kind of empirical research. Among the five qualitative research methods such as experiments, investigations, document analysis, history, and case strategies, three research strategies can be based on forms of research questions: whether it needs to be manipulated in behavior and events, and whether the research focus is on the current problem to measure.…”

“…Case studies also do not need to manipulate variables in behavior and events, and focus on the events at the time, which is more flexible. According to Yin [34], Douce [31], and Olsson et al [32] and Garcia Portilla [33]'s suitability of case study strategy, the case study method does not need to control variables in behavioral events, and the research focus is on the current problem, only exploring how and why the event happened, so it is more flexible. The sharing economy is currently regarded as a new type of economy, and the economic model needs to be expanded through digital platforms.…”

Towards a Modern Quality Framework

“…The latter two are also called non-functional requirements. Differentiating between functional and quality requirements as well as ways how to elicit, structure and represent quality requirements have been a widely discussed topic in the requirements engineering community for many years [5], [7], [8], [9].…”

“…So we decided to base our new quality model on (1) the popular existing frameworks, and (2) our insights about the problems of the existing frameworks. As a sanity check, we looked at the quality characteristics identified in two systematic literature reviews [8], [16].…”

Specifying Requirements for Modern Software Development: A Test-Oriented Methodology

“…There are several proposals, e.g. the NFR method, QUPER, quality models and i*, which consider QR but as a separate process from FR specification [12]. A common practice in agile development is the design of test cases in early phases, which then guide software development [6].…”