A systematic review of quality attributes and measures for software product lines

Abstract

It is widely accepted that software measures provide an appropriate mechanism for understanding, monitoring, controlling and predicting the quality of software development projects. In Software Product Lines (SPL), quality is even more important than in a single software product since, owing to systematic reuse, a fault or an inadequate design decision could be propagated to several products in the family. Over the last few years, a great number of quality attributes and measures for assessing the quality of SPL have been reported in literature. However, no studies summarizing the current knowledge about them exist. This paper presents a systematic literature review with the objective of identifying and interpreting all the available studies from 1996 to date that present quality attributes and/or measures for SPL. These attributes and measures have been classified using a set of criteria that includes the life cycle phase in which the measures are applied; the corresponding quality characteristics; their support for specific SPL characteristics (e.g., variability, compositionality); the procedure used to validate the measures, etc. We found 165 measures related to 98 different quality attributes. The results of the review indicated that 92% of the measures evaluate attributes that are related to maintainability. In addition, 67% of the measures are used during the design phase of Domain Engineering, and 56% are applied to evaluate the product line architecture. However, only 25% of them have been empirically validated.In conclusion, the results provide a global vision of the state of the research within this area in order to help researchers in detecting weaknesses, directing research efforts, and identifying new research lines. In particular, there is a need for new measures with which to evaluate both the quality of the artifacts produced during the entire SPL life cycle and other quality characteristics. There is also a need for more validation (both theoretical and empirical) of existing measures. In addition, our results may be useful as a reference guide for practitioners to assist them in the selection or the adaptation of existing measures for evaluating their software product lines.

An Evaluation of the Challenges of Multilingualism in Data Warehouse Development

“…The adopted approach for data extraction was based on the work of Montagud et al (Montagud et al, 2012). For each research question was defined some values that could be the answers presented by the papers.…”

Towards guidelines for building a business case and gathering evidence of software reference architectures in industry

“…The development of a family of software systems has some characteristics that distinguish it from the development of software systems (Montagud et al 2011). Therefore, existing evidences for product line architectures can be also used to evaluate RAs, namely: generic characteristics such as "variability, reusability, commonality, and compositionality" (Montagud et al 2011); the propagation of architectural decisions while reusing common assets (Montagud et al 2011); and lower development costs with respect to developing systems individually (Ali et al 2009;Linden et al 2004;Montagud et al 2011).…”

“…Due to these reasons, the authors of the paper and two more software architects from everis elaborated further this analysis considering the specific characteristics of RAs as described in (Angelov et al 2008(Angelov et al , 2012Galster and Avgeriou 2011;Graaf et al 2005;Nakagawa et al 2011), commonalities with other architecture-centric approaches such as product line architectures (Ali et al 2009;Deelstra et al 2005;Linden et al 2004;Montagud et al 2011), and experience from everis. The resulting aspects for understanding and evaluating RAs are detailed below and summarized in Table 2.…”

“…For instance, variability shows how well an RA could be expanded or modified to produce new SAs of applications. Besides, an RA could address more architectural qualities than an SA (e.g., variability, reusability, commonality, compositionality, and applicability) (Angelov et al 2008;Montagud et al 2011). Quality attributes analysis should be wider for RAs in this sense.…”

“…Many prominent researchers (Angelov et al 2008;Falessi et al 2010) highlight the importance of architectural decisions for the SA design process and the architectural evaluation. For RAs, architectural decisions are even more important than in a single software system since, owing to systematic reuse, an inadequate design decision could be propagated to several software systems (Montagud et al 2011).…”