A systematic review of evaluation of variability management approaches in software product lines

Abstract

a b s t r a c tContext: Variability management (VM) is one of the most important activities of software product-line engineering (SPLE), which intends to develop software-intensive systems using platforms and mass customization. VM encompasses the activities of eliciting and representing variability in software artefacts, establishing and managing dependencies among different variabilities, and supporting the exploitation of the variabilities for building and evolving a family of software systems. Software product line (SPL) community has allocated huge amount of effort to develop various approaches to dealing with variability related challenges during the last two decade. Several dozens of VM approaches have been reported. However, there has been no systematic effort to study how the reported VM approaches have been evaluated. Objective: The objectives of this research are to review the status of evaluation of reported VM approaches and to synthesize the available evidence about the effects of the reported approaches. Method: We carried out a systematic literature review of the VM approaches in SPLE reported from 1990s until December 2007. Results: We selected 97 papers according to our inclusion and exclusion criteria. The selected papers appeared in 56 publication venues. We found that only a small number of the reviewed approaches had been evaluated using rigorous scientific methods. A detailed investigation of the reviewed studies employing empirical research methods revealed significant quality deficiencies in various aspects of the used quality assessment criteria. The synthesis of the available evidence showed that all studies, except one, reported only positive effects. Conclusion: The findings from this systematic review show that a large majority of the reported VM approaches have not been sufficiently evaluated using scientifically rigorous methods. The available evidence is sparse and the quality of the presented evidence is quite low. The findings highlight the areas in need of improvement, i.e., rigorous evaluation of VM approaches. However, the reported evidence is quite consistent across different studies. That means the proposed approaches may be very beneficial when they are applied properly in appropriate situations. Hence, it can be concluded that further investigations need to pay more attention to the contexts under which different approaches can be more beneficial.

Variability Management in Software Product Lines: An Investigation of Contemporary Industrial Challenges

“…Specifically, by research output we refer to the VM approaches proposed in the literature. Some of these approaches have been systematically reviewed from different angles and purposes and reported in our different publications [2,5,[7][8]. The focus of the discussion in this section is to discuss the findings from the focus group in light of the findings from literature reviews we have already reported in order to highlight the issues that still remain unsolved according to the perceptions of the participants of our study.…”

“…To facilitate the discussion, we have clustered the issues into two groups based on the number of approaches reported in the literature [2,5,[7][8]. The two groups are shown in Table 4.…”

“…Especially, the findings of the study shed light on non-technical challenges (i.e., issues regarding people, mindset change, management support, organizational structure, and business model) faced by practitioners in their daily practice of SPL. These non-technical challenges appear to have been hardly addressed by existing VM approaches, which seem to be mainly focused on technical aspects of VM [2,5,[7][8].…”

Three Cases of Feature-Based Variability Modeling in Industry

“…The "Software Product Line Hall of Fame" [1], a catalog of SPLE case studies [37], and a practice-oriented book [28] contain information on adoption practices, organizational structures, and architectures, but offer little insight into the use of variability models, their sizes and contents, and the techniques used to build them. In fact, recent literature reviews on the evaluation of variability modeling lament the lack of empirical work on this topic [11,9,22,10]. Exceptions are industrial experience reports.…”

User interfaces for semantic authoring of textual content: A systematic literature review

“…The distribution of studies per year as shown in Figure 3 indicates an increasing intensity of research in the area of semantic content authoring. The remarkable rise after 2008 can be explained with the emergence and adoption of weak semantic techniques (the so-called 'lowercase' Semantic Web), such as the use of Microformats 12 , RDFa 13 and Microdata 14 . These techniques facilitate semantic content authoring by embedding semantic annotations into the HTML Web pages.…”