Strategies for product-line verification: Case studies and experiments

Abstract

Abstract-Product-line technology is increasingly used in mission-critical and safety-critical applications. Hence, researchers are developing verification approaches that follow different strategies to cope with the specific properties of product lines. While the research community is discussing the mutual strengths and weaknesses of the different strategies-mostly at a conceptual level-there is a lack of evidence in terms of case studies, tool implementations, and experiments. We have collected and prepared six product lines as subject systems for experimentation. Furthermore, we have developed a modelchecking tool chain for C-based and Java-based product lines, called SPLVERIFIER, which we use to compare sample-based and family-based strategies with regard to verification performance and the ability to find defects. Based on the experimental results and an analytical model, we revisit the discussion of the strengths and weaknesses of product-line-verification strategies.

Domain Types: Abstract-Domain Selection Based on Variable Usage

“…The area below the graph is proportional to the accumulated verification time. We also provide a supplementary web page 5 , where the detailed results of all verification runs (including the raw data and the log files) are available for download and as interactive plots.…”

“…The LOOPS benchmark set consists of verification tasks that require the analysis of loops with non-static loop bounds. The benchmark set PRODUCT LINES models three software product lines used in featureinteraction detection [5]. Domain Types.…”

“…For example, using an explicitvalue domain [14] was efficient on many benchmarks from the recent competition on software verification [9], while using a BDD domain [15] was more efficient on eventcondition-action (ECA) systems that involve only simple operations over integers in an ECA competition [30]. In the context of product-line verification, it has been shown that BDD-encodings of feature variables improve verification performance [5,24]. The key insight is that different abstract domains are successful on different programs, and for every abstract domain, we can find programs for which the abstract domain is not successful.…”

Facilitating Reuse in Multi-goal Test-Suite Generation for Software Product Lines

“…For a variability-aware generation of product-line test suites, we integrated a BDD-based feature-parameter analysis into CPA/tiger and added further code to determine which test goals of which variants are covered by a given test case. For our experiments, we selected two subject systems that are well-known from several benchmarks, e.g., in context of product-line verification and feature interaction detection [3].…”

“…There, the explicit-state model checker SPIN has been used for reachability analysis, with a posteriori reasoning about test-case reuse among product variants, but without any reuse of reachability results among test goals. Applying CPAchecker for product-line verification has been proposed [3], incorporating BDD analysis for reuse of verification results [2]. Reuse of reachability analysis results for different test goals [7,8] has been presented and implemented as CPA/tiger on top of CPAchecker and corresponding reuse concepts have been applied to intermediate verification results [10].…”