Search-based Similarity-driven Behavioural SPL Testing

Abstract

International audienceDissimilar test cases have been proven to be effective to reveal faults in software systems. In the Software Product Line (SPL) context, this criterion has been applied successfully to mimic combinatorial interaction testing in an efficient and scalable manner by selecting and prioritising most dissimilar configurations of feature models using evolutionary algorithms. In this paper, we extend dissimilarity to behavioural SPL models (FTS) in a search-based approach, and evaluate its effectiveness in terms of product and fault coverage. We investigate different distances as well as as single-objective algorithms, (dissimilarity on actions, random , all-actions). Our results on four case studies show the relevance of dissimilarity-based test generation for be-havioural SPL models, especially on the largest case-study where no other approach can match it

Search‐based crash reproduction using behavioural model seeding

“…From the inferred models, we extract a set of dissimilar (based on the Jaccard distance ) abstract object behaviours (➃ in Figure ). For abstract object behaviour extraction, we use the VIBeS model‐based testing tool.…”

Many-Objective Test Suite Generation for Software Product Lines

“…Another (complementary) approach that has been used is to separately consider test selection and prioritisation (see, for example, [1,19,29,31,47,48,55,58,61]). These approaches can decrease the difficulty of the considered problem from many objectives to multiple objectives.…”

“…Thus, the problem is to find a test suite (sequence of valid products) that is likely to be a good solution (in terms of selection and prioritisation). It has been observed that a number of properties of good test suites can be captured by objective functions that map a test suite to a value that represents how 'good' this test suite is according to the properties (see, for example [19,23,29,31,32,48,51,58,61,71,72,74]). For example, a fault might be associated with the interaction of a pair of features and so we might want to test as many such interactions as possible (pairwise coverage).…”

“…Since a number of objectives/fitness functions have been identified, the problem of finding a good solution has been expressed as a multi-objective optimisation problem [19,23,24,31,47,48,55,58,71,72,74]. Previous work developed test generation approaches based on evolutionary multi-objective algorithms and found these to be effective [19,23,31,48,58,71,72,74]; a recent survey was provided by Herrejon et al [49].…”

“…An alternative approach would be to weight the different objectives, in order to turn a multiobjective optimisation problem into a single-objective optimisation problem (see, for example, [19,31,71,74]). However, the multi-objective approach has a number of advantages.…”