Pairwise testing for software product lines: comparison of two approaches

Perrouin, Gilles; Oster, Sebastian; Sen, Sagar; Klein, Jacques; Baudry, Benoît; Traon, Yves Le

doi:10.1007/s11219-011-9160-9

Cited by 95 publications

(90 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Just to mention a few, there are works using multi-objective approaches [56,34,47,29,30] for test selection, while some others explore t-wise testing [24,49,50,48]. Although out of the scope of this work, FLAME could also be used to formalize these new operations.…”

Section: Related Workmentioning

confidence: 99%

FLAME: a formal framework for the automated analysis of software product lines validated by automated specification testing

et al. 2015

View full text Add to dashboard Cite

In a literature review on the last 20 years of automated analysis of feature models, the formalization of analysis operations was identified as the most relevant challenge in the field. This formalization could provide very valuable assets for tool developers such as a precise definition of the analysis operations and, what is more, a reference implementation, i.e. a trustworthy, not necessarily efficient implementation to compare different tools outputs. In this article, we present the FLAME framework as the result of facing this challenge. FLAME is a formal framework that can be used to formally specify not only feature models, but other variability modeling languages (VMLs) as well. This reusability is achieved by its two-layered architecture. The abstract foundation layer is the bottom layer in which all VMLindependent analysis operations and concepts are specified. On top of the foundation layer, a family of characteristic model layers-one for each VML to be formally specifiedcan be developed by redefining some abstract types and relations. The verification and validation of FLAME has fol- lowed a process in which formal verification has been performed traditionally by manual theorem proving, but validation has been performed by integrating our experience on metamorphic testing of variability analysis tools, something that has shown to be much more effective than manuallydesigned test cases. To follow this automated, test-based validation approach, the specification of FLAME, written in Z, was translated into Prolog and 20,000 random tests were automatically generated and executed. Tests results helped to discover some inconsistencies not only in the formal specification, but also in the previous informal definitions of the analysis operations and in current analysis tools. After this process, the Prolog implementation of FLAME is being used as a reference implementation for some tool developers, some analysis operations have been formally specified for the first time with more generic semantics, and more VMLs are being formally specified using FLAME.

show abstract

Section: Related Workmentioning

confidence: 99%

FLAME: a formal framework for the automated analysis of software product lines validated by automated specification testing

et al. 2015

View full text Add to dashboard Cite

show abstract

“…Moreover, the tasks in the two invariants are not required to be different in the generated model, hence we obtain a model with two tasks instead of three. In the MDE community, pairwise testing is being successfully used for software product line testing [39,40], considering pairs of features in a feature model. In our case, there are additional challenges, because our specifications do not explicitly encode dependencies between their requirements, and the model generation procedure has to consider the constraints given by the input meta-model and preconditions as well.…”

Section: Coverage Criteria For Input Model Generationmentioning

confidence: 99%

Specification-driven model transformation testing

Guerra

Soeken

2013

Softw Syst Model

View full text Add to dashboard Cite

Testing model transformations poses several challenges, among them the automatic generation of appropriate input test models and the specification of oracle functions. Most approaches to the generation of input models ensure a certain coverage of the source meta-model or the transformation implementation code, whereas oracle functions are frequently defined using query or graph languages. However, these two tasks are usually performed independently regardless of their common purpose, and sometimes there is a gap between the properties exhibited by the generated input models and those considered by the transformations.Recently, we proposed a formal specification language for the declarative formulation of transformation properties (by means of invariants, pre-and postconditions) from which we generated partial oracle functions used for transformation testing. Here, we extend the usage of our specification language for the automated generation of input test models by SAT solving. The testing process becomes more intentional because the generated models ensure a certain coverage of the transformation requirements. Moreover, we use the same specification to consistently derive both the input test models and the oracle functions. A set of experiments is presented, aimed at measuring the efficacy of our technique.

show abstract

“…In the MDE community, pairwise testing is being successfully used for software product line testing [19,20], considering pairs of features in a feature model. In our case there are additional challenges, because our specifications do not explicitly encode dependencies between their requirements, and the model generation procedure has to consider the constraints given by the input meta-model and preconditions.…”

Section: Input Model Generation For Different Coverage Criteriamentioning

confidence: 99%

Specification-Driven Test Generation for Model Transformations

Guerra

2012

Theory and Practice of Model Transformations

View full text Add to dashboard Cite

Esta es la versión de autor de la comunicación de congreso publicada en: This is an author produced version of a paper published in: El acceso a la versión del editor puede requerir la suscripción del recurso Access to the published version may require subscription Specification-driven test generation for model transformationsEsther GuerraUniversidad Autónoma de Madrid (Spain) Esther.Guerra@uam.es Abstract. Testing model transformations poses several challenges, among them the automatic generation of appropriate input test models and the specification of oracle functions. Most approaches to the generation of input models ensure a certain level of source meta-model coverage, whereas the oracle functions are frequently defined using query or graph languages. Both tasks are usually performed independently regardless their common purpose, and sometimes there is a gap between the properties exhibited by the generated input models and those demanded to the transformations (as given by the oracles).Recently, we proposed a formal specification language for the declarative formulation of transformation properties (invariants, pre-and postconditions) from which we generated partial oracle functions that facilitate testing of the transformations. Here we extend the usage of our specification language for the automated generation of input test models by constraint solving. The testing process becomes more intentional because the generated models ensure a certain coverage of the interesting properties of the transformation. Moreover, we use the same specification to consistently derive both the input test models and the oracle functions.

show abstract

Pairwise testing for software product lines: comparison of two approaches

Cited by 95 publications

References 48 publications

FLAME: a formal framework for the automated analysis of software product lines validated by automated specification testing

FLAME: a formal framework for the automated analysis of software product lines validated by automated specification testing

Specification-driven model transformation testing

Specification-Driven Test Generation for Model Transformations

Contact Info

Product

Resources

About