Test Selection Criteria for Quantifier-Free First-Order Specifications

Theoretical Aspects of Computing - ICTAC 2009

2009

Self Cite

Abstract. Testing from first-order specifications has mainly been studied for flat specifications, that are specifications of a single software module. However, the specifications of large software systems are generally built out of small specifications of individual modules, by enriching their union. The aim of integration testing is to test the composition of modules assuming that they have previously been verified, i.e. assuming their correctness. One of the main method for the selection of test cases from first-order specifications, called axiom unfolding, is based on a proof search for the different instances of the property to be tested, thus allowing the coverage of this property. The idea here is to use deduction modulo as a proof system for structured first-order specifications in the context of integration testing, so as to take advantage of the knowledge of the correctness of the individual modules.Testing is a very common practice in the software validation process. The principle of testing is to execute the software system on a subset of its possible inputs in order to detect failures. A failure is detected if the system behaves in a non-conformant way with respect to its specification.The testing process is usually decomposed into three phases: the selection of a relevant subset of the set of all the possible inputs of the system, called a test set; the submission of this test set to the system; the decision of the success or the failure of the test set submission, called the oracle problem. We focus here on the selection phase, which is the crucial point for the relevance and the efficiency of the testing process. In the approach called black-box testing, tests are selected from a (formal or informal) specification of the system, without any knowledge about the implementation.Our work follows the framework defined by Gaudel, Bernot and Marre [1], for testing from specifications expressed in a logical formalism. One approach to selection consists first in dividing an exhaustive test set into subsets, and then in choosing one test case in each of these subsets, thus building a finite test set which covers the initial exhaustive test set. One of the most studied selection method for testing from equational (and then first-order) specifications is known as axiom unfolding [1][2][3][4]. Its principle is to divide the initial exhaustive test set according to criteria derived from the axioms of the specification, using the well-known and efficient proof techniques associated to first-order logic.

Section: Deduction Modulomentioning

confidence: 99%

Integration Testing from Structured First-Order Specifications via Deduction Modulo

Longuet

Theoretical Aspects of Computing - ICTAC 2009

2009

Self Cite

“…In this section, we study the problem of test case selection for dynamic specications, by adapting a selection criteria based on axiom unfolding which has been widely and eciently applied in the algebraic specication setting [1,2,4,6,7].…”

Section: Test Of Attributes By Axiom Unfoldingmentioning

confidence: 99%

“…Hence, system correctness can be asymptotically reached by making an increasingly ne partition of the exhaustive set. This selection criterion leading to a partition of the exhaustive set has been mainly and extensively applied for specications dened as theories of equational logic [1,5,4,6,7,12] and more recently of quantier-free rst-order logic [2]. In these works, the partitioning of the exhaustive set under consideration is made in an algorithmic way by unfolding axioms.…”

Section: Introductionmentioning

confidence: 99%

“…In these works, the partitioning of the exhaustive set under consideration is made in an algorithmic way by unfolding axioms. This axiom unfolding makes a case analysis of test purposes dened as simple equations in [1] and quantier-free formulae in [2]. Test cases are then extracted from specications by building input data dened by ground equations or formulae, matching the dierent cases dened in the specication.…”

Section: Introductionmentioning

confidence: 99%

“…10 We recall that the data part of the system has been specied by classic algebraic specications. So, testing this part can be done by using selection methods dened in [1,2]. The input domain of an attribute f is the projection of the reference test set T (Sp) on f , that is, the subset of T (Sp) dealing with f .…”

mentioning

confidence: 99%

See 2 more Smart Citations

Test Selection Criteria for Modal Specifications of Reactive Systems

First Joint IEEE/IFIP Symposium on Theoretical Aspects of Software Engineering (TASE '07)

Longuet

2007

Self Cite

In the framework of functional testing from algebraic specications, the strategy of test selection which has been widely and eciently applied is based on axiom unfolding. In this paper, we propose to extend this selection strategy to a modal formalism used to specify dynamic and reactive systems. Such a work is then a rst step to tackle testing of such systems more abstractly than most of the works dealing with what is called conformance testing. We get a higher level of abstraction since our specications account for what is usually called underspecication, i.e. they do not denote a unique model but a class of models. Hence, the testing process can be applied at every design level.

Exhaustive test sets for algebraic specifications

Software Testing Verif & Rel

Arnould

Gall

et al. 2016

Self Cite

In the context of testing from algebraic specifications, test cases are ground formulas chosen amongst the ground semantic consequences of the specification, according to some possible additional observability conditions. A test set is said to be exhaustive if every programme P passing all the tests is correct and if for every incorrect programme P , there exists a test case on which P fails. Because correctness can be proved by testing on such a test set, it is an appropriate basis for the selection of a test set of practical size. The largest candidate test set is the set of observable consequences of the specification. However, depending on the nature of specifications and programmes, this set is not necessarily exhaustive. In this paper, we study conditions to ensure the exhaustiveness property of this set for several algebraic formalisms (equational, conditional positive, quantifier free and with quantifiers) and several test hypotheses. EXHAUSTIVE TEST SETS FOR ALGEBRAIC SPECIFICATIONS 295 decision procedure, hence are considered to be observable. The notion of observable contexts has been introduced to systematically observe non-observable sorts through successive applications of operations leading to a result of observable sort [16][17][18]. Therefore, equations of non-observable sort are converted into a family of equations built by surrounding terms that occur in the initial equation with the same observable context. This has been particularly used for object-oriented software testing where, by assumption, object states are encapsulated [7,14,19].Because test cases are defined up to observability issues, the notion of correctness is closely related to observability assumptions. Roughly speaking, correctness is reached when there is a (possibly infinite) test set that verifies that any correct (resp. incorrect) programme successfully (resp. unsuccessfully) passes the test. Because any programme that satisfies all test cases has to be considered correct, correctness is defined according to an observational approach similar to those used to define specification refinement [20-24]: a concrete specification is said to be a refinement of an abstract specification if any algebra of the concrete specification is observationally equivalent to an algebra of the abstract specification. Here, by analogy, we say that a programme P is correct with respect to a specification SP if it is equivalent to an algebra of SP, up to the observable formulas in Obs [1,9]. Let us point out that by default, the model class defined by SP is not restricted to a unique model (such as the initial one or the terminal one [25]). Thus, we follow a loose semantics approach, that is the model class associated to SP can contain several models, each of them likely to represent a different programme. The interest is to be able to accept programmes that possibly implement some extra functionalities or properties (contrarily to other approaches considering terminal semantics [14,19]).As usual, SP denotes the set of all semantic consequences of ...