Exhaustive test sets for algebraic specifications

Abstract

In the context of testing from algebraic specifications, test cases are ground formulas chosen amongst the ground semantic consequences of the specification, according to some possible additional observability conditions. A test set is said to be exhaustive if every programme P passing all the tests is correct and if for every incorrect programme P , there exists a test case on which P fails. Because correctness can be proved by testing on such a test set, it is an appropriate basis for the selection of a test set of practical size. The largest candidate test set is the set of observable consequences of the specification. However, depending on the nature of specifications and programmes, this set is not necessarily exhaustive. In this paper, we study conditions to ensure the exhaustiveness property of this set for several algebraic formalisms (equational, conditional positive, quantifier free and with quantifiers) and several test hypotheses. EXHAUSTIVE TEST SETS FOR ALGEBRAIC SPECIFICATIONS 295 decision procedure, hence are considered to be observable. The notion of observable contexts has been introduced to systematically observe non-observable sorts through successive applications of operations leading to a result of observable sort [16][17][18]. Therefore, equations of non-observable sort are converted into a family of equations built by surrounding terms that occur in the initial equation with the same observable context. This has been particularly used for object-oriented software testing where, by assumption, object states are encapsulated [7,14,19].Because test cases are defined up to observability issues, the notion of correctness is closely related to observability assumptions. Roughly speaking, correctness is reached when there is a (possibly infinite) test set that verifies that any correct (resp. incorrect) programme successfully (resp. unsuccessfully) passes the test. Because any programme that satisfies all test cases has to be considered correct, correctness is defined according to an observational approach similar to those used to define specification refinement [20-24]: a concrete specification is said to be a refinement of an abstract specification if any algebra of the concrete specification is observationally equivalent to an algebra of the abstract specification. Here, by analogy, we say that a programme P is correct with respect to a specification SP if it is equivalent to an algebra of SP, up to the observable formulas in Obs [1,9]. Let us point out that by default, the model class defined by SP is not restricted to a unique model (such as the initial one or the terminal one [25]). Thus, we follow a loose semantics approach, that is the model class associated to SP can contain several models, each of them likely to represent a different programme. The interest is to be able to accept programmes that possibly implement some extra functionalities or properties (contrarily to other approaches considering terminal semantics [14,19]).As usual, SP denotes the set of all semantic consequences of ...

A Theory of Black-Box Tests

“…Functional requirements generally correspond to obligations, which are irrefutable. 3 Yet, functional testing is common practice [39,36]. This seeming discrepancy can be resolved by noticing that the practice of functional testing is based upon (implicit) auxiliary assumptions, as the following example illustrates.…”

“…Our definition of refutability is inspired by Popper's notion of testable theories [41]. Theories of black-box testing proposed in the software engineering literature are largely concerned with the notions of test selection, test adequacy, and exhaustiveness; see, e.g., [23,53,20,48,3]. Refutable requirements have not been investigated in prior work, except for temporal properties and hyper-properties, which we discussed in §7.2 and §8.2.…”