Implementation relations and test generation for systems with distributed interfaces

Abstract

This is the Pre-print version of the Article. The official published version can be accessed from the link below - Copyright @ 2011 Springer-VerlagSome systems interact with their environment at physically distributed interfaces called ports and we separately observe sequences of inputs and outputs at each port. As a result we cannot reconstruct the global sequence that occurred and this reduces our ability to distinguish different systems in testing or in use. In this paper we explore notions of conformance for an input output transition system that has multiple ports, adapting the widely used ioco implementation relation to this situation. We consider two different scenarios. In the first scenario the agents at the different ports are entirely independent. Alternatively, it may be feasible for some external agent to receive information from more than one of the agents at the ports of the system, these local behaviours potentially being brought together and here we require a stronger implementation relation. We define implementation relations for these scenarios and prove that in the case of a single-port system the new implementation relations are equivalent to ioco. In addition, we define what it means for a test case to be controllable and give an algorithm that decides whether this condition holds. We give a test generation algorithm to produce sound and complete test suites. Finally, we study two implementation relations to deal with partially specified systems.This work was supported in part by Leverhulme Trust grant number F/00275/D,\ud Natural Sciences and Engineering Research Council (NSERC) of Canada grant number OGP00000976, Testing State Based Systems, and Engineering and Physical Sciences Research Council grant number GR/R43150, Formal Methods and Testing (FORTEST)

A survey on formal active and passive testing with applications to the cloud

“…Although the special issue targeted the broad scope of testing distributed and networked systems, where formal testing is well established [48,36,13,63,75,31,32], cloud computing represents the natural evolution of these systems and work on testing is in their initial stages. First, we would like to mention the difference between two apparently similar concepts: testing in the cloud and testing the cloud.…”

A More Precise Implementation Relation for Distributed Testing

“…Given port p ∈ P, the set of observations that can be made at p is Act p = I p ∪ O p ∪ {δ}. Figure 1 gives a previously defined IOTS M 0 that represents a distributed majority voting system [16]. The model interacts with two agents that we call U and L. We included the initial state s 0 twice to simplify the diagram.…”

“…In MBT, automation is based on a model, with the model often being statebased (see, for example, [1,2]). There has been particular interest in testing from a model in the form of a finite state machine (FSM) [3,4,5,6,7,8,9,10,11,12,13] or an input output transition system (IOTS) [14,15,16,17,18]; while the developer might produce a model in some other formalism, MBT tools often analyse an FSM or IOTS that represents the semantics of the model. This paper concerns testing from an IOTS model.…”

“…It is important to use a suitable implementation relation since otherwise testing might be unsound (it might declare a correct SUT to be faulty) and might also be inefficient (we might use test cases that cannot lead to faults being found). The implementation relation dioco has been defined for distributed testing from an IOTS, with this initially requiring that processes are not output-divergent 1 [20], and then being generalised to allow output-divergent processes [16]. The initial definition compared the quiescent traces 2 of the SUT with the quiescent traces of the specification using an equivalence relation ∼.…”

“…If a process is output-divergent then it may have a trace σ that is not a prefix of a quiescent trace: the initial definition of dioco then has the problem that it would effectively ignore σ. The generalisation [16] instead compared a class of infinite traces of the SUT with infinite traces of the specification.…”