A temporal approach for testing distributed systems

Abstract

This paper deals with testing distributed software systems. In the past, two important problems have been determined for executing tests using a distributed test architecture: controllability and observability problems. A coordinated test method has subsequently been proposed to solve these two problems. In the present article: 1) we show that controllability and observability are indeed resolved if and only if the test system respects some timing constraints, even when the system under test is non-real-time; 2) we determine these timing constraints; 3) we determine other timing constraints which optimize the duration of test execution; 4) we show that the communication medium used by the test system has not to be necessarily FIFO; and 5) we show that the centralized test method can be considered just as a particular case of the proposed coordinated test method.

Using status messages in the distributed test architecture

“…This avoids issues such as one message overtaking another and corresponds to the notion of a slow environment (see, for example, [23]). See [19] for a discussion of the issues introduced when the time taken to send a message is not negligible. We also assume that the status messages have been implemented correctly.…”

“…However, the introduction of such a network can increase the cost of testing. Further, the exchange of messages through the external network introduces delays and these may be problematic if there are real-time constraints [19]. While there are test sequence generation techniques that do not require such an external network, these place restriction on the SUT and so are not generally applicable (see, for example, [5,7,16,17,27,29,31]).…”

“…Note that previous work such as [4,19,26] resolves the controllability and observability problems through the exchange of messages on an external network while we introduce a method that does not require such an external network but does require there to be correctly implemented status messages. This paper is structured as follows.…”

Overcoming controllability problems with fewest channels between testers

“…Previous work on testing in the distributed test architecture has focussed on testing from an FSM since FSMs are suitable for modelling state-based systems and can be easily adapted to the case where there are multiple ports (see, for example, [6,7,[13][14][15]). However, some of the results simply concern the problem of applying a given test sequence and thus do not depend on the use of an FSM.…”

Overcoming observability problems in distributed test architectures

“…The timing issues can be particularly problematic if the SUT responds rapidly to inputs, relative to the network used for coordination messages 1 . See [4] for a discussion of some of the timing issues that arise in using coordination messages. This paper investigates conditions that must be satisfied by an FSM for the existence of input sequences that can be applied in a distributed test architecture without encountering controllability and observability problems and without using external coordination messages.…”