Efficient Distributed Test Architectures for Large-Scale Systems

Abstract

Abstract. Typical testing architectures for distributed software rely on a centralized test controller, which decomposes test cases in steps and deploy them across distributed testers. The controller also guarantees the correct execution of test steps through synchronization messages. These architectures are not scalable while testing large-scale distributed systems due to the cost of synchronization management, which may increase the cost of a test and even prevent its execution. This paper presents a distributed architecture to synchronize the test execution sequence. This approach organizes the testers in a tree, where messages are exchanged among parents and children. The experimental evaluation shows that the synchronization management overhead can be reduced by several orders of magnitude. We conclude that testing architectures should scale up along with the distributed system under test.

BDTest, a System to Test Big Data Frameworks

“…PeerUnit creates complex unit tests for distributed (peer-to-peer) systems without modifying the SUT. Test scenarios in PeerUnit are synchronised over all the computing nodes using either a centralised or a decentralised architecture [16] -PeerUnit can work using both modes. BDTest follows some of the general ideas proposed by PeerUnit, but we extend PeerUnit in two original directions: (i) BDtest can manage parallel executions (typical of Big Data tasks) and (ii) BDTest can run heterogeneous nodes.…”

Distinguishing Sequences for Distributed Testing: Adaptive Distinguishing Sequences

“…If there is no global clock and the local testers do not synchronise during testing then we are testing in the ISO standardised distributed test architecture [17]. Sometimes, we allow the testers to exchange coordination messages through a network in order to synchronise their actions (see, for example, [32][33][34][35]). However, this can make testing more expensive, since it requires us to establish a network to connect the local testers, and may not be feasible where there are timing constraints.…”

Generating Complete Controllable Test Suites for Distributed Testing

“…Thus, the traces of χ max (M ) that are not traces of M are exactly those that an SUT might have despite passing all controllable test cases. It is thus possible to reason about the effectiveness of controllable testing on the basis of χ max (M ) and determine whether It is sometimes possible to synchronise testers through the exchange of coordination messages [21], [30], [36]; it is then possible to add messages that overcome controllability problems. For example, if x i is supplied by the tester at p and then x i+1 is to be supplied by the tester at q = p then a corresponding controllability problem can be resolved by the tester at p sending a message to the tester at q after it supplies x i .…”