Optimal strategies for testing nondeterministic systems

Abstract

This paper deals with testing of nondeterministic software systems. We assume that a model of the nondeterministic system is given by a directed graph with two kind of vertices: states and choice points. Choice points represent the nondeterministic behaviour of the implementation under test (IUT). Edges represent transitions. They have costs and probabilities. Test case generation in this setting amounts to generation of a game strategy. The two players are the testing tool (TT) and the IUT. The game explores the graph. The TT leads the IUT by selecting an edge at the state vertices. At the choice points the control goes to the IUT. A game strategy decides which edge should be taken by the TT in each state. This paper presents three novel algorithms 1) to determine an optimal strategy for the bounded reachability game, where optimality means maximizing the probability to reach any of the given final states from a given start state while at the same time minimizing the costs of traversal; 2) to determine a winning strategy for the bounded reachability game, which guarantees that given final vertices are reached, regardless how the IUT reacts; 3) to determine a fast converging edge covering strategy, which guarantees that the probability to cover all edges quickly converges to 1 if TT follows the strategy.

Adaptive Testing of Deterministic Implementations Specified by Nondeterministic FSMs

“…The method of [1] allows to test whether a given FSM is r-compatible with the specification FSM (but not that the former is a reduction of the latter, as we do), and [3] extends this result to a set of FSMs. The work in [5] also addresses adaptive testing of nondeterministic machines, but its results cannot be used to prove that a given DFSM is a reduction of the specification machine. Compared to [7], we pre-compute all the test fragments, simplifying the test generation process.…”

Model-based Testing of the Conference Protocol with Spec Explorer

“…Depending on the presence of observable actions in M and whether M is deterministic (i.e., the set of enabled actions in each state of M is a singleton set) or nondeterministic, different traversal algorithms have to be employed. In the general case, Spec Explorer incorporates a variant of Dijkstra's shortest path algorithm for alternating reachability and a nondeterministic extensions of the Chinese postman tour algorithm [12]. For generating tests that optimize the expected cost with respect to action probabilities, adapted algorithms from Markov decision process theory are employed [3].…”

Testing Systems of Concurrent Black-Boxes—An Automata-Theoretic and Decompositional Approach

“…However, there are testable units that are not necessarily output deterministic (e.g., [24,32,27]). Therefore, to make our algorithms (presented later) more general, we do not apply this assumption (under which, obviously, our algorithm still applies).…”