Correct Passive Testing Algorithms and Complete Fault Coverage

Abstract

Abstract. The aim of passive testing is to detect faults in a system while observing the system during normal operation, that is, without forcing the system to specialized inputs explicitly for the purposes of testing. We formulate some general correctness requirements on any passive-testing algorithm which we term soundness and completeness. With respect to these definitions, we show that the homing algorithm, first proposed in [4], and subsequently used in [6][7][8], is sound and complete for passively testing the conformance of an implementation for several distinct conformance notions ranging from trace-containment to observational equivalence to even exact identity. This implies that, for some notions of conformance, there are faulty implementations that would not be detectable by any sound passive testing algorithm. We define a property to be passively testable as one admitting complete fault coverage under passive testing, i.e., one for which any faulty execution can be detected through passive testing. We provide an exact characterization of passively testable properties as being a natural subclass of safety properties, namely, those that are trace-contained in sets that are prefix-and suffix-closed. For such properties, we derive efficient complete passive testing algorithms that take constant time. We demonstrate the applicability of these results to networks and network devices by considering the problem of passively testing an implementation for conformance to the TCP protocol.

Passive mid-stream monitoring of real-time properties

“…In this paper, we present results on passively testing for real-time behavioral properties that can be applied to a large class of systems including those that can be modeled as timed automata. Our results provide a natural extension of the passive testing study conducted in [17] for untimed properties. We have implemented our approach using the real-time model checker UPPAAL, and we report on its application to passively test fault tolerance software in a telecommunications switch developed at Lucent Technologies.…”

“…Our adoption of interval-timed traces facilitates a technically appealing development of our results in that they form a natural extension of those for the untimed setting described in [17]. Over interval-timed traces, we can define the notions of timed concatenation, timed prefix, and timed suffix; the passive testing results in this paper then generalize those in the untimed setting [17] by replacing the roles of concatenation, prefix and suffix (applicable to untimed traces) by their timed counterparts (over intervaltimed traces).…”

“…Over interval-timed traces, we can define the notions of timed concatenation, timed prefix, and timed suffix; the passive testing results in this paper then generalize those in the untimed setting [17] by replacing the roles of concatenation, prefix and suffix (applicable to untimed traces) by their timed counterparts (over intervaltimed traces).…”

RiskStructures : A design algebra for risk-aware machines

“…This situation requires further state estimation (e.g. a homing algorithm [NSV03]) to disclose further information about the tracked obstacle. We also discuss this below in Sect.…”

“…Following up on Sect. 4.4, if we can show that a risk structure is an Input/Output LTS and it can be transformed to a non-deterministic finite Mealy machine, the state estimation problem can be solved, for example, by homing algorithms used for passive testing[NSV03].Partially specified risk factors (e.g. because of partial activation or mitigation analysis according to Tables…”