Probabilistic Temporal Logic Falsification of Cyber-Physical Systems

Abstract

We present a Monte-Carlo optimization technique for finding system behaviors that falsify a Metric Temporal Logic (MTL) property. Our approach performs a random walk over the space of system inputs guided by a robustness metric defined by the MTL property. Robustness is guiding the search for a falsifying behavior by exploring trajectories with smaller robustness values. The resulting testing framework can be applied to a wide class of Cyber-Physical Systems (CPS). We show through experiments on complex system models that using our framework can help automatically falsify properties with more consistency as compared to other means such as uniform sampling.

Automated testing of hybrid Simulink/Stateflow controllers: industrial case studies

“…We follow the translation of the temporal logic formulas into quantitative functions provided by [1] for this purpose. Below, we have shown the quantitative function obtained from ϕ 1 following the translation of [1]:…”

“…These tools would be useful in practice if they can effectively deal with the large input spaces of controllers. Specifically, they should be able (1) to visualize the behaviors of controllers over their input spaces, and (2) to find individual test scenarios that violate or are close to violating controller requirements. In our earlier work, we presented automated testing techniques and tools with mechanisms for input space exploration and exploitation [4,6].…”

A trajectory splicing approach to concretizing counterexamples for hybrid systems

“…Nevertheless, we find that such very low cost trajectories are almost always confirmed as real violations through simulation. The robustness-guided falsification proposed by Fainekos et al associates each trajectory with a robustness metric that measures how close a given trajectory is to a violation and uses robustness as a cost function in a global optimization framework [2]. Robustness metrics for trajectories have been defined for Metric Temporal Logic (MTL) by Fainekos et al [20] and for Signal Temporal Logic (STL) by Donze et al [19].…”

Querying Parametric Temporal Logic Properties on Embedded Systems

“…Further details on the necessity and implications of the aforementioned assumptions can be found in [12]. Under Assumption 3, a system Σ can be viewed as a function Δ Σ : X 0 × U → Y N × T which takes as an input an initial condition x 0 ∈ X 0 and an input signal u ∈ U and it produces as output a signal y : N → Y (also referred to as trajectory) and a timing function τ : N → R + .…”

“…Since no other technique can solve the parameter estimation problem for MTL formulas over hybrid systems, we compare our method with the falsification methods that we have developed in the past [12,7]. A detailed description of the benchmark problems can be found in [12,7] and the benchmarks can be downloaded with the S-TaLiRo distribution 2 . In order to be able to compare the two methods, when performing parameter estimation, we regard a parameter value less than the constant in the MTL formula as falsification.…”