Explaining Safety Violations in Real-Time Systems

Abstract

We tackle the problem of explaining faults in real-time systems. Intuitively, an explanation of the violation of a safety property by an execution is a concise excerpt of the faulty execution that retains only the elements that were relevant for entailing the violation, thus exhibiting how causes accumulate over time and propagate to entail the effect. Fault explanation therefore goes beyond the well-known concepts of fault diagnosis and localization. We provide a formal definition of causal explanations on dense-time models, based on the wellstudied formalisms of timed automata and zone-based abstractions. Our approach is able to account for limited observability of the faulty execution. We propose a symbolic formalization to effectively construct such explanations, which we have implemented in a prototype tool. We illustrate our approach on several examples.

Conflict-Based Search for Explainable Multi-Agent Path Finding

“…In those works, various forms of explanations are explored, but visual explanations seem to be dominant for their ease of interpretability, especially for classifiers (Lapuschkin et al 2019). In other cases, such as fault detection algorithms, explanations are typically in the form of witness executions (Mari, Dang, and Gössler 2021). In the planning community, explanations are mostly studied in the context of single agent problem, and they often take a non-visual form.…”