Automata-based Controller Synthesis for Stochastic Systems: A Game Framework via Approximate Probabilistic Relations

Abstract

In this work, we propose an abstraction and refinement methodology for the controller synthesis of discrete-time stochastic systems to enforce complex logical properties expressed by deterministic finite automata (a.k.a. DFA). Our proposed scheme is based on a notion of so-called ( , δ)-approximate probabilistic relations, allowing one to quantify the similarity between stochastic systems modeled by discrete-time stochastic games and their corresponding finite abstractions. Leveraging this type of relations, the lower bound for the probability of satisfying the desired specifications can be well ensured by refining controllers synthesized over abstract systems to the original games. Moreover, we propose an algorithmic procedure to construct such a relation for a particular class of nonlinear stochastic systems with slope restrictions on the nonlinearity. The proposed methods are demonstrated on a quadrotor example, and the results indicate that the desired lower bound for the probability of satisfaction is guaranteed.

Sandboxing (AI-based) Unverified Controllers in Stochastic Games: An Abstraction-based Approach with Safe-visor Architecture

“…Therefore, we have V ρ ,λ * (ρ ) H (x 0 , q0 ) ≤ η whenapplying the supervisor as in Definition 3.7. According to[27, Theorem 5.10], one has P Ω y ωH |= A ≤ η, which completes the proof.…”

“…Accordingly, the formal safety guarantee for the desired safety specification is provided based on an ( , δ)-approximate probabilistic relation between D and D. To establish such a relation, one can employ existing results (e.g. [26,Section 4], [27,Section 4]). Due to lack of space, we are not providing details of those results.…”

“…In general, the construction of a Safe-visor architecture includes offline computation of a safety advisor enforcing the desired safety specification, and online implementation of a supervisor that detects harmful inputs from the unverified controller and decides the input fed to the system accordingly (either the one from the unverified controller or the safety advisor). Here, we leverage the results in [27] for synthesizing the safety advisor. Due to space limitations, we omit their technical details here.…”

“…Definition 3.7). The computation of V * is presented as in Definition 3.5, which is adapted from [27]. Definition 3.5.…”

“…Additionally, we need the definitions of Markov policy, Product gDTSG, and a cost-to-go function, which are borrowed from [27].…”