Robust Dynamic Programming for Temporal Logic Control of Stochastic Systems

Haesaert, Sofie; Soudjani, Sadegh

doi:10.1109/tac.2020.3010490

Cited by 39 publications

(82 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proof can be found in the longer version of this paper [22]. Computation of the reachability probability has been studied extensively in the literature for both infinite horizon [9,[32][33][34] and finite horizon [13, 14, 18-20, 28-30, 35] using different abstract models and computational methods. These approaches can be used to provide a lower bound on the probability of satisfaction of the Büchi condition.…”

Section: Problem Definitionmentioning

confidence: 99%

“…While synthesis for reachability and safety properties have been studied in this setting both for infinite horizon [9,34] and for finite horizon [13, 14, 18-20, 27, 35], there are few techniques for synthesis against Büchi specifications, which requires the trajectory to visit a given set of states infinitely often. In this paper, we consider the problem of synthesizing controllers for controlled Markov processes for properties specified as Büchi conditions.…”

Section: Introductionmentioning

confidence: 99%

“…Moreover, we show that such a decomposition is able to provide a lower bound on the quantitative part of the problem. Thus, if one can compute (an approximation of) the winning set, a lower bound on the quantitative solution can be obtained by Bellman iteration or by other techniques for (unbounded) reachability in the continuous-state setting [9,34].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Symbolic controller synthesis for Büchi specifications on stochastic systems

Majumdar

Mallik

Soudjani

2020

Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control

Self Cite

View full text Add to dashboard Cite

We consider the policy synthesis problem for continuous-state controlled Markov processes evolving in discrete time, when the specification is given as a Büchi condition (visit a set of states infinitely often). We decompose computation of the maximal probability of satisfying the Büchi condition into two steps. The first step is to compute the maximal qualitative winning set, from where the Büchi condition can be enforced with probability one. The second step is to find the maximal probability of reaching the already computed qualitative winning set. In contrast with finite-state models, we show that such a computation only gives a lower bound on the maximal probability where the gap can be non-zero. In this paper we focus on approximating the qualitative winning set, while pointing out that the existing approaches for unbounded reachability computation can solve the second step. We provide an abstraction-based technique to approximate the qualitative winning set by simultaneously using an over-and under-approximation of the probabilistic transition relation. Since we are interested in qualitative properties, the abstraction is non-probabilistic; instead, the probabilistic transitions are assumed to be under the control of a (fair) adversary. Thus, we reduce the original policy synthesis problem to a Büchi game under a fairness assumption and characterize upper and lower bounds on winning sets as nested fixed point expressions in the-calculus. This characterization immediately provides a symbolic algorithm scheme. Further, a winning strategy computed on the abstract game can be refined to a policy on the controlled Markov process. We describe a concrete abstraction procedure and demonstrate our algorithm on two case studies. We show that our techniques are able to provide tight approximations to the qualitative winning set for the Van der Pol oscillator and a 3-d Dubins' vehicle. CCS CONCEPTS • Computing methodologies → Computational control theory.

show abstract

Section: Problem Definitionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Symbolic controller synthesis for Büchi specifications on stochastic systems

Majumdar

Mallik

Soudjani

2020

Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control

Self Cite

View full text Add to dashboard Cite

show abstract

“…Another approach to the stochastic synthesis problem with safety requirements is to directly maximize the safety probability under all classes of policies [48]. The related computational techniques generally rely on abstraction methods [18], [47]. A set-based computation approach to controller synthesis with safety guarantees is proposed in [8].…”

Section: Introductionmentioning

confidence: 99%

Automated Synthesis of Safe Digital Controllers for Sampled-Data Stochastic Nonlinear Systems

et al. 2020

Self Cite

View full text Add to dashboard Cite

We present a new method for the automated synthesis of digital controllers with formal safety guarantees for systems with nonlinear dynamics, noisy output measurements, and stochastic disturbances. Our method derives digital controllers such that the corresponding closed-loop system, modeled as a sampled-data stochastic control system, satisfies a safety specification with probability above a given threshold. Our technique uses a fast solver and an optimization method to search for candidate controllers, which are then formally evaluated in closed-loop with the system in question by a verified solver. Unstable candidate controllers are discarded by efficiently checking a sufficient condition for Lyapunov stability of sampled-data nonlinear systems. We evaluate our technique on three case studies: an artificial pancreas model, a powertrain control model, and a quadruple-tank process.

show abstract

“…In particular, the recent efforts on using Barrier Certificates for formal verification and synthesis of stochastic systems [40] has not been included in this edition of the ARCH competition. The notion of approximate similarity relation based on coupling stochastic processes [31,32] is able to handle systems with partial state observations but is not used by any of the tools in this year. We also welcome tools that are developed for specific sub-classes of stochastic systems having a particular property, e.g., mixed-monotonicity [20].…”

Section: Development and Implementation Of New Approachesmentioning

confidence: 99%

ARCH-COMP20 Category Report: Stochastic Models

Abate¹,

Blom²,

Cauchi³

et al.

EPiC Series in Computing

Self Cite

View full text Add to dashboard Cite

This report presents the results of a friendly competition for formal verification and policy synthesis of stochastic models. It also introduces new benchmarks within this category, and recommends next steps for this category towards next year’s edition of the competition. The friendly competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in Spring/Summer 2020.

show abstract

Robust Dynamic Programming for Temporal Logic Control of Stochastic Systems

Cited by 39 publications

References 39 publications

Symbolic controller synthesis for Büchi specifications on stochastic systems

Symbolic controller synthesis for Büchi specifications on stochastic systems

Automated Synthesis of Safe Digital Controllers for Sampled-Data Stochastic Nonlinear Systems

ARCH-COMP20 Category Report: Stochastic Models

Contact Info

Product

Resources

About