Optimal control of MDPs with temporal logic constraints

Svoreňová, Mária; Černá, Ivana; Belta, Călin

doi:10.1109/cdc.2013.6760491

Cited by 38 publications

(31 citation statements)

References 19 publications

(28 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors present a solution that is optimal only in special cases. In [34], we prove that our approach results in an optimal solution for any MDP and LTL formula. We also strongly believe that a similar approach can be used for non-deterministic transition systems but one first needs to properly define the optimization objective APPC that accounts for the non-determinism of transitions.…”

Section: Discussionmentioning

confidence: 95%

See 1 more Smart Citation

Optimal Temporal Logic Control for Deterministic Transition Systems With Probabilistic Penalties

Svoreňová

Černá

Belta

2015

IEEE Trans. Automat. Contr.

Self Cite

View full text Add to dashboard Cite

We consider an optimal control problem for a weighted deterministic transition system required to satisfy a constraint expressed as a Linear Temporal Logic (LTL) formula over its labels. By assuming that the executions of the system incur time-varying penalties modeled as Markov chains, our goal is to minimize the expected average cumulative penalty incurred between consecutive satisfactions of a desired property. Using concepts from theoretical computer science, we provide two solutions to this problem. First, we derive a provably correct optimal strategy within the class of strategies that do not exploit values of penalties sensed in real time. Second, we show that by taking advantage of locally sensing the penalties, we can construct heuristic strategies leading to lower collected penalty. While still ensuring satisfaction of the LTL constraint, we cannot guarantee optimality in the latter case. We provide a user-friendly implementation of the proposed algorithms and analysis of two case studies.

show abstract

Section: Discussionmentioning

confidence: 95%

“…Our preliminary results on this topic can be found in [34], where we consider the analogous problem for MDPs with static costs. This problem was also investigated in [35].…”

Section: Discussionmentioning

confidence: 99%

Optimal Temporal Logic Control for Deterministic Transition Systems With Probabilistic Penalties

Svoreňová

Černá

Belta

2015

IEEE Trans. Automat. Contr.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Since the satisfactory runs for a temporal logic are usually of infinite length, it is not trivial to assign costs to a particular run. Accumulative, average, weight average, and expected average costs for transitions or between two consecutive satisfactions are all considered in the literature, see e.g., [71,72,75].…”

Section: Optimalitymentioning

confidence: 99%

“…In [16], the authors synthesized a control policy such that the MDP satisfies the given specification almost surely, if such a policy exists. The control strategies synthesis for MDP so to minimize the expected average cost between two consecutive satisfactions of a desired LTL property was considered in [72] by using results from the game theory [10].…”

Section: Optimalitymentioning

confidence: 99%

Mission Accomplished: An Introduction to Formal Methods in Mobile Robot Motion Planning and Control

Lin

2014

Un. Sys.

View full text Add to dashboard Cite

A new trend in the robotic motion planning literature is to use formal methods, like model checking, reactive synthesis and supervisory control theory, to automatically design controllers that drive a mobile robot to accomplish some high level missions in a guaranteed manner. This is also known as the correct-by-construction method. The high level missions are usually specified as temporal logics, particularly as linear temporal logic formulas, due to their similarity to human natural languages. This paper provides a brief overview of the recent developments in this newly emerged research area. A number of fundamental topics, such as temporal logic, model checking, bisimulation quotient transition systems and reachability controller design are reviewed. Additionally, the key challenges and possible future directions in this area are briefly discussed with references given for further reading.

show abstract

“…While the synthesis problem also has a long tradition [9,3,22], it has gained significant attention in formal methods more recently. These techniques are being deployed in control and path planning in particular: model checking techniques can be adapted to synthesize (optimal) controllers for deterministic finite systems [24,14], Büchi and Rabin games can be reformulated as control strategies for nondeterministic systems [29,27], and probabilistic games can be used to control finite probabilistic systems such as Markov decision processes [23,18].…”

Section: Introductionmentioning

confidence: 99%

Temporal logic control for stochastic linear systems using abstraction refinement of probabilistic games

Svoreňová

Křetínský

Chmelík

et al. 2015

Proceedings of the 18th International Conference on Hybrid Systems: Computation and Control

Self Cite

View full text Add to dashboard Cite

We consider the problem of computing the set of initial states of a dynamical system such that there exists a control strategy to ensure that the trajectories satisfy a temporal logic specification with probability 1 (almost-surely). We focus on discrete-time, stochastic linear dynamics and specifications given as formulas of the Generalized Reactivity(1) fragment of Linear Temporal Logic over linear predicates in the states of the system. We propose a solution based on iterative abstraction-refinement, and turn-based 2-player probabilistic games. While the theoretical guarantee of our algorithm after any finite number of iterations is only a partial solution, we show that if our algorithm terminates, then the result is the set of satisfying initial states. Moreover, for any (partial) solution our algorithm synthesizes witness control strategies to ensure almost-sure satisfaction of the temporal logic specification. We demonstrate our approach on an illustrative case study.

show abstract

Optimal control of MDPs with temporal logic constraints

Cited by 38 publications

References 19 publications

Optimal Temporal Logic Control for Deterministic Transition Systems With Probabilistic Penalties

Optimal Temporal Logic Control for Deterministic Transition Systems With Probabilistic Penalties

Mission Accomplished: An Introduction to Formal Methods in Mobile Robot Motion Planning and Control

Temporal logic control for stochastic linear systems using abstraction refinement of probabilistic games

Contact Info

Product

Resources

About