Controller Synthesis for Probabilistic Systems (Extended Abstract)

Baier, Christel; Größer, Marcus; Leucker, Martin; Bollig, Benedikt; Ciesinski, Frank

doi:10.1007/1-4020-8141-3_38

Cited by 58 publications

(80 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The set of accepting end components can be computed in time polynomial in the size of the product MDP. For a reachability specification ϕ, for any state, a policy that maximizes the probability of satisfying ϕ indeed maximizes the probability of hitting a set of final states in the product MDP from that state (see [11]), our result is still applicable and there is no need to compute the end components.…”

Section: Definition 4 (Accepting End Components) the End Componentmentioning

confidence: 93%

Probably Approximately Correct MDP Learning and Control With Temporal Logic Constraints

Topcu²

2014

Robotics: Science and Systems X

106

102

View full text Add to dashboard Cite

Abstract-We consider synthesis of controllers that maximize the probability of satisfying given temporal logic specifications in unknown, stochastic environments. We model the interaction between the system and its environment as a Markov decision process (MDP) with initially unknown transition probabilities. The solution we develop builds on the so-called model-based probably approximately correct Markov decision process (PAC-MDP) method. The algorithm attains an ε-approximately optimal policy with probability 1−δ using samples (i.e. observations), time and space that grow polynomially with the size of the MDP, the size of the automaton expressing the temporal logic specification,and a finite time horizon. In this approach, the system maintains a model of the initially unknown MDP, and constructs a product MDP based on its learned model and the specification automaton that expresses the temporal logic constraints. During execution, the policy is iteratively updated using observation of the transitions taken by the system. The iteration terminates in finitely many execution steps. With high probability, the resulting policy is such that, for any state, the difference between the probability of satisfying the specification under this policy and the optimal one is within a predefined bound.

show abstract

Section: Definition 4 (Accepting End Components) the End Componentmentioning

confidence: 93%

Probably Approximately Correct MDP Learning and Control With Temporal Logic Constraints

Topcu²

2014

Robotics: Science and Systems X

106

102

View full text Add to dashboard Cite

show abstract

“…Hence verification typically takes the form of considering a requirement ϕ (for example reachability, which specifies that a state with a node F is eventually reached, but more generally an ω-regular property), and a threshold λ ∈ [0, 1], and then relies on determining whether the maximum probability of satisfying ϕ is at least λ. Controller synthesis approaches take a similar form although, recalling that control of probabilistic systems is typically stated in terms of a stochastic game [3,6], in that setting there are strategies belonging to each player, and relies in determining whether the controller player can guarantee that ϕ is satisfied with probability at least λ, regardless of the behaviour of the environment player.…”

Section: Background and Motivationmentioning

confidence: 99%

“…The approach taken in [19,12,15] to analyse a stochastic hybrid automaton S is to construct overapproximating probabilistic hybrid automaton P, where P is obtained from S by replacing the probabilistic choice involved in a probabilistic reset (over a continuous domain) by a discrete probabilistic choice (over a finite domain) over a number of intervals that cover the domain of the probabilistic reset, followed by a nondeterministic choice within the chosen interval. For example, consider the probabilistic reset in which a variable x is updated according to a uniform distribution over [1,3]. The probabilistic reset can be replaced by a discrete probabilistic choice over (for example) the intervals [1,2] and [2,3], each of which correspond to probability 1 2 , in accordance with the original uniform distribution, and which is then followed by a nondeterministic choice over the chosen interval.…”

Section: Approximation With Probabilistic Rectangular Hybrid Automatamentioning

confidence: 99%

“…For example, consider the probabilistic reset in which a variable x is updated according to a uniform distribution over [1,3]. The probabilistic reset can be replaced by a discrete probabilistic choice over (for example) the intervals [1,2] and [2,3], each of which correspond to probability 1 2 , in accordance with the original uniform distribution, and which is then followed by a nondeterministic choice over the chosen interval. If, in all other respects (nodes, flows etc.…”

Section: Approximation With Probabilistic Rectangular Hybrid Automatamentioning

confidence: 99%

See 1 more Smart Citation

Verification and Control of Probabilistic Rectangular Hybrid Automata

Sproston

2015

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Hybrid systems are characterised by a combination of discrete and continuous components. In many application areas for hybrid systems, such as vehicular control and systems biology, stochastic behaviour is intrinsic. This has led to the development of stochastic extensions of formalisms, such as hybrid automata, for the modelling of hybrid systems, together with their associated verification and controller synthesis algorithms, in order to allow reasoning about quantitative properties such as "the vehicle's speed will reach 50kph within 10 seconds with probability at least 0.99". We consider probabilistic rectangular hybrid automata, which generalise the well-known class of rectangular hybrid automata with the possibility of representing random behavior of the discrete components of the system, permitting the modeling of the likelihood of faults, choices in randomised algorithms and message losses. We highlight the differences between verification and control problems for probabilistic rectangular hybrid automata and the corresponding problems for non-probabilistic rectangular hybrid automata. Furthermore, we will describe the effect of assumptions on the underlying model of time (discrete or continuous) on the considered verification and control problems. Finally, we will also consider how probabilistic rectangular hybrid automata can be used as abstract models for more general classes of stochastic hybrid systems.

show abstract

“…If the bound is restricted just to 0 and 1, we obtain the qualitative fragment of a given logic. Controller synthesis for MDPs with branching-time objectives has been considered in [1] where it is shown that strategies for fairly simple qualitative PCTL objectives may require memory and/or randomization. Hence, the classes of MD, MR, HD, and HR strategies (see above) form a strict hierarchy.…”

Section: Introductionmentioning

confidence: 99%

Controller Synthesis and Verification for Markov Decision Processes with Qualitative Branching Time Objectives

Brázdil

Forejt

Kučera

Automata, Languages and Programming

View full text Add to dashboard Cite

Abstract. We show that the controller synthesis and verification problems for Markov decision processes with qualitative PECTL * objectives are 2-EXPTIME complete. More precisely, the algorithms are polynomial in the size of a given Markov decision process and doubly exponential in the size of a given qualitative PECTL * formula. Moreover, we show that if a given qualitative PECTL * objective is achievable by some strategy, then it is also achievable by an effectively constructible one-counter strategy, where the associated complexity bounds are essentially the same as above. For the fragment of qualitative PCTL objectives, we obtain EXPTIME completeness and the algorithms are only singly exponential in the size of the formula.

show abstract

Controller Synthesis for Probabilistic Systems (Extended Abstract)

Cited by 58 publications

References 23 publications

Probably Approximately Correct MDP Learning and Control With Temporal Logic Constraints

Probably Approximately Correct MDP Learning and Control With Temporal Logic Constraints

Verification and Control of Probabilistic Rectangular Hybrid Automata

Controller Synthesis and Verification for Markov Decision Processes with Qualitative Branching Time Objectives

Contact Info

Product

Resources

About