Reachability analysis of uncertain systems using bounded-parameter Markov decision processes

Wu, Di; Koutsoukos, Xenofon

doi:10.1016/j.artint.2007.12.002

Cited by 32 publications

(24 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Secondly, from the side of algorithmic developments, several verification methods for uncertain Markov models have been proposed. The problems of computing reachability probabilities and expected total reward for IMC s and IMDP s were first investigated in [10,36]. Then, several of their PCTL and LTL model checking algorithms were introduced in [2,8,10] and [23,33,35], respectively.…”

Section: Introductionmentioning

confidence: 99%

Multi-objective Robust Strategy Synthesis for Interval Markov Decision Processes

Hahn

Hashemi

Hermanns

et al. 2017

Quantitative Evaluation of Systems

View full text Add to dashboard Cite

Interval Markov decision processes (IMDP s) generalise classical MDPs by having interval-valued transition probabilities. They provide a powerful modelling tool for probabilistic systems with an additional variation or uncertainty that prevents the knowledge of the exact transition probabilities. In this paper, we consider the problem of multi-objective robust strategy synthesis for interval MDPs, where the aim is to find a robust strategy that guarantees the satisfaction of multiple properties at the same time in face of the transition probability uncertainty. We first show that this problem is PSPACE-hard. Then, we provide a value iteration-based decision algorithm to approximate the Pareto set of achievable points. We finally demonstrate the practical effectiveness of our proposed approaches by applying them on several case studies using a prototypical tool. arXiv:1706.06875v2 [cs.SY] 6 Jul 2017 E. M. Hahn et al. Consider now the second case: if ∼ h = ≤, then M σ |= Π [r h ] ≤k h ≤r h if and only if max π∈Π ξ r h [k h ](ξ) dPr σ,π M ≤ r h . Since for each path ξ ∈ Paths, −r h [k]( (ξ)) = r h [k](ξ), by the definition of the components I ,r h , and σ it is the case that max π∈Π ξThis completes the analysis of the case ϕ h = [r h ] ≤k h ∼ h r h for each h ∈ {n + 1, . . . , m}; since M σ |= Π ϕ j for each j ∈ {1, . . . , m}, it follows that ϕ is satisfiable in M, as required to prove that "if ϕ is satisfiable in M , then ϕ is satisfiable in M".Having proved both implications, the statement of the proposition "ϕ is satisfiable in M if and only if ϕ is satisfiable in M " holds, as required.

show abstract

Section: Introductionmentioning

confidence: 99%

Multi-objective Robust Strategy Synthesis for Interval Markov Decision Processes

Hahn

Hashemi

Hermanns

et al. 2017

Quantitative Evaluation of Systems

View full text Add to dashboard Cite

show abstract

“…Interval MDPs [28] (also called Bounded-parameter MDPs [13,37]) address this need by bounding the probabilities of each successor state by an interval instead of a fixed number. In such a model, the transition probabilities are not fully specified and this uncertainty again needs to be resolved nondeterministically.…”

Section: Introductionmentioning

confidence: 99%

Probabilistic Bisimulations for PCTL Model Checking of Interval MDPs

Hashemi

Hatefi

Krčál

2014

Electron. Proc. Theor. Comput. Sci.

View full text Add to dashboard Cite

Verification of PCTL properties of MDPs with convex uncertainties has been investigated recently by Puggelli et al. However, model checking algorithms typically suffer from state space explosion. In this paper, we address probabilistic bisimulation to reduce the size of such an MDPs while preserving PCTL properties it satisfies. We discuss different interpretations of uncertainty in the models which are studied in the literature and that result in two different definitions of bisimulations. We give algorithms to compute the quotients of these bisimulations in time polynomial in the size of the model and exponential in the uncertain branching. Finally, we show by a case study that large models in practice can have small branching and that a substantial state space reduction can be achieved by our approach.

show abstract

“…The objective in this problem is to find the maximum probability that a set of states can be reached from any other state in an mdp. Prior work also solves the maximal reachability probability problem for a bmdp [30]. The key difference for a bmdp is that the expected value (maximum probability) for each state is not a scalar value, but rather a range derived from the transition probability bounds.…”

Section: Product Bmdp and Optimal Policymentioning

confidence: 99%

Asymptotically Optimal Stochastic Motion Planning with Temporal Goals

Luna

Lahijanian

Moll

et al. 2015

Springer Tracts in Advanced Robotics

View full text Add to dashboard Cite

Abstract. This work presents a planning framework that allows a robot with stochastic action uncertainty to achieve a high-level task given in the form of a temporal logic formula. The objective is to quickly compute a feedback control policy to satisfy the task specification with maximum probability. A top-down framework is proposed that abstracts the motion of a continuous stochastic system to a discrete, boundedparameter Markov decision process (bmdp), and then computes a control policy over the product of the bmdp abstraction and a dfa representing the temporal logic specification. Analysis of the framework reveals that as the resolution of the bmdp abstraction becomes finer, the policy obtained converges to optimal. Simulations show that high-quality policies to satisfy complex temporal logic specifications can be obtained in seconds, orders of magnitude faster than existing methods.

show abstract

Reachability analysis of uncertain systems using bounded-parameter Markov decision processes

Cited by 32 publications

References 18 publications

Multi-objective Robust Strategy Synthesis for Interval Markov Decision Processes

Multi-objective Robust Strategy Synthesis for Interval Markov Decision Processes

Probabilistic Bisimulations for PCTL Model Checking of Interval MDPs

Asymptotically Optimal Stochastic Motion Planning with Temporal Goals

Contact Info

Product

Resources

About