Finding Provably Optimal Markov Chains

Spel, Jip; Katoen, Joost-Pieter; Junges, Sebastian

doi:10.26226/morressier.604907f41a80aac83ca25cf0

Cited by 1 publication

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, for models where most time is spent on model checking the macro-level MDP, the approach is less suitable. We furthermore conjecture that tailored algorithms may exploit some of these dimensions, e.g., when there is the macro-MDP or the subMDPs are indeed MCs or perhaps acyclic, depending on the number of parameters and their influence [36], or based on the relative weight of the uncertain rewards compared to rewards in the macro-MDP.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Abstraction-Refinement for Hierarchical Probabilistic Models

Junges¹,

Spaan²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Markov decision processes are a ubiquitous formalism for modelling systems with non-deterministic and probabilistic behavior. Verification of these models is subject to the famous state space explosion problem. We alleviate this problem by exploiting a hierarchical structure with repetitive parts. This structure not only occurs naturally in robotics, but also in probabilistic programs describing, e.g., network protocols. Such programs often repeatedly call a subroutine with similar behavior. In this paper, we focus on a local case, in which the subroutines have a limited effect on the overall system state. The key ideas to accelerate analysis of such programs are (1) to treat the behavior of the subroutine as uncertain and only remove this uncertainty by a detailed analysis if needed, and (2) to abstract similar subroutines into a parametric template, and then analyse this template. These two ideas are embedded into an abstraction-refinement loop that analyses hierarchical MDPs. A prototypical implementation shows the efficacy of the approach.

show abstract

Section: Methodsmentioning

confidence: 99%

“…We denote this region also with [[u − , u + ]]. For regions, we may compute a lower bound on min u∈R ER max M[u] (♦T ) and an upper bound on max u∈R ER max M[u] (♦T ) via parameter lifting [33,36].…”

Section: Regions and Parametric Model Checking A Set Of Valuations De...mentioning

confidence: 99%