Scenario-Based Verification of Uncertain MDPs

Cubuktepe, Murat; Jansen, Nils; Junges, Sebastian; Katoen, Joost-Pieter; Topcu, Ufuk

doi:10.1007/978-3-030-45190-5_16

Cited by 17 publications

(13 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These are used to evaluate e.g., robustness of systems [1], or to synthesise POMDP controllers [43]. Many state-of-the-art approaches [19,26,21] require the evaluation of various instantiated Markov chains, and Rubicon shows it is well-suited to this setting. More generally, support of inference techniques opens the door to a variety of algorithms for additional queries, e.g, computing conditional probabilities [3,8].…”

Section: Discussion Related Work and Conclusionmentioning

confidence: 99%

Model Checking Finite-Horizon Markov Chains with Probabilistic Inference

Holtzen¹,

Junges²,

Vazquez-Chanlatte³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We revisit the symbolic verification of Markov chains with respect to finite horizon reachability properties. The prevalent approach iteratively computes step-bounded state reachability probabilities. By contrast, recent advances in probabilistic inference suggest symbolically representing all horizon-length paths through the Markov chain. We ask whether this perspective advances the state-of-the-art in probabilistic model checking. First, we formally describe both approaches in order to highlight their key differences. Then, using these insights we develop Rubicon, a tool that transpiles Prism models to the probabilistic inference tool Dice. Finally, we demonstrate better scalability compared to probabilistic model checkers on selected benchmarks. All together, our results suggest that probabilistic inference is a valuable addition to the probabilistic model checking portfolio -with Rubicon as a first step towards integrating both perspectives.

show abstract

Section: Discussion Related Work and Conclusionmentioning

confidence: 99%

Model Checking Finite-Horizon Markov Chains with Probabilistic Inference

Holtzen¹,

Junges²,

Vazquez-Chanlatte³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Contributions. This paper revises an earlier conference paper [20] as follows. Due to new results in [29] that lift some assumptions required for the scenario approach, we can simplify and generalize our approach by a simplified chance-constrained program.…”

Section: Uncertainty On Mdpsmentioning

confidence: 91%

“…This change in the approach yields completely revised technical sections of the paper. Furthermore, this paper fixes a technical error in [20]. The (new) bounds in Theorem 1 of this paper are less pessimistic.…”

Section: Uncertainty On Mdpsmentioning

confidence: 94%

See 1 more Smart Citation

Scenario-Based Verification of Uncertain Parametric MDPs

Badings,

Cubuktepe,

Jansen

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We consider parametric Markov decision processes (pMDPs) that are augmented with unknown probability distributions over parameter values. The problem is to compute the probability to satisfy a temporal logic specification within any concrete MDP that corresponds to a sample from these distributions. As this problem is infeasible to solve precisely, we resort to sampling techniques that exploit the so-called scenario approach. Based on a finite number of samples of the parameters, the proposed method yields high-confidence bounds on the probability of satisfying the specification. The number of samples required to obtain a high confidence on these bounds is independent of the number of states and the number of random parameters. Experiments on a large set of benchmarks show that several thousand samples suffice to obtain tight and high-confidence lower and upper bounds on the satisfaction probability.

show abstract

“…Further variations of parameter synthesis, e.g., consider statistical guarantees for parameter synthesis, often with some prior on the parameter values [34]- [36]. These approaches cannot provide the absolute guarantees on an answer that the methods in this paper provide.…”

Section: B Related Workmentioning

confidence: 98%

Convex Optimization for Parameter Synthesis in MDPs

Cubuktepe¹,

Jansen²,

Junges³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Probabilistic model checking aims to prove whether a Markov decision process (MDP) satisfies a temporal logic specification. The underlying methods rely on an often unrealistic assumption that the MDP is precisely known. Consequently, parametric MDPs (pMDPs) extend MDPs with transition probabilities that are functions over unspecified parameters. The parameter synthesis problem is to compute an instantiation of these unspecified parameters such that the resulting MDP satisfies the temporal logic specification. We formulate the parameter synthesis problem as a quadratically constrained quadratic program (QCQP), which is nonconvex and is NP-hard to solve in general. We develop two approaches that iteratively obtain locally optimal solutions. The first approach exploits the socalled convex-concave procedure (CCP), and the second approach utilizes a sequential convex programming (SCP) method. The techniques improve the runtime and scalability by multiple orders of magnitude compared to black-box CCP and SCP by merging ideas from convex optimization and probabilistic model checking. We demonstrate the approaches on a satellite collision avoidance problem with hundreds of thousands of states and tens of thousands of parameters and their scalability on a wide range of commonly used benchmarks.M. Cubuktepe and U. Topcu are with the Department of Aerospace Engineering and Engineering Mechanics, Austin, USA.

show abstract

Scenario-Based Verification of Uncertain MDPs

Cited by 17 publications

References 41 publications

Model Checking Finite-Horizon Markov Chains with Probabilistic Inference

Model Checking Finite-Horizon Markov Chains with Probabilistic Inference

Scenario-Based Verification of Uncertain Parametric MDPs

Convex Optimization for Parameter Synthesis in MDPs

Contact Info

Product

Resources

About