Synthesis in pMDPs: A Tale of 1001 Parameters

Automated Technology for Verification and Analysis

Katoen

2019

Self Cite

This paper presents a simple algorithm to check whether reachability probabilities in parametric Markov chains are monotonic in (some of) the parameters. The idea is to construct-only using the graph structure of the Markov chain and local transition probabilities-a pre-order on the states. Our algorithm cheaply checks a sufficient condition for monotonicity. Experiments show that monotonicity in several benchmarks is automatically detected, and monotonicity can speed up parameter synthesis up to orders of magnitude faster than a symbolic baseline.

Section: Example 2 For the Pmc Inmentioning

confidence: 99%

Section: Making and Discharging Assumptionsmentioning

confidence: 99%

“…Despite the significant progress in the last years in analysing pMCs [10,15,41], the scalability of algorithms severely lacks behind methods for ordinary MCs.…”

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Are Parametric Markov Chains Monotonic?

Spel

Automated Technology for Verification and Analysis

Katoen

2019

Self Cite

“…Most of the work in parameter synthesis focus on finding one parameter value that satisfies the specification. The approaches involve computing a rational function of the reachability probabilities [11,17,41], utilizing convex optimization [34,40], and sampling-based methods [26,29]. The problem of whether there exists a value in the parameter space that satisfies a reachability specification is ETR-complete 4 [47], and finding a satisfying parameter value is exponential in the number of parameters.…”

Section: Introductionmentioning

confidence: 99%

Scenario-Based Verification of Uncertain MDPs

Cubuktepe

Jansen

Tools and Algorithms for the Construction and Analysis of Systems

et al. 2020

Self Cite

We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are unknown. The problem is to compute the probability to satisfy a temporal logic specification within any MDP that corresponds to a sample from these unknown distributions. In general, this problem is undecidable, and we resort to techniques from so-called scenario optimization. Based on a finite number of samples of the uncertain parameters, each of which induces an MDP, the proposed method estimates the probability of satisfying the specification by solving a finite-dimensional convex optimization problem. The number of samples required to obtain a high confidence on this estimate is independent from the number of states and the number of random parameters. Experiments on a large set of benchmarks show that a few thousand samples suffice to obtain high-quality confidence bounds with a high probability.

Synthesis in pMDPs: A Tale of 1001 Parameters

Cubuktepe

Jansen²,

Automated Technology for Verification and Analysis

et al. 2018

Self Cite

This paper considers parametric Markov decision processes (pMDPs) whose transitions are equipped with affine functions over a finite set of parameters. The synthesis problem is to find a parameter valuation such that the instantiated pMDP satisfies a (temporal logic) specification under all strategies. We show that this problem can be formulated as a quadratically-constrained quadratic program (QCQP) and is non-convex in general. To deal with the NP-hardness of such problems, we exploit a convex-concave procedure (CCP) to iteratively obtain local optima. An appropriate interplay between CCP solvers and probabilistic model checkers creates a procedure -realized in the tool PROPheSYthat solves the synthesis problem for models with thousands of parameters.