Abstract:This paper considers parametric Markov decision processes (pMDPs) whose transitions are equipped with affine functions over a finite set of parameters. The synthesis problem is to find a parameter valuation such that the instantiated pMDP satisfies a (temporal logic) specification under all strategies. We show that this problem can be formulated as a quadratically-constrained quadratic program (QCQP) and is non-convex in general. To deal with the NP-hardness of such problems, we exploit a convex-concave proced… Show more
“…The parameter feasibility problem considered in e.g. [14,15,19,23,27,30,41] is: Given a pMC M, a threshold λ ∈ [0, 1], and a graph-preserving region R, is there an instantiation u ∈ R s.t. Pr sI →T M (u) ≥ λ?…”
Section: Example 2 For the Pmc Inmentioning
confidence: 99%
“…By considering all possible orderings of s 1 and s 2 , we remain sound. The fact that parametric states typically have only two direct successors (as most pMCs are simple [15,34]) limits the number of orders.…”
Section: Making and Discharging Assumptionsmentioning
confidence: 99%
“…Despite the significant progress in the last years in analysing pMCs [10,15,41], the scalability of algorithms severely lacks behind methods for ordinary MCs.…”
Section: Introductionmentioning
confidence: 99%
“…Using a local NLP. The idea is to locally (at s 1 and s 2 ) consider the pMC and its characterising non-linear program (NLP) [4,5,15,19], together with the inequalities encoded by A . To refute an assumption to be globally valid, a single instantiation u refuting the assumption suffices.…”
This paper presents a simple algorithm to check whether reachability probabilities in parametric Markov chains are monotonic in (some of) the parameters. The idea is to construct-only using the graph structure of the Markov chain and local transition probabilities-a pre-order on the states. Our algorithm cheaply checks a sufficient condition for monotonicity. Experiments show that monotonicity in several benchmarks is automatically detected, and monotonicity can speed up parameter synthesis up to orders of magnitude faster than a symbolic baseline.
“…The parameter feasibility problem considered in e.g. [14,15,19,23,27,30,41] is: Given a pMC M, a threshold λ ∈ [0, 1], and a graph-preserving region R, is there an instantiation u ∈ R s.t. Pr sI →T M (u) ≥ λ?…”
Section: Example 2 For the Pmc Inmentioning
confidence: 99%
“…By considering all possible orderings of s 1 and s 2 , we remain sound. The fact that parametric states typically have only two direct successors (as most pMCs are simple [15,34]) limits the number of orders.…”
Section: Making and Discharging Assumptionsmentioning
confidence: 99%
“…Despite the significant progress in the last years in analysing pMCs [10,15,41], the scalability of algorithms severely lacks behind methods for ordinary MCs.…”
Section: Introductionmentioning
confidence: 99%
“…Using a local NLP. The idea is to locally (at s 1 and s 2 ) consider the pMC and its characterising non-linear program (NLP) [4,5,15,19], together with the inequalities encoded by A . To refute an assumption to be globally valid, a single instantiation u refuting the assumption suffices.…”
This paper presents a simple algorithm to check whether reachability probabilities in parametric Markov chains are monotonic in (some of) the parameters. The idea is to construct-only using the graph structure of the Markov chain and local transition probabilities-a pre-order on the states. Our algorithm cheaply checks a sufficient condition for monotonicity. Experiments show that monotonicity in several benchmarks is automatically detected, and monotonicity can speed up parameter synthesis up to orders of magnitude faster than a symbolic baseline.
“…Most of the work in parameter synthesis focus on finding one parameter value that satisfies the specification. The approaches involve computing a rational function of the reachability probabilities [11,17,41], utilizing convex optimization [34,40], and sampling-based methods [26,29]. The problem of whether there exists a value in the parameter space that satisfies a reachability specification is ETR-complete 4 [47], and finding a satisfying parameter value is exponential in the number of parameters.…”
We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are unknown. The problem is to compute the probability to satisfy a temporal logic specification within any MDP that corresponds to a sample from these unknown distributions. In general, this problem is undecidable, and we resort to techniques from so-called scenario optimization. Based on a finite number of samples of the uncertain parameters, each of which induces an MDP, the proposed method estimates the probability of satisfying the specification by solving a finite-dimensional convex optimization problem. The number of samples required to obtain a high confidence on this estimate is independent from the number of states and the number of random parameters. Experiments on a large set of benchmarks show that a few thousand samples suffice to obtain high-quality confidence bounds with a high probability.
This paper considers parametric Markov decision processes (pMDPs) whose transitions are equipped with affine functions over a finite set of parameters. The synthesis problem is to find a parameter valuation such that the instantiated pMDP satisfies a (temporal logic) specification under all strategies. We show that this problem can be formulated as a quadratically-constrained quadratic program (QCQP) and is non-convex in general. To deal with the NP-hardness of such problems, we exploit a convex-concave procedure (CCP) to iteratively obtain local optima. An appropriate interplay between CCP solvers and probabilistic model checkers creates a procedure -realized in the tool PROPheSYthat solves the synthesis problem for models with thousands of parameters.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.