2018 # Synthesis in pMDPs: A Tale of 1001 Parameters

**Abstract:** This paper considers parametric Markov decision processes (pMDPs) whose transitions are equipped with affine functions over a finite set of parameters. The synthesis problem is to find a parameter valuation such that the instantiated pMDP satisfies a (temporal logic) specification under all strategies. We show that this problem can be formulated as a quadratically-constrained quadratic program (QCQP) and is non-convex in general. To deal with the NP-hardness of such problems, we exploit a convex-concave proced…

Help me understand this report

Search citation statements

Paper Sections

Select...

2

1

1

Citation Types

0

47

0

Year Published

2018

2020

Publication Types

Select...

6

Relationship

4

2

Authors

Journals

(47 citation statements)

0

47

0

“…The parameter feasibility problem considered in e.g. [14,15,19,23,27,30,41] is: Given a pMC M, a threshold λ ∈ [0, 1], and a graph-preserving region R, is there an instantiation u ∈ R s.t. Pr sI →T M (u) ≥ λ?…”

confidence: 99%

“…The parameter feasibility problem considered in e.g. [14,15,19,23,27,30,41] is: Given a pMC M, a threshold λ ∈ [0, 1], and a graph-preserving region R, is there an instantiation u ∈ R s.t. Pr sI →T M (u) ≥ λ?…”

confidence: 99%

“…By considering all possible orderings of s 1 and s 2 , we remain sound. The fact that parametric states typically have only two direct successors (as most pMCs are simple [15,34]) limits the number of orders.…”

confidence: 99%

“…Most of the work in parameter synthesis focus on finding one parameter value that satisfies the specification. The approaches involve computing a rational function of the reachability probabilities [11,17,41], utilizing convex optimization [34,40], and sampling-based methods [26,29]. The problem of whether there exists a value in the parameter space that satisfies a reachability specification is ETR-complete 4 [47], and finding a satisfying parameter value is exponential in the number of parameters.…”

confidence: 99%