Parametric Markov Chains: PCTL Complexity and Fraction-free Gaussian Elimination

Abstract

Parametric Markov chains have been introduced as a model for families of stochastic systems that rely on the same graph structure, but differ in the concrete transition probabilities. The latter are specified by polynomial constraints for the parameters. Among the tasks typically addressed in the analysis of parametric Markov chains are (1) the computation of closed-form solutions for reachabilty probabilities and other quantitative measures and (2) finding symbolic representations of the set of parameter valuations for which a given temporal logical formula holds as well as (3) the decision variant of (2) that asks whether there exists a parameter valuation where a temporal logical formula holds. Our contribution to (1) is to show that existing implementations for computing rational functions for reachability probabilities or expected costs in parametric Markov chains can be improved by using fraction-free Gaussian elimination, a long-known technique for linear equation systems with parametric coefficients. Our contribution to (2) and (3) is a complexity-theoretic discussion of the model checking problem for parametric Markov chains and probabilistic computation tree logic (PCTL) formulas. We present an exponential-time algorithm for (2) and a PSPACE upper bound for (3). Moreover, we identify fragments of PCTL and subclasses of parametric Markov chains where (1) and (3) are solvable in polynomial time and establish NP-hardness for other PCTL fragments.

Are Parametric Markov Chains Monotonic?

“…The closed-form of Pr s→T on a graph-preserving region is a rational function over V , i.e., a fraction of two polynomials over V . Various methods for computing this closed form on a graph-preserving region have been proposed [18,19,22,27,30]. Such a closed-form can be exponential in the number of parameters [30], and is typically (very) large already with one or two parameters [19,27].…”

“…The parameter feasibility problem considered in e.g. [14,15,19,23,27,30,41] is: Given a pMC M, a threshold λ ∈ [0, 1], and a graph-preserving region R, is there an instantiation u ∈ R s.t. Pr sI →T M (u) ≥ λ?…”

“…This problem is square-rootsum hard [14]. For any fixed number of parameters, this problem is decidable in P [30]. Fig.…”

“…If assumptions are needed (l. [21][22][23][24][25][26], then we push the three assumptions to the Queue. Therefore, l. 27, should not be executed, so we continue to the next iteration of the while loop after adding assumptions (l. 26) At the end, we check for global monotonicity (l. [29][30][31][32][33][34][35][36][37].…”

Unpredictable Planning Under Partial Observability

“…In this regard, FSCs represent a subset of controllers which may, in general, utilize the whole information history. Following the results of [8], we show that the entropy maximization problem over FSCs can be recast as a so-called parameter synthesis problem for a parametric Markov chain (pMC) [9], [10], under certain assumptions on the memory state transitions of the FSC. We can then efficiently solve for an entropy maximizing FSC for a fixed number of memory states.…”