Probabilistic Reachability for Parametric Markov Models

Abstract

Abstract. Given a parametric Markov model, we consider the problem of computing the formula expressing the probability of reaching a given set of states. To attack this principal problem, Daws has suggested to first convert the Markov chain into a finite automaton, from which a regular expression is computed. Afterwards, this expression is evaluated to a closed form expression representing the reachability probability. This paper investigates how this idea can be turned into an effective procedure. It turns out that the bottleneck lies in an exponential growth of the regular expression relative to the number of states. We therefore proceed differently, by tightly intertwining the regular expression computation with its evaluation. This allows us to arrive at an effective method that avoids the exponential blow up in most practical cases. We give a detailed account of the approach, also extending to parametric models with rewards and with non-determinism. Experimental evidence is provided, illustrating that our implementation provides meaningful insights on non-trivial models.

Are Parametric Markov Chains Monotonic?

“…The closed-form of Pr s→T on a graph-preserving region is a rational function over V , i.e., a fraction of two polynomials over V . Various methods for computing this closed form on a graph-preserving region have been proposed [18,19,22,27,30]. Such a closed-form can be exponential in the number of parameters [30], and is typically (very) large already with one or two parameters [19,27].…”

“…The parameter feasibility problem considered in e.g. [14,15,19,23,27,30,41] is: Given a pMC M, a threshold λ ∈ [0, 1], and a graph-preserving region R, is there an instantiation u ∈ R s.t. Pr sI →T M (u) ≥ λ?…”

“…Applied to pMCs [31], it preserves the reachability probabilities of target set T . For each SCC, all nonentry states (i.e., states without incoming transitions from outside the SCC) are eliminated by state elimination [18,27] and transitions between entry states are deleted [19]. In the resulting acyclic pMC M ′ , transitions from entry state s of an eliminated SCC directly lead to states t outside this SCC.…”

“…Applications of pMCs include model repair [5,12,13,24,40], strategy synthesis in AI models such as partially observable MDPs [34], and optimising randomised distributed algorithms [2]. PRISM and Storm, as well as dedicated tools including PARAM [27] and PROPhESY [19] support pMC analysis.…”

Efficient Parametric Model Checking Using Domain Knowledge

“…Parametric model checking (PMC) [20], [34], [36] is a formal technique for the analysis of Markov chains with transitions probabilities specified as rational functions over a set of continuous variables. When the analysed Markov chains model software systems, these variables represent configurable parameters of the software or environment parameters unknown until runtime.…”