Probabilistic reachability for parametric Markov models

Abstract

Abstract. Given a parametric Markov model, we consider the problem of computing the rational function expressing the probability of reaching a given set of states. To attack this principal problem, Daws has suggested to first convert the Markov chain into a finite automaton, from which a regular expression is computed. Afterwards, this expression is evaluated to a closed form function representing the reachability probability. This paper investigates how this idea can be turned into an effective procedure. It turns out that the bottleneck lies in the growth of the regular expression relative to the number of states (n Θ(log n) ). We therefore proceed differently, by tightly intertwining the regular expression computation with its evaluation. This allows us to arrive at an effective method that avoids this blow up in most practical cases. We give a detailed account of the approach, also extending to parametric models with rewards and with non-determinism. Experimental evidence is provided, illustrating that our implementation provides meaningful insights on non-trivial models.

PARAM: A Model Checker for Parametric Markov Models

“…We consider parametric Markov chains (PMCs) [1], where e.g., certain transition probabilities are symbolic parameters. As a result, the analysis of properties, such as the probability of reaching a set of goal states, yields symbolic expressions [2,3] rather than concrete values. These symbolic expressions are represented by multivariate rational functions, i.e.…”

“…It can lead to dramatic speed-ups. For parametric Markov decision processes (PMDPs), lumping maintains only maximal reachability [2,3].…”

“…The algorithmic basis for our tool has been laid out in a previous publication [2] together with a prototypical implementation. PARAM 1.0 [3] improves and enhances this prototype in many aspects.…”

“…PARAM 1.0 [3] improves and enhances this prototype in many aspects. We give an overview of important distinguishing features below.…”

SMT-Based Bisimulation Minimisation of Markov Models

“…Although -like for BDD-based quotienting-there are examples with smaller improvements or where this approach is inferior, we believe that Smtbased quotienting offers clear potential. Other advantages of our approach are that it is applicable to infinite probabilistic programs whose bisimulation quotient is finite and that it is directly applicable to parametric Markov chains [8].…”