Modelling, Reduction and Analysis of Markov Automata

Abstract

Markov automata (MA) constitute an expressive continuoustime compositional modelling formalism. They appear as semantic backbones for engineering frameworks including dynamic fault trees, Generalised Stochastic Petri Nets, and AADL. Their expressive power has thus far precluded them from effective analysis by probabilistic (and statistical) model checkers, stochastic game solvers, or analysis tools for Petri net-like formalisms. This paper presents the foundations and underlying algorithms for efficient MA modelling, reduction using static analysis, and most importantly, quantitative analysis. We also discuss implementation pragmatics of supporting tools and present several case studies demonstrating feasibility and usability of MA in practice. ⋆ This work is funded by the EU FP7-projects MoVeS, SENSATION and MEALS, the DFG-NWO bilateral project ROCKS, the NWO projects SYRUP (grant 612.063.817), the STW project ArRangeer (grant 12238), and the DFG Sonderforschungsbereich AVACS. PreliminariesMarkov automata. An MA is a transition system with two types of transitions: probabilistic (as in PAs) and Markovian transitions (as in CTMCs). Let Act be a universe of actions with internal action τ ∈ Act, and Distr(S) denote the set of distribution functions over the countable set S.Definition 1 (Markov automaton). A Markov automaton (MA) is a tuple M = (S, A, − → , =⇒, s 0 ) where S is a nonempty, finite set of states with initial state s 0 ∈ S, A ⊆ Act is a finite set of actions, and -− → ⊆ S × A × Distr(S) is the probabilistic transition relation, and -=⇒ ⊆ S × R >0 × S is the Markovian transition relation.We abbreviate (s, α, µ) ∈ − → by s α − − → µ and (s, λ, s ′ ) ∈ =⇒ by s λ =⇒ s ′ . An MA can move between states via its probabilistic and Markovian transitions. If s a −→ µ, it can leave state s by executing the action a, after which the probability to go to some state s ′ ∈ S is given by µ(s ′ ). If s λ =⇒ s ′ , it moves from s to s ′ with rate λ, except if s enables a τ -labelled transition. In that case, the MA will always take such a transition and never delays. This is the maximal progress assumption [13]. The rationale behind this assumption is that internal transitions are not subject to interaction and thus can happen immediately, whereas 1 Stand-alone download as well as web-based interface available from

Modelling and Analysis of Markov Reward Automata

“…Typical examples where these algorithms can be used are respectively: to minimise the average energy consumption needed to download and install a medium-size software update; to minimise the average maintenance cost of a railroad line over the first year of deployment; and to maximise the yearly revenues of a data center over a long time horizon. In the following we lift the algorithms from [18] to the realm of rewards. We focus on maximising the properties.…”

“…In this section we focus on the second step. The first step can be performed by a graph-based algorithm [8,10] and the third step is as in [18].…”

“…To obtain the long-run average reward in an arbitrary MRA, we have to weigh the obtained long-run rewards in each maximal end component with the probability to reach those from s. This is equivalent to the third step in the long-run average computation of [18]. Further, for the discrete time setting [7] considers multiple long-run average objectives.…”

“…Notions of strong, weak and branching bisimulation were defined to equate behaviourally equivalent MAs [15,28,12,31], and the process-algebraic language MAPA was introduced for easily specifying large MAs in a concise manner [32]. Several types of reduction techniques [34,33] have been defined for the MAPA language and implemented in the tool SCOOP, optimising specifications to decrease the state space of the corresponding MAs while staying bisimilar [30,18]. This way, MAs can be generated efficiently in a direct way (as opposed to first generating a large model and then reducing), thus partly circumventing the omnipresent state space explosion.…”

“…Additionally, the game-based abstraction refinement technique developed in [6] provides a sound approximation of time-bounded reachability over a substantially reduced abstract model. The tool IMCA [17,18] was developed to analyse the concrete MAs that are generated by SCOOP. It includes algorithms for computing timebounded reachability probabilities, expected times and long-run averages for sets of goal states within an MA.…”