On the Semantics of Markov Automata

Deng, Yuxin; Hennessy, Matthew

doi:10.1007/978-3-642-22012-8_24

Cited by 32 publications

(50 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An appropriate version of bisimulation equivalence is shown to be compositional, in the sense of our Theorem 3.4: it is preserved by the operators of a process calculus interpreted as IMCs. Recently a combination of probabilistic automata and IMCs has been studied in [16,10], where again compositional theories based on weak bisimulation are proposed. Here there is some sense in which at least the time-delay actions have weights associated with them; nevertheless the intuition governing them, Markovian distributions, is entirely divorced from the notion of cost or benefit as we have used in the current paper.…”

Section: Y Deng M Hennessy / Science Of Computer Programming ( ) -mentioning

confidence: 99%

Compositional reasoning for weighted Markov decision processes

Deng

Hennessy

2013

Science of Computer Programming

Self Cite

View full text Add to dashboard Cite

h i g h l i g h t s• We develop compositional methods for reasoning about weighted Markov decision processes (MDPs).• A coinductive simulation-based behavioural preorder is proposed for weighted MDPs.• The preorder is preserved by structural operators for constructing weighted MDPs from components.• For finitary convergent processes, the preorder can be characterised by a probabilistic logic and a novel form of testing. a r t i c l e i n f o b s t r a c tWeighted Markov decision processes (MDPs) have long been used to model quantitative aspects of systems in the presence of uncertainty. However, much of the literature on such MDPs takes a monolithic approach, by modelling a system as a particular MDP; properties of the system are then inferred by analysis of that particular MDP. In contrast in this paper we develop compositional methods for reasoning about weighted MDPs, as a possible basis for compositional reasoning about their quantitative behaviour. In particular we approach these systems from a process algebraic point of view. For these we define a coinductive simulation-based behavioural preorder which is compositional in the sense that it is preserved by structural operators for constructing weighted MDPs from components.For finitary convergent processes, which are finite-state and finitely branching systems without divergence, we provide two characterisations of the behavioural preorder. The first uses a novel quantitative probabilistic logic, while the second is in terms of a novel form of testing, in which benefits are accrued during the execution of tests.

show abstract

Section: Y Deng M Hennessy / Science Of Computer Programming ( ) -mentioning

confidence: 99%

Compositional reasoning for weighted Markov decision processes

Deng

Hennessy

2013

Science of Computer Programming

Self Cite

View full text Add to dashboard Cite

show abstract

“…This now allows us to verify probabilistic as well as hard and soft real-time systems, modelled by timed automata, Markov decision processes, probabilistic automata, continuous-time Markov chains, interactive Markov chains, and Markov automata. Except for timed automatawhich incorporate real-time deadlines-all other models are subsumed by the Markov automaton (MA) [14,13,12]. MAs can therefore be used as a semantic model for a wide range of formalisms, such as generalised stochastic Petri nets (GSPNs) [2], dynamic fault trees [9], Arcade [8] and the domain-specific language AADL [10].…”

Section: Introductionmentioning

confidence: 99%

Confluence reduction for Markov automata

Timmer

Katoen

Pol

et al. 2016

Theoretical Computer Science

View full text Add to dashboard Cite

Abstract. Markov automata are a novel formalism for specifying systems exhibiting nondeterminism, probabilistic choices and Markovian rates. Recently, the process algebra MAPA was introduced to efficiently model such systems. As always, the state space explosion threatens the analysability of the models generated by such specifications. We therefore introduce confluence reduction for Markov automata, a powerful reduction technique to keep these models small. We define the notion of confluence directly on Markov automata, and discuss how to syntactically detect confluence on the MAPA language as well. That way, Markov automata generated by MAPA specifications can be reduced on-the-fly while preserving divergence-sensitive branching bisimulation. Three case studies demonstrate the significance of our approach, with reductions in analysis time up to an order of magnitude.

show abstract

“…Notions of strong, weak and branching bisimulation were defined to equate behaviourally equivalent MAs [15,28,12,31], and the process-algebraic language MAPA was introduced for easily specifying large MAs in a concise manner [32]. Several types of reduction techniques [34,33] have been defined for the MAPA language and implemented in the tool SCOOP, optimising specifications to decrease the state space of the corresponding MAs while staying bisimilar [30,18].…”

Section: Introductionmentioning

confidence: 99%

Modelling and Analysis of Markov Reward Automata

Guck

Timmer

Hatefi

et al. 2014

Automated Technology for Verification and Analysis

View full text Add to dashboard Cite

Abstract. Costs and rewards are important ingredients for many types of systems, modelling critical aspects like energy consumption, task completion, repair costs, and memory usage. This paper introduces Markov reward automata, an extension of Markov automata that allows the modelling of systems incorporating rewards (or costs) in addition to nondeterminism, discrete probabilistic choice and continuous stochastic timing. Rewards come in two flavours: action rewards, acquired instantaneously when taking a transition; and state rewards, acquired while residing in a state. We present algorithms to optimise three reward functions: the expected cumulative reward until a goal is reached, the expected cumulative reward until a certain time bound, and the long-run average reward. We have implemented these algorithms in the SCOOP/IMCA tool chain and show their feasibility via several case studies.

show abstract

On the Semantics of Markov Automata

Cited by 32 publications

References 21 publications

Compositional reasoning for weighted Markov decision processes

Compositional reasoning for weighted Markov decision processes

Confluence reduction for Markov automata

Modelling and Analysis of Markov Reward Automata

Contact Info

Product

Resources

About