A Modest Markov Automata Tutorial

Abstract

Distributed computing systems provide many important services. To explain and understand why and how well they work, it is common practice to build, maintain, and analyse models of the systems' behaviours. Markov models are frequently used to study operational phenomena of such systems. They are often represented with discrete state spaces, and come in various flavours, overarched by Markov automata. As such, Markov automata provide the ingredients that enable the study of a wide range of quantitative properties related to risk, cost, performance, and strategy. This tutorial paper gives an introduction to the formalism of Markov automata, to practical modelling of Markov automata in the Modest language, and to their analysis with the Modest Toolset. As case studies, we optimise an attack on Bitcoin, and evaluate the performance of a small but complex resource-sharing computing system.Authors are listed alphabetically. This work has received financial support by DFG grant 389792660 as part of TRR 248 (see perspicuous-computing.science), by ERC Advanced Grant 69561 (POWVER), and by NWO VENI grant 639.021.754.

Machine Learning Algorithms Used in PSE Environments: A Didactic Approach and Critical Perspective

“…280 Commonly, the agent responses and decision-making are addressed using the Markov decision process (MDP). It is a well-known algorithm addressed in the works of Hartmanns and Hermanns 281 and White and White. 282 Other popular methods used for reinforcement learning are Monte Carlo search trees.…”

A Modest Approach to Markov Automata

“…We have now covered most of the basic constructs of Modest. There are many features not used in this small model; we refer the interested reader to our extended tutorial on Modest for MA [51], where we continue by tackling two different realistic case studies with Modest in a stepby-step fashion. Additional Modest MA models are also part of the Quantitative Verification Benchmark Set (QVBS) [53] at qcomp.org.…”

“…This article extends upon our conference paper [22] presented at the 16th International Conference on Quantitative Evaluation of Systems (QEST 2019). We have expanded explanations throughout and incorporated a step-by-step introduction to modelling MA with Modest in Section 3 extracted from the larger tutorial in Reference [51]. We consolidated the information and experiments concerning probabilistic model checking in Section 4.…”