Automatic verification of concurrent stochastic systems

Abstract

Automated verification techniques for stochastic games allow formal reasoning about systems that feature competitive or collaborative behaviour among rational agents in uncertain or probabilistic settings. Existing tools and techniques focus on turn-based games, where each state of the game is controlled by a single player, and on zero-sum properties, where two players or coalitions have directly opposing objectives. In this paper, we present automated verification techniques for concurrent stochastic games (CSGs), which provide a more natural model of concurrent decision making and interaction. We also consider (social welfare) Nash equilibria, to formally identify scenarios where two players or coalitions with distinct goals can collaborate to optimise their joint performance. We propose an extension of the temporal logic rPATL for specifying quantitative properties in this setting and present corresponding algorithms for verification and strategy synthesis for a variant of stopping games. For finite-horizon properties the computation is exact, while for infinite-horizon it is approximate using value iteration. For zero-sum properties it requires solving matrix games via linear programming, and for equilibria-based properties we find social welfare or social cost Nash equilibria of bimatrix games via the method of labelled polytopes through an SMT encoding. We implement this approach in PRISM-games, which required extending the tool’s modelling language for CSGs, and apply it to case studies from domains including robotics, computer security and computer networks, explicitly demonstrating the benefits of both CSGs and equilibria-based properties.

Strategy Synthesis for Zero-sum Neuro-symbolic Concurrent Stochastic Games (Extended Version)

“…where 𝛽 ∈ (0, 1) is the discount factor. Solving a zero-sum NS-CSG reduces to the problem of solving a 2-agent NS-CSG (which we can formalise with the definition of a CSG coalition game [19]). While coalitions remain useful from a modelling perspective, and for the definition of temporal logic specifications [19], to simplify presentation in this paper we will simply restrict our attention to NS-CSGs with 2 agents.…”

“…An important class of dynamic games is stochastic games [35], which move between states according to transition probabilities controlled jointly by multiple agents (players). Extending both strategicform games to dynamic environments and Markov decision processes to multiple players, stochastic games have long been used to model sequential decision-making problems with more than one agent, ranging from multi-agent reinforcement learning [40], to quantitative verification and synthesis for equilibria [19] and economics [1].…”

Rational verification: game-theoretic verification of multi-agent systems

“…A separate though closely related thread of research into the verification of multi-agent systems has emerged from the probabilistic model-checking community. The most prominent example of this in recent years is the expansion of PRISM [54], a popular tool for probabilistic model-checking, to handle first Turn-Based [11] and now Concurrent Stochastic Games (Markov Games) [55,56]. Earlier work was limited to non-cooperative turn-based or zero-sum concurrent settings.…”

“…Earlier work was limited to non-cooperative turn-based or zero-sum concurrent settings. Later efforts considering cooperative, concurrent games were initially restricted to those with only two coalitions, but this restriction has been partially lifted in the most recent instantiation of the work, which supports model-checking of arbitrary numbers of coalitions in the special case of stopping games -those in which eventually, with probability one, the outcome of each player's objective becomes fixed [56]. We note further that the current version of the tool also supports the use of Probabilistic Timed Automata in verifying Turn-Based Markov Games with real-valued clocks [57].…”

“…-where ψ and χ are LTL formulae and similarly for expected rewards. More information can be found in [56]. An alternative specification formalism that can express equilibria concepts is Probabilistic Strategy Logic [8], but it has no associated implementation.…”