On Stochastic Games with Multiple Objectives

Abstract

Abstract. We study two-player stochastic games, where the goal of one player is to satisfy a formula given as a positive boolean combination of expected total reward objectives and the behaviour of the second player is adversarial. Such games are important for modelling, synthesis and verification of open systems with stochastic behaviour. We show that finding a winning strategy is PSPACE-hard in general and undecidable for deterministic strategies. We also prove that optimal strategies, if they exists, may require infinite memory and randomisation. However, when restricted to disjunctions of objectives only, memoryless deterministic strategies suffice, and the problem of deciding whether a winning strategy exists is NP-complete. We also present algorithms to approximate the Pareto sets of achievable objectives for the class of stopping games.

Multi-cost Bounded Reachability in MDP

“…For multi-objective analysis, the model checking community typically focuses on probabilities and expected costs as in the seminal works [15,22]. Implementations are typically based on a value iteration approach in [24], and have been extended to stochastic games [16], Markov automata [42], and interval MDPs [28]. Other considered cases include e.g.…”

PRISM-games: verification and strategy synthesis for stochastic multi-player games with multiple objectives

“…For multi-objective properties (which, again, only need to be considered on a stochastic two-player game), PRISMgames implements the techniques presented in [5,7,14,16]. The syntax described in Sect.…”

“…In particular, Boolean combinations of expectation objectives are converted to conjunctions by selecting appropriate weights for the individual objectives (see [14,Theorem 6] for details).…”

“…The underlying verification and strategy synthesis techniques developed for PRISM-games have been published elsewhere, in [5,7,12,14,16,44]. Existing short tool papers focusing on the functionality added in versions 1.0 and 2.0 of PRISM-games were presented in [13] and [34], respectively.…”

Quantitative Verification and Synthesis of Attack-Defence Scenarios

“…For attack-defence trees, it is natural to model the interactions between attacker and defender as a two-player game [11]. So, in our setting, where quantitative and probabilistic aspects are essential, we use stochastic two-player games [12], building upon the probabilistic model checking techniques for stochastic games proposed in [13], [14], and implemented in the PRISM-games model checking tool [15].…”