Expectations or Guarantees? I Want It All! A crossroad between games and MDPs

Abstract

When reasoning about the strategic capabilities of an agent, it is important to consider the nature of its adversaries. In the particular context of controller synthesis for quantitative specifications, the usual problem is to devise a strategy for a reactive system which yields some desired performance, taking into account the possible impact of the environment of the system. There are at least two ways to look at this environment. In the classical analysis of two-player quantitative games, the environment is purely antagonistic and the problem is to provide strict performance guarantees. In Markov decision processes, the environment is seen as purely stochastic: the aim is then to optimize the expected payoff, with no guarantee on individual outcomes.In this expository work, we report on recent results [10,9] introducing the beyond worst-case synthesis problem, which is to construct strategies that guarantee some quantitative requirement in the worst-case while providing an higher expected value against a particular stochastic model of the environment given as input. This problem is relevant to produce system controllers that provide nice expected performance in the everyday situation while ensuring a strict (but relaxed) performance threshold even in the event of very bad (while unlikely) circumstances. It has been studied for both the mean-payoff and the shortest path quantitative measures. *

Meet your expectations with guarantees: Beyond worst-case synthesis in quantitative games

“…Related work. This paper extends previous works presented in conferences [8,7] and in a PhD thesis [37]: it gives a full presentation of the technical details, along with additional results. Our problems generalize the corresponding problems for two-player zero-sum games and MDPs.…”

“…prescribes acting like λ cmb 1 , which induces satisfaction7 More complex switching schemes could be used, such as only switching if the edge taken is really dangerous (i.e., part of a non strictly positive cycle), switching after a bounded number of deviations from the support, etc. But this simple scheme proves to be sufficient to realize Thm.…”

Non-Zero Sum Games for Reactive Synthesis

“…Hence this shows that the classical solution concepts do not suffice if one wants to go beyond the worst-case and mix guarantees on the worst-case and the expected performance of strategies. In contrast, with the framework developed in [13,12], it is indeed possible for the considered arena ( Fig. 4) to build a strategy for Eve that ensures the worst-case constraint (C 1 ) and at the same time, yields an expected value arbitrarily close to the optimal expectation achieved by strategy σ exp ∃ .…”

“…Games with an expected adversary In [13,12,22], we combined the classical formalism of two-player zero-sum games (where the environment is con-sidered to be completely antagonistic) with Markov decision processes (MDPs), a well-known model for decision-making inside a stochastic environment. The motivation is that one has often a good idea of the expected behavior (i.e., average-case) of the environment represented as a stochastic model based on statistical data such as the frequency of requests for a computer server, the average traffic in a town, etc.…”

Variations on the Stochastic Shortest Path Problem

“…Additional results. In [5,4], we also study the so-called beyond worst-case synthesis for models with the mean-payoff function instead of the truncated sum. Mean-payoff games [12] are infinite-duration, two-player zero-sum games played on weighted graphs.…”