Games and Markov Decision Processes with Mean-Payoff Parity and Energy Parity Objectives

Chatterjee, Krishnendu; Doyen, Laurent

doi:10.1007/978-3-642-25929-6_3

Cited by 13 publications

(11 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Classical models. Two-player zero-sum quantitative games [17,31,8] and Markov decision processes (MDPs) [27,11] are two popular formalisms for modeling decision making in adversarial and uncertain environments respectively. In the former, two players compete with opposite goals (zero-sum), and we want strategies for player 1 (the system) that ensure a given minimal performance against all possible strategies of player 2 (its environment).…”

Section: Introductionmentioning

confidence: 99%

Expectations or Guarantees? I Want It All! A crossroad between games and MDPs

Bruyère

Filiot

Randour

et al. 2014

Electron. Proc. Theor. Comput. Sci.

View full text Add to dashboard Cite

When reasoning about the strategic capabilities of an agent, it is important to consider the nature of its adversaries. In the particular context of controller synthesis for quantitative specifications, the usual problem is to devise a strategy for a reactive system which yields some desired performance, taking into account the possible impact of the environment of the system. There are at least two ways to look at this environment. In the classical analysis of two-player quantitative games, the environment is purely antagonistic and the problem is to provide strict performance guarantees. In Markov decision processes, the environment is seen as purely stochastic: the aim is then to optimize the expected payoff, with no guarantee on individual outcomes.In this expository work, we report on recent results [10,9] introducing the beyond worst-case synthesis problem, which is to construct strategies that guarantee some quantitative requirement in the worst-case while providing an higher expected value against a particular stochastic model of the environment given as input. This problem is relevant to produce system controllers that provide nice expected performance in the everyday situation while ensuring a strict (but relaxed) performance threshold even in the event of very bad (while unlikely) circumstances. It has been studied for both the mean-payoff and the shortest path quantitative measures. *

show abstract

Section: Introductionmentioning

confidence: 99%

Expectations or Guarantees? I Want It All! A crossroad between games and MDPs

Bruyère

Filiot

Randour

et al. 2014

Electron. Proc. Theor. Comput. Sci.

View full text Add to dashboard Cite

show abstract

“…A strategy is said optimal for the mean-payoff objective if it achieves this value. Such a strategy may not need to exist in general, even in one-player games [22,11,18] (Fig. 8, P 1 has to delay its visits of s 1 for longer and longer intervals in order to tend towards value 1).…”

Section: Randomization and Single Mean-payoff Parity Gamesmentioning

confidence: 99%

Strategy Synthesis for Multi-Dimensional Quantitative Objectives

Chatterjee

Randour

Raskin

2012

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. Multi-dimensional mean-payoff and energy games provide the mathematical foundation for the quantitative study of reactive systems, and play a central role in the emerging quantitative theory of verification and synthesis. In this work, we study the strategy synthesis problem for games with such multi-dimensional objectives along with a parity condition, a canonical way to express ω-regular conditions. While in general, the winning strategies in such games may require infinite memory, for synthesis the most relevant problem is the construction of a finite-memory winning strategy (if one exists). Our main contributions are as follows. First, we show a tight exponential bound (matching upper and lower bounds) on the memory required for finite-memory winning strategies in both multi-dimensional mean-payoff and energy games along with parity objectives. This significantly improves the triple exponential upper bound for multi energy games (without parity) that could be derived from results in literature for games on VASS (vector addition systems with states). Second, we present an optimal symbolic and incremental algorithm to compute a finite-memory winning strategy (if one exists) in such games. Finally, we give a complete characterization of when finite memory of strategies can be traded off for randomness. In particular, we show that for one-dimension mean-payoff parity games, randomized memoryless strategies are as powerful as their pure finite-memory counterparts.

show abstract

“…More recently they were also shown to be solvable in pseudo-quasi-polynomial time [26]. Related results on non-stochastic games (e.g., mean-payoff parity) are summarized in [18].…”

Section: Introductionmentioning

confidence: 99%

“…Most existing work on combined objectives for stochastic systems, for example [17,18,9,40], is restricted to Markov decision processes (MDPs; aka 1 1 2 -player games). Almost-sure energy-parity objectives for MDPs were first considered in [17,18], where a direct reduction to ordinary energy games was proposed. This reduction relies on the assumption that maximizer can win using finite memory if at all.…”

Section: Introductionmentioning

confidence: 99%

Simple Stochastic Games with Almost-Sure Energy-Parity Objectives are in NP and coNP

Mayr

Schewe

Totzke

et al. 2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We study stochastic games with energy-parity objectives, which combine quantitative rewards with a qualitative $$\omega $$ ω -regular condition: The maximizer aims to avoid running out of energy while simultaneously satisfying a parity condition. We show that the corresponding almost-sure problem, i.e., checking whether there exists a maximizer strategy that achieves the energy-parity objective with probability 1 when starting at a given energy level k, is decidable and in $$\mathsf {NP}\cap \mathsf {coNP}$$ NP ∩ coNP . The same holds for checking if such a k exists and if a given k is minimal.

show abstract

Games and Markov Decision Processes with Mean-Payoff Parity and Energy Parity Objectives

Cited by 13 publications

References 26 publications

Expectations or Guarantees? I Want It All! A crossroad between games and MDPs

Expectations or Guarantees? I Want It All! A crossroad between games and MDPs

Strategy Synthesis for Multi-Dimensional Quantitative Objectives

Simple Stochastic Games with Almost-Sure Energy-Parity Objectives are in NP and coNP

Contact Info

Product

Resources

About