Playing Stochastic Games Precisely

Chen, Taolue; Forejt, Vojtěch; Kwiatkowska, Marta; Simaitis, Aistis; Trivedi, Ashutosh; Ummels, Michael

doi:10.1007/978-3-642-32940-1_25

Cited by 14 publications

(17 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…-We provide Pareto set approximation algorithms for stopping games. This result directly applies to the important class of discounted rewards for nonstopping games, due to an off-the-shelf reduction [8].…”

Section: Introductionmentioning

confidence: 81%

“…As a corollary of the theorem, using a simple reduction (see e.g. [8]) we get an approximation algorithm for the Pareto sets in non-stopping games with (multiple) discounted reward objectives.…”

Section: ·2mentioning

confidence: 97%

“…A stochastic game G, together with a strategy pair (π, σ) ∈ Π × Σ and a starting state s, induces an infinite Markov chain on the game (see e.g. [8]). We define the probability measure of this Markov chain by Pr π,σ G,s .…”

Section: Stochastic Gamesmentioning

confidence: 99%

“…Theorem 1 (Non-determinacy, optimal strategies [8]). Stochastic games with multiple objectives are, in general, not determined, and optimal strategies might not exist, already for CQs with two objectives.…”

Section: Conjunctions Of Objectivesmentioning

confidence: 99%

“…However, already for MDPs, such logics are so powerful that it is not decidable whether there is an optimal controller [3]. A special case of the problem studied in this paper is the case where the goal of Player 1 is to achieve a precise value of the expectation of an objective function [8]. As regards applications, stochastic games with a single objective function have been employed and implemented for quantitative abstraction refinement for MDP models in [16].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

On Stochastic Games with Multiple Objectives

Chen

Forejt

Kwiatkowska

et al. 2013

Mathematical Foundations of Computer Science 2013

Self Cite

129

View full text Add to dashboard Cite

Abstract. We study two-player stochastic games, where the goal of one player is to satisfy a formula given as a positive boolean combination of expected total reward objectives and the behaviour of the second player is adversarial. Such games are important for modelling, synthesis and verification of open systems with stochastic behaviour. We show that finding a winning strategy is PSPACE-hard in general and undecidable for deterministic strategies. We also prove that optimal strategies, if they exists, may require infinite memory and randomisation. However, when restricted to disjunctions of objectives only, memoryless deterministic strategies suffice, and the problem of deciding whether a winning strategy exists is NP-complete. We also present algorithms to approximate the Pareto sets of achievable objectives for the class of stopping games.

show abstract

Section: Introductionmentioning

confidence: 81%

Section: ·2mentioning

confidence: 97%

Section: Stochastic Gamesmentioning

confidence: 99%

Section: Conjunctions Of Objectivesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

On Stochastic Games with Multiple Objectives

Chen

Forejt

Kwiatkowska

et al. 2013

Mathematical Foundations of Computer Science 2013

Self Cite

129

View full text Add to dashboard Cite

show abstract

Run-Time Optimization for Learned Controllers Through Quantitative Games

Avni

Bloem

Chatterjee

et al. 2019

Computer Aided Verification

View full text Add to dashboard Cite

A controller is a device that interacts with a plant. At each time point, it reads the plant's state and issues commands with the goal that the plant operates optimally. Constructing optimal controllers is a fundamental and challenging problem. Machine learning techniques have recently been successfully applied to train controllers, yet they have limitations. Learned controllers are monolithic and hard to reason about. In particular, it is difficult to add features without retraining, to guarantee any level of performance, and to achieve acceptable performance when encountering untrained scenarios. These limitations can be addressed by deploying quantitative run-time shields that serve as a proxy for the controller. At each time point, the shield reads the command issued by the controller and may choose to alter it before passing it on to the plant. We show how optimal shields that interfere as little as possible while guaranteeing a desired level of controller performance, can be generated systematically and automatically using reactive synthesis. First, we abstract the plant by building a stochastic model. Second, we consider the learned controller to be a black box. Third, we measure controller performance and shield interference by two quantitative run-time measures that are formally defined using weighted automata. Then, the problem of constructing a shield that guarantees maximal performance with minimal interference is the problem of finding an optimal strategy in a stochastic 2-player game "controller versus shield" played on the abstract state space of the plant with a quantitative objective obtained from combining the performance and interference measures. We illustrate the effectiveness of our approach by automatically constructing lightweight shields for learned traffic-light controllers in various road networks. The shields we generate avoid liveness bugs, improve controller performance in untrained and changing traffic situations, and add features to learned controllers, such as giving priority to emergency vehicles.

show abstract