Qualitative reachability in stochastic BPA games

Brázdil, Tomǎš; Broek, VáClav; Kučera, Antonín; Obdrálek, Jan

doi:10.1016/j.ic.2011.02.002

Cited by 28 publications

(57 citation statements)

References 15 publications

(44 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One of the most wellstudied model of infinite-state games with qualitative objectives are pushdown games (or games on recursive state machines) that can model reactive systems with recursion (or model the control flow of sequential programs with recursion). Pushdown games with reachability and parity objectives have been studied in [33,32,5,4] (also see [19,20,10,9] for sample research in stochastic pushdown games). The most well-studied quantitative objective is the mean-payoff objective, where a reward is associated Table 1.…”

Section: Introductionmentioning

confidence: 99%

Mean-Payoff Pushdown Games

Chatterjee,

Velner

2012

2012 27th Annual IEEE Symposium on Logic in Computer Science

View full text Add to dashboard Cite

Abstract. Two-player games on graphs are central in many problems in formal verification and program analysis such as synthesis and verification of open systems. In this work we consider solving recursive game graphs (or pushdown game graphs) that can model the control flow of sequential programs with recursion. While pushdown games have been studied before with qualitative objectives, such as reachability and ω-regular objectives, in this work we study for the first time such games with the most well-studied quantitative objective, namely, mean-payoff objectives. In pushdown games two types of strategies are relevant: (1) global strategies, that depend on the entire global history; and (2) modular strategies, that have only local memory and thus do not depend on the context of invocation, but only on the history of the current invocation of the module. Our main results are as follows: (1) One-player pushdown games with mean-payoff objectives under global strategies are decidable in polynomial time. (2) Twoplayer pushdown games with mean-payoff objectives under global strategies are undecidable. (3) One-player pushdown games with mean-payoff objectives under modular strategies are NPhard. (4) Two-player pushdown games with mean-payoff objectives under modular strategies can be solved in NP (i.e., both one-player and two-player pushdown games with mean-payoff objectives under modular strategies are NP-complete). We also establish the optimal strategy complexity showing that global strategies for mean-payoff objectives require infinite memory even in one-player pushdown games; and memoryless modular strategies are sufficient in twoplayer pushdown games. Finally we also show that all the problems have the same complexity if the stack boundedness condition is added, where along with the mean-payoff objective the player must also ensure that the stack height is bounded.

show abstract

Section: Introductionmentioning

confidence: 99%

Mean-Payoff Pushdown Games

Chatterjee,

Velner

2012

2012 27th Annual IEEE Symposium on Logic in Computer Science

View full text Add to dashboard Cite

show abstract

“…If we further assume that |m (i+1) − m (i) | c almost surely for all i 0, we can apply the Azuma-Hoeffding inequality 5 , which says that the following holds for all t > 0 and n 0:…”

Section: Bounding Counter Value N For Maximizing Oc-mdpsmentioning

confidence: 99%

“…It was also shown in [10] that for stochastic context-free games (1-exit RSSGs), which correspond to pushdown stochastic games with only one state, both qualitative and quantitative termination problems are decidable, and in fact qualitative termination problems are decidable in NP ∩ coNP [11], while quantitative termination problems are decidable in PSPACE. Solving termination objectives is a key ingredient for many more general analyses and model checking problems for such stochastic games (see, e.g., [4,5]). OC-SSGs are incompatible with stochastic context-free games.…”

Section: Introductionmentioning

confidence: 99%

Approximating the termination value of one-counter MDPs and stochastic games

Brázdil

Broek

Etessami

et al. 2013

Information and Computation

Self Cite

View full text Add to dashboard Cite

One-counter MDPs (OC-MDPs) and one-counter simple stochastic games (OC-SSGs) are 1-player, and 2-player turn-based zero-sum, stochastic games played on the transition graph of classic one-counter automata (equivalently, pushdown automata with a 1-letter stack alphabet). A key objective for the analysis and verification of these games is the termination objective, where the players aim to maximize (minimize, respectively) the probability of hitting counter value 0, starting at a given control state and given counter value.Recently, we studied qualitative decision problems ("is the optimal termination value equal to 1?") for OC-MDPs (and OC-SSGs) and showed them to be decidable in polynomial time (in NP ∩ coNP, respectively). However, quantitative decision and approximation problems ("is the optimal termination value at least p", or "approximate the termination value within ε") are far more challenging. This is so in part because optimal strategies may not exist, and because even when they do exist they can have a highly non-trivial structure.It thus remained open even whether any of these quantitative termination problems are computable. In this paper we show that all quantitative approximation problems for the termination value for OC-MDPs and OC-SSGs are computable. Specifically, given an OC-SSG, and given ε > 0, we can compute a value v that approximates the value of the OC-SSG termination game within additive error ε, and furthermore we can compute ε-optimal strategies for both players in the game.A key ingredient in our proofs is a subtle martingale, derived from solving certain linear programs that we can associate with a maximizing OC-MDP. An application of Azuma's inequality on these martingales yields a computable bound for the "wealth" at which a "rich person's strategy" becomes ε-optimal for OC-MDPs.

show abstract

“…Related work: One-counter automata can also be seen as pushdown automata with one letter stack alphabet. Stochastic games and MPDs generated by pushdown automata and stateless pushdown automata (also known as BPA) with termination and reachability objectives have been studied in [13,14,6,7]. To the best of our knowledge, the only prior work on the expected termination time (or, more generally, total accumulated reward) objective for a class of infinite-state MDPs or stochastic games is [11], where this problem is studied for stochastic BPA games.…”

Section: Introductionmentioning

confidence: 99%

Minimizing Expected Termination Time in One-Counter Markov Decision Processes

Brázdil

Kučera

Novotný

et al. 2012

Automata, Languages, and Programming

Self Cite

View full text Add to dashboard Cite

Abstract.We consider the problem of computing the value and an optimal strategy for minimizing the expected termination time in one-counter Markov decision processes. Since the value may be irrational and an optimal strategy may be rather complicated, we concentrate on the problems of approximating the value up to a given error ε > 0 and computing a finite representation of an ε-optimal strategy. We show that these problems are solvable in exponential time for a given configuration, and we also show that they are computationally hard in the sense that a polynomial-time approximation algorithm cannot exist unless P=NP.

show abstract

Qualitative reachability in stochastic BPA games

Cited by 28 publications

References 15 publications

Mean-Payoff Pushdown Games

Mean-Payoff Pushdown Games

Approximating the termination value of one-counter MDPs and stochastic games

Minimizing Expected Termination Time in One-Counter Markov Decision Processes

Contact Info

Product

Resources

About