Lossy stochastic game abstraction with bounds

Abstract

Abstraction followed by equilibrium finding has emerged as the leading approach to solving games. Lossless abstraction typically yields games that are still too large to solve, so lossy abstraction is needed. Unfortunately, prior lossy game abstraction algorithms have no guarantees on solution quality. We developed a framework that enables the design of lossy game abstraction algorithms with guarantees on solution quality. It simultaneously handles state and action abstraction. We define a measure of reward approximation error and transition probability error achieved by state and action abstraction in stochastic games such that the regret of the equilibrium found in the abstract game when implemented in the original, unabstracted game is upper-bounded by a function of those measures. We then develop the first lossy game abstraction algorithms with bounds on solution quality. Both of them work level-by-level up from the end of the game. One of the algorithms is greedy and the other is an integer linear program. We also prove that the abstraction problem is NP-complete (even with just action abstraction, 2 agents, and a 1-step game), but point out that this does not mean that the game abstraction problems that occur in practice cannot be solved quickly.

Extensive-form game abstraction with bounds

“…As a special case, our results generalize the results of Sandholm and Singh [2012], since any DAG-structured stochastic game can be converted to an extensive form game. We also tighten the bound given in their paper.…”

“…Here we introduce two classes of mappings from abstract strategies to real strategies. First, we present the natural extension of lifted strategies introduced by Sandholm and Singh [2012] to extensive-form games. In this definition, a valid mapping σ ↑σ is any strategy profile such that for each information set I and abstract action a ∈ A f (I) , the probability σ (f (I), a ) of selecting that abstract action is divided any way across the actions g −1 (a ) ∩ I.…”

“…Then, the strategy mapping just consists of playing the strategy computed in the restricted game and ignoring the remaining actions. All prior action abstraction algorithms have operated within that family (e.g., [Gilpin et al 2008;Schnizlein et al 2009;Hawkin et al 2011Hawkin et al , 2012Sandholm and Singh 2012;Ganzfried and Sandholm 2013;Brown and Sandholm 2014]). Such lifting (as long as the information abstraction is done within the framework of our paper described above) is a form of undivided strategy lifting.…”

“…Their result depends on the counterfactual regret minimization (CFR) algorithm being used for equilibrium computation in the abstraction, and focuses on abstraction via information coarsening, thus allowing neither action abstraction nor reduction of the size of the game tree in terms of nodes. Sandholm and Singh [2012] provide lossy abstraction algorithms with bounds for stochastic games. They leave as an open problem whether similar bounds can be achieved for extensive-form games.…”

Faster First-Order Methods for Extensive-Form Game Solving

“…Lately, action abstraction approaches have gained considerable interest [Hawkin et al 2011[Hawkin et al , 2012Brown and Sandholm 2014;Kroer and Sandholm 2014a,b]. Sequential game abstraction approaches with solution quality bounds have also emerged for stochastic [Sandholm and Singh 2012] and extensive-form [Lanctot et al 2012;Kroer and Sandholm 2014a,b] games more recently.…”