Solving Large Extensive-Form Games with Strategy Constraints

Davis, Trevor; Waugh, Kevin; Bowling, Michael

doi:10.1609/aaai.v33i01.33011861

Cited by 6 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As an application, we show that the correctness and convergence rate of the CFR algorithm can be proven easily through our calculus. We also show that the recent Constrained CFR algorithm (Davis et al, 2019) can be constructed via our framework. Our framework enables the construction of two algorithms for that problem.…”

Section: Introductionmentioning

confidence: 93%

“…Farina et al (2019) study opponent exploitation where the goal is to compute a best response, subject to a penalty for moving away from a precomputed Nash equilibrium strategy; this is captured by having d 1 or d 2 include a penalty term that penalizes distance from the Nash equilibrium strategy. and Kroer et al (2017) study constraints on individual decision points, and Davis et al (2019) study additional constraints on the overall EFG polytopes X , Y. Regret minimization in those settings requires regret minimizers that can operate on more general domains X , Y than the sequence form.…”

Section: Connection To Convex-concave Saddle-pointmentioning

confidence: 99%

“…Such constraints break the recursive nature of the treeplex, and are thus not easily incorporated into standard regret-minimization or first-order methods for EFG solving. Davis et al (2019) propose a Lagrangian relaxation approach called Constrained CFR (CCFR): each strategy constraint is added to the objective with a Lagrangian multiplier, and a regret minimizer is used to penalize violation of the strategy constraints. They prove that if the regret minimizer for the Lagrange multipliers has the optimal Lagrangian multipliers as part of their strategy space, the average output strategy converges to an approximate solution to the constrained game.…”

Section: Application: Handling Strategy Constraintsmentioning

confidence: 99%

See 2 more Smart Citations

Regret Circuits: Composability of Regret Minimizers

Farina,

Kroer,

Sandholm

2018

Preprint

View full text Add to dashboard Cite

Regret minimization is a powerful tool for solving large-scale problems; it was recently used in breakthrough results for large-scale extensiveform game solving. This was achieved by composing simplex regret minimizers into an overall regret-minimization framework for extensiveform game strategy spaces. In this paper we study the general composability of regret minimizers. We derive a calculus for constructing regret minimizers for composite convex sets that are obtained from convexity-preserving operations on simpler convex sets. We show that local regret minimizers for the simpler sets can be combined with additional regret minimizers into an aggregate regret minimizer for the composite set. As one application, we show that the CFR framework can be constructed easily from our framework. We also show ways to include curtailing (constraining) operations into our framework. For one, they enables the construction of CFR generalization for extensive-form games with general convex strategy constraints that can cut across decision points.

show abstract

Section: Introductionmentioning

confidence: 93%

Section: Connection To Convex-concave Saddle-pointmentioning

confidence: 99%

Section: Application: Handling Strategy Constraintsmentioning

confidence: 99%

See 1 more Smart Citation

Regret Circuits: Composability of Regret Minimizers

Farina,

Kroer,

Sandholm

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…Related work also includes approaches for opponents which can change strategy over time (Powers & Shoham, 2005) and 3-player games (Ganzfried et al, 2018). Recent work introduces several forms of counterfactual regret minimization (Farina et al, 2019;Davis et al, 2019) and deep reinforcement learning (Brown et al, 2020) to find Nash equilibria. However, it has been shown that computing optimal strategies when playing a limited lookahead opponent in an imperfect-information game is NP-hard in all but the most restricted cases (Kroer & Sandholm, 2020).…”

Section: Opponent Modelingmentioning

confidence: 99%

A Survey of Opponent Modeling in Adversarial Domains

Nashed

Zilberstein

2022

jair

View full text Add to dashboard Cite

Opponent modeling is the ability to use prior knowledge and observations in order to predict the behavior of an opponent. This survey presents a comprehensive overview of existing opponent modeling techniques for adversarial domains, many of which must address stochastic, continuous, or concurrent actions, and sparse, partially observable payoff structures. We discuss all the components of opponent modeling systems, including feature extraction, learning algorithms, and strategy abstractions. These discussions lead us to propose a new form of analysis for describing and predicting the evolution of game states over time. We then introduce a new framework that facilitates method comparison, analyze a representative selection of techniques using the proposed framework, and highlight common trends among recently proposed methods. Finally, we list several open problems and discuss future research directions inspired by AI research on opponent modeling and related research in other disciplines.

show abstract

“…These methods exploit the hierarchical structure of the sequential strategy spaces of the players to construct a regret minimizer that recursively minimizes regret locally at each decision point in the game tree. This has inspired regret-based algorithms for other solution concepts in game theory, such as extensive-form perfect equilibria (Farina et al, 2017), Nash equilibrium with strategy constraints (Farina et al, 2017;b;Davis et al, 2019), and quantal-response equilibrium (Farina et al, 2019a).…”

Section: Introductionmentioning

confidence: 99%

Efficient Regret Minimization Algorithm for Extensive-Form Correlated Equilibrium

Farina,

Ling,

Fang

et al. 2019

Preprint

View full text Add to dashboard Cite

Self-play methods based on regret minimization have become the state of the art for computing Nash equilibria in large two-players zero-sum extensive-form games. These methods fundamentally rely on the hierarchical structure of the players' sequential strategy spaces to construct a regret minimizer that recursively minimizes regret at each decision point in the game tree. In this paper, we introduce the first efficient regret minimization algorithm for computing extensive-form correlated equilibria in large two-player general-sum games with no chance moves. Designing such an algorithm is significantly more challenging than designing one for the Nash equilibrium counterpart, as the constraints that define the space of correlation plans lack the hierarchical structure and might even form cycles. We show that some of the constraints are redundant and can be excluded from consideration, and present an efficient algorithm that generates the space of extensive-form correlation plans incrementally from the remaining constraints. This structural decomposition is achieved via a special convexity-preserving operation that we coin scaled extension. We show that a regret minimizer can be designed for a scaled extension of any two convex sets, and that from the decomposition we then obtain a global regret minimizer. Our algorithm produces feasible iterates. Experiments show that it significantly outperforms prior approaches and for larger problems it is the only viable option.

show abstract

Solving Large Extensive-Form Games with Strategy Constraints

Cited by 6 publications

References 11 publications

Regret Circuits: Composability of Regret Minimizers

Regret Circuits: Composability of Regret Minimizers

A Survey of Opponent Modeling in Adversarial Domains

Efficient Regret Minimization Algorithm for Extensive-Form Correlated Equilibrium

Contact Info

Product

Resources

About