Simple Uncoupled No-Regret Learning Dynamics for Extensive-Form Correlated Equilibrium

Farina, Gabriele; Celli, Andrea; Marchesi, Alberto; Gatti, Nicola

doi:10.48550/arxiv.2104.01520

Cited by 5 publications

(14 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Later, Farina et al (2019) propose a min-max optimization formulation of EFCEs which can be solved by first-order methods. Celli et al (2020) and its extended version (Farina et al, 2021a) design the first uncoupled no-regret algorithm for computing EFCEs. Their algorithms are based on minimizing the trigger regret (first considered in Dudik and Gordon (2012); Gordon et al (2008)) via counterfactual regret decomposition (Zinkevich et al, 2007).…”

Section: Related Workmentioning

confidence: 99%

“…The iteration complexity for computing an ε-approximate correlated equilibrium in both (Farina et al, 2021a;Morrill et al, 2021) scales quadratically in max i∈[m] X i . Our K-EFR algorithm for the full feedback setting builds upon the EFR algorithm, but specializes to the notion of K-EFCE, and achieve an improved linear in max i∈[m] X i iteration complexity.…”

Section: Related Workmentioning

confidence: 99%

“…An EFCE is a correlated policy that can be thought of as a "mediator" of the game who recommends actions to each player privately and sequentially (at visited information sets), in a way that disincentivizes any player to deviate from the recommended actions. Polynomial-time algorithms for computing EFCEs have been established, by formulating as a linear program and using the ellipsoid method (Huang and von Stengel, 2008;Papadimitriou and Roughgarden, 2008;Jiang and Leyton-Brown, 2015), min-max optimization (Farina et al, 2019), or uncoupled no-regret dynamics using variants of counterfactual regret minimization (Celli et al, 2020;Farina et al, 2021a;Morrill et al, 2021;Anagnostides et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Sample-Efficient Learning of Correlated Equilibria in Extensive-Form Games

Song¹,

Song²

2022

Preprint

View full text Add to dashboard Cite

Imperfect-Information Extensive-Form Games (IIEFGs) is a prevalent model for real-world games involving imperfect information and sequential plays. The Extensive-Form Correlated Equilibrium (EFCE) has been proposed as a natural solution concept for multi-player general-sum IIEFGs. However, existing algorithms for finding an EFCE require full feedback from the game, and it remains open how to efficiently learn the EFCE in the more challenging bandit feedback setting where the game can only be learned by observations from repeated playing.This paper presents the first sample-efficient algorithm for learning the EFCE from bandit feedback. We begin by proposing K-EFCE-a more generalized definition that allows players to observe and deviate from the recommended actions for K times. The K-EFCE includes the EFCE as a special case at K = 1, and is an increasingly stricter notion of equilibrium as K increases. We then design an uncoupled noregret algorithm that finds an ε-approximate K-EFCE within O(maxi XiA K i /ε 2 ) iterations in the full feedback setting, where Xi and Ai are the number of information sets and actions for the i-th player. Our algorithm works by minimizing a wide-range regret at each information set that takes into account all possible recommendation histories. Finally, we design a sample-based variant of our algorithm that learns an ε-approximate K-EFCE within O(maxi XiA K+1 i /ε 2 ) episodes of play in the bandit feedback setting. When specialized to K = 1, this gives the first sample-efficient algorithm for learning EFCE from bandit feedback.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Sample-Efficient Learning of Correlated Equilibria in Extensive-Form Games

Song¹,

Song²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Moreover, given that no-external regret learning dynamics converge to the set of coarse correlated equilibria (see [8,20]), Blum et al [9] study the price of total anarchy in games in which players take decisions so as to minimize their external regret. No-regret learning dynamics converging to correlated equilibria have been studied in various settings (see, e.g., [36,42,18,19,32]). Moreover, Hartline et al [43] and Caragiannis et al [15] study the quality of outcomes emerging from no-regret dynamics in Bayesian settings.…”

Section: Related Workmentioning

confidence: 99%

Learning Fair Equilibria in Sponsored Search Auctions

Birmpas¹,

Celli²,

Colini-Baldeschi³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In this work we investigate the strategic learning implications of the deployment of sponsored search auction mechanisms that obey to fairness criteria. We introduce a new class of mechanisms composing a traditional Generalized Second Price auction (GSP) with different fair division schemes to achieve some desired level of fairness between two groups of Bayesian strategic advertisers. We propose two mechanisms, β-Fair GSP and GSP-EFX, that compose GSP with, respectively, an envy-free up to one item (EF1), and an envy-free up to any item (EFX) fair division scheme. The payments of GSP are adjusted in order to compensate the advertisers that suffer a loss of efficiency due the fair division stage. We prove that, for both mechanisms, if bidders play so as to minimize their external regret they are guaranteed to reach an equilibrium with good social welfare. We also prove that the mechanisms are budget balanced, so that the payments charged by the traditional GSP mechanism are a good proxy of the total compensation offered to the advertisers. Finally, we evaluate the quality of the allocations of the two mechanisms through experiments on real-world data.

show abstract

“…The problem of computing one EFCE (and, therefore, one NFCCE/EFCCE) can be solved in polynomial time in the size of the game tree [Huang and von Stengel, 2008] via a variation of the Ellipsoid Against Hope algorithm [Jiang andLeyton-Brown, 2015, Papadimitriou andRoughgarden, 2008]. Moreover, there exist decentralized no-regret learning dynamics guaranteeing that the empirical frequency of play after 𝑇 rounds is an 𝑂 (1/ √ 𝑇 )-approximate EFCE with high probability, and an EFCE almost surely in the limit [Celli et al, 2020, Farina et al, 2021b. Furthermore, NFCCE is the equilibrium notion that gets satisfied when all players in a game play according to any regret minimizer.…”

Section: Introductionmentioning

confidence: 99%

Optimal Correlated Equilibria in General-Sum Extensive-Form Games: Fixed-Parameter Algorithms, Hardness, and Two-Sided Column-Generation

Zhang¹,

Farina²,

Celli³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

We study the problem of finding optimal correlated equilibria of various sorts: normal-form coarse correlated equilibrium (NFCCE), extensive-form coarse correlated equilibrium (EFCCE), and extensive-form correlated equilibrium (EFCE). This is NP-hard in the general case and has been studied in special cases, most notably triangle-free games , which include all two-player games with public chance moves. However, the general case is not well understood, and algorithms usually scale poorly. In this paper, we make two primary contributions.First, we introduce the correlation DAG, a representation of the space of correlated strategies whose structure and size are dependent on the specific solution concept desired. It extends the team belief DAG of Zhang et al.[2022] to general-sum games. For each of the three solution concepts, its size depends exponentially only on a parameter related to the information structure of the game. We also prove a fundamental complexity gap: while our size bounds for NFCCE are similar to those achieved in the case of team games by Zhang et al. [2022], this is impossible to achieve for the other two concepts under standard complexity assumptions.Second, we propose a two-sided column generation approach to compute optimal correlated strategies in extensive-form games. Our algorithm improves upon the one-sided approach of Farina et al. [2021a] by means of a new decomposition of correlated strategies which allows players to re-optimize their sequence-form strategies with respect to correlation plans which were previously added to the support.Experiments show that our techniques outperform the prior state of the art for computing optimal generalsum correlated equilibria, and that our two families of approaches have complementary strengths: the correlation DAG is fast when the parameter is small and the two-sided column generation approach is superior when the parameter is large. For team games, we show that the two-sided column generation approach vastly outperforms standard column generation approaches, making it the state of the art algorithm when the parameter is large. Along the way, we also introduce two new benchmark games: a trick-taking game that emulates the endgame phase of the card game bridge, and a ride-sharing game, where two drivers traversing a graph are competing to reach specific nodes and serve requests.

show abstract

Simple Uncoupled No-Regret Learning Dynamics for Extensive-Form Correlated Equilibrium

Cited by 5 publications

References 30 publications

Sample-Efficient Learning of Correlated Equilibria in Extensive-Form Games

Sample-Efficient Learning of Correlated Equilibria in Extensive-Form Games

Learning Fair Equilibria in Sponsored Search Auctions

Optimal Correlated Equilibria in General-Sum Extensive-Form Games: Fixed-Parameter Algorithms, Hardness, and Two-Sided Column-Generation

Contact Info

Product

Resources

About