Optimal Correlated Equilibria in General-Sum Extensive-Form Games: Fixed-Parameter Algorithms, Hardness, and Two-Sided Column-Generation

Abstract

We study the problem of finding optimal correlated equilibria of various sorts: normal-form coarse correlated equilibrium (NFCCE), extensive-form coarse correlated equilibrium (EFCCE), and extensive-form correlated equilibrium (EFCE). This is NP-hard in the general case and has been studied in special cases, most notably triangle-free games , which include all two-player games with public chance moves. However, the general case is not well understood, and algorithms usually scale poorly. In this paper, we make two primary contributions.First, we introduce the correlation DAG, a representation of the space of correlated strategies whose structure and size are dependent on the specific solution concept desired. It extends the team belief DAG of Zhang et al.[2022] to general-sum games. For each of the three solution concepts, its size depends exponentially only on a parameter related to the information structure of the game. We also prove a fundamental complexity gap: while our size bounds for NFCCE are similar to those achieved in the case of team games by Zhang et al. [2022], this is impossible to achieve for the other two concepts under standard complexity assumptions.Second, we propose a two-sided column generation approach to compute optimal correlated strategies in extensive-form games. Our algorithm improves upon the one-sided approach of Farina et al. [2021a] by means of a new decomposition of correlated strategies which allows players to re-optimize their sequence-form strategies with respect to correlation plans which were previously added to the support.Experiments show that our techniques outperform the prior state of the art for computing optimal generalsum correlated equilibria, and that our two families of approaches have complementary strengths: the correlation DAG is fast when the parameter is small and the two-sided column generation approach is superior when the parameter is large. For team games, we show that the two-sided column generation approach vastly outperforms standard column generation approaches, making it the state of the art algorithm when the parameter is large. Along the way, we also introduce two new benchmark games: a trick-taking game that emulates the endgame phase of the card game bridge, and a ride-sharing game, where two drivers traversing a graph are competing to reach specific nodes and serve requests.

Polynomial-Time Optimal Equilibria with a Mediator in Extensive-Form Games

“…We ran our algorithm for communication and full-certification equilibria on various two-player games, and compared the results to those given by notions of optimal correlation in games. The games tested are the same as the ones tested by Zhang et al [24]; refer to that paper for detailed descriptions. All experiments were allocated four CPU cores and 64 GB of RAM.…”

“…Linear programs were solved with Gurobi 9.5. When payments Values and timings for optimal correlated equilibria were taken from Zhang et al [24] and are included here for purposes of comparison. When payments are used, the mediator is informed before making the payment of whether the player was honest, and the optimization objective is the social welfare of the original terminal state, minus any payments made.…”

“…Indeed, in all histories h Ia of Γ, we must have I h. Therefore, we have | Ĥ| ≤ |H|BD where B is the maximum branching factor and D is the depth of the game tree, i.e., the size of Γ goes from essentially quadratic to essentially quasilinear in |H|. The mediator's decision points in Γ for a full-certification equilibrium are the trigger histories used by Zhang et al [24] in their analysis of various notions of correlated equilibria. Later, we will draw further connections between full certification and correlation.…”

“…For perfect-recall mediators, this is possible in polynomial time via the sequence form; for imperfect-recall mediators, the problem is NP-hard, in general, but the team belief DAG of Zhang et al [25] can be used to recover fixed-parameter algorithms. For the notions of correlated equilibrium, this method results in basically the same construction as the correlation DAG of Zhang et al [24].…”

“…For example, correlated equilibria are usually arrived at in the "truth known, imperfect recall" setting; the "truthful" correlated equilibria are more difficult to motivate in this respect. Further, even the fixed-parameter algorithms of Zhang et al [24] would fail in this setting, because "public states" can no longer be treated as public due to the possibility of lying players. We leave to future research the problem of finding a motivation for the notions that we do not reference elsewhere in the paper.…”