A Simple Adaptive Procedure Leading to Correlated Equilibrium

Hart, Sergiu; Mas-Colell, Andreu

doi:10.1142/9789814390705_0002

Cited by 135 publications

(287 citation statements)

References 11 publications

Supporting

Mentioning

285

Contrasting

Order By: Relevance

“…The third algorithm in this class is regret matching [23], which we include for completeness (since it is commonly applied) and as a benchmark. Let:…”

Section: Definition 44 (Fading-memory Regret-based Dynamics With Inementioning

confidence: 99%

“…This problem is addressed by the literature on learning in games; the dynamics of learning processes in repeated games is a well investigated branch of game theory (see [6], for example). In particular, the results that are relevant to this work are the guaranteed convergence to Nash equilibrium in potential games of a variety of action adaptation processes, including finite-memory better reply processes [8], adaptive play [9], joint-strategy fictitious play [10], fading-memory regret monitoring [11], and generalised weakened fictitious play [7]; we also include in our investigation regret-matching [23], which converges to the set of correlated equilibria. Thus, a decentralised solution to an optimisation problem can be found by, first, constructing a potential game from the optimisation problem, and then using one of these algorithms to compute an equilibrium.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning in Unknown Reward Games: Application to Sensor Networks

Chapman

Leslie²,

Rogers

et al. 2013

The Computer Journal

View full text Add to dashboard Cite

This paper demonstrates a decentralised method for optimisation using game-theoretic multi-agent techniques, applied to a sensor network management problem. Our first major contribution is to show how the marginal contribution utility design is used to construct a unknown-reward potential game formulation of the problem. This formulation exploits the sparse structure of sensor network problems, and allows us to apply a bound to the price of anarchy of the Nash equilibria of the induced game. Furthermore, since the game is a potential game, solutions can be found using multiagent learning techniques. The techniques we derive use Q-learning to estimate an agent's rewards, while an action adaptation process responds to an agent's opponents' behaviour. However, there are many different algorithmic configurations that could be used to solve these games. Thus, our second major contribution is an extensive evaluation of several action adaptation processes. Specifically, we compare six algorithms across a variety of parameter settings to ascertain the quality of the solutions they produce, their speed of convergence, and their robustness to pre-specified parameter choices. Our results show that they each perform similarly across a wide range of parameters. There is, however, a significant effect from moving to a learning policy with sampling probabilities that go to zero too quickly for rewards to be accurately estimated.

show abstract

“…The third algorithm in this class is regret matching [23], which we include for completeness (since it is commonly applied) and as a benchmark. Let:…”

Section: Definition 44 (Fading-memory Regret-based Dynamics With Inementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Learning in Unknown Reward Games: Application to Sensor Networks

Chapman

Leslie²,

Rogers

et al. 2013

The Computer Journal

View full text Add to dashboard Cite

show abstract

“…If all players use regret matching, then the empirical frequency of the joint actions converges almost surely to the set of coarse correlated equilibria, a generalization of Nash equilibria, in any game [18]. We prove that if all players use JSFP with inertia, then the action profile converges almost surely to a pure Nash equilibrium, albeit in the special glass of generalized ordinal potential games.…”

Section: F Relationship Between Regret Matching and Jsfpmentioning

confidence: 90%

“…It turns out that JSFP is strongly related to the learning algorithm regret matching, from [18], in which players choose their actions based on their regret for not choosing particular actions in the past steps.…”

Section: F Relationship Between Regret Matching and Jsfpmentioning

confidence: 99%

“…It turns out that there is a strong similarity between the JSFP discussed herein and the regret matching algorithm [18]. A player's regret for a particular choice is defined as the difference between 1) the utility that would have been received if that particular choice was played for all the previous stages and 2) the average utility actually received in the previous stages.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Joint Strategy Fictitious Play with Inertia for Potential Games

Marden

Arslan

Shamma

Proceedings of the 44th IEEE Conference on Decision and Control

View full text Add to dashboard Cite

Abstract-We consider multi-player repeated games involving a large number of players with large strategy spaces and enmeshed utility structures. In these "large-scale" games, players are inherently faced with limitations in both their observational and computational capabilities. Accordingly, players in large-scale games need to make their decisions using algorithms that accommodate limitations in information gathering and processing. This disqualifies some of the well known decision making models such as "Fictitious Play" (FP), in which each player must monitor the individual actions of every other player and must optimize over a high dimensional probability space. We will show that Joint Strategy Fictitious Play (JSFP), a close variant of FP, alleviates both the informational and computational burden of FP. Furthermore, we introduce JSFP with inertia, i.e., a probabilistic reluctance to change strategies, and establish the convergence to a pure Nash equilibrium in all generalized ordinal potential games in both cases of averaged or exponentially discounted historical data. We illustrate JSFP with inertia on the specific class of congestion games, a subset of generalized ordinal potential games. In particular, we illustrate the main results on a distributed traffic routing problem and derive tolling procedures that can lead to optimized total traffic congestion. Index Terms-Fictitious play (FP), joint strategy fictitious play (JSFP).

show abstract

Game Theoretic Approaches for Resource Management in Multi‐Tier Networks

Hossain

Niyato

2013

Radio Resource Management in Multi‐Tier Cellular Wireless Networks

View full text Add to dashboard Cite

A Simple Adaptive Procedure Leading to Correlated Equilibrium

Cited by 135 publications

References 11 publications

Learning in Unknown Reward Games: Application to Sensor Networks

Learning in Unknown Reward Games: Application to Sensor Networks

Joint Strategy Fictitious Play with Inertia for Potential Games

Game Theoretic Approaches for Resource Management in Multi‐Tier Networks

Contact Info

Product

Resources

About