A Simple Adaptive Procedure Leading to Correlated Equilibrium

Abstract

We propose a new and simple adaptive procedure for playing a game: ''regret-matching.'' In this procedure, players may depart from their current play with probabilities that are proportional to measures of regret for not having used other strategies in the past. It is shown that our adaptive procedure guarantees that, with probability one, the empirical distributions of play converge to the set of correlated equilibria of the game.

Brown–von Neumann–Nash dynamics: The continuous strategy case

“…[20] for the corresponding property in finite games; Sandholm [27] calls this class 'stable games'). Note first that linear quadratic games like f (x, y) = −x 2 + axy satisfy condition (16) if and only if a ≤ 0 as one can easily check. Furthermore, every symmetric zero-sum game satisfies condition (16).…”

“…Note first that linear quadratic games like f (x, y) = −x 2 + axy satisfy condition (16) if and only if a ≤ 0 as one can easily check. Furthermore, every symmetric zero-sum game satisfies condition (16). By definition of a symmetric zero-sum game, f (x, y) + f (y, x) = 0 for all x, y ∈ S. This implies that E(P, Q) +E(Q, P ) = 0, and in particular E(P, P ) = 0.…”

“…The first term in parentheses, E ¡ R P − P, R P − P ¢ ≤ 0 by Assumption (16), and the second term, E ¡ R P − P, P ¢ ≥ 0 by definition of the relative excess measure (see (6)). Thus, we obtain d dt H(P ) ≤ 0 and the inequality is strict whenever P is not a stationary point.…”

“…This careful, conservative rule is reminiscent of the proportional imitation rule of Schlag [29] who shows that his "proportional imitation rule" has certain optimality properties in simple decision problems. It is also closely related to the regret-matching rule of Hart and Mas-Colell [16], [17] where better strategies are also adopted with a probability that is proportional to the apparent gain the new strategy yields, only that gains are calculated using the regret one has for not having played the new strategy right from the start.…”

Hannan Consistency in On-Line Learning in Case of Unbounded Losses Under Partial Monitoring

“…Throughout the paper we use this framework in which the algorithm has a limited access to the losses. For example, in the so called multi-armed bandit problem the algorithm has only information on the loss of the chosen expert, and no information is available about the loss it would have suffered had it made a different decision (see, e.g., Auer et al [1], Hart and Mas Colell [13]). Another example is label efficient prediction, where it is expensive to obtain the losses of the experts, and therefore the algorithm has the option to query this information (see Cesa-Bianchi et.…”