On the Convergence Problem in Mean Field Games: A Two State Model without Uniqueness

Abstract

We consider N -player and mean field games in continuous time over a finite horizon, where the position of each agent belongs to {−1, 1}. If there is uniqueness of mean field game solutions, e.g. under monotonicity assumptions, then the master equation possesses a smooth solution which can be used to prove convergence of the value functions and of the feedback Nash equilibria of the N -player game, as well as a propagation of chaos property for the associated optimal trajectories. We study here an example with anti-monotonous costs, and show that the mean field game has exactly three solutions. We prove that the value functions converge to the entropy solution of the master equation, which in this case can be written as a scalar conservation law in one space dimension, and that the optimal trajectories admit a limit: they select one mean field game soution, so there is propagation of chaos. Moreover, viewing the mean field game system as the necessary conditions for optimality of a deterministic control problem, we show that the N -player game selects the optimizer of this problem. 1 2 ALEKOS CECCHIN, PAOLO DAI PRA, MARKUS FISCHER, AND GUGLIELMO PELINO are shown to be concentrated on weak solutions of the corresponding mean field game. This concept of solution is also used in another, more recent work by Lacker; see below.Here, we are interested in the convergence problem for Nash equilibria in Markov feedback strategies with full state information. A first result in this direction was given by Gomes, Mohr, and Souza [19] in the case of finite state dynamics. There, convergence of Markovian Nash equilibria to the mean field game limit is proved, but only if the time horizon is small enough. A breakthrough was achieved by Cardaliaguet, Delarue, Lasry, and Lions in [7]. In the setting of games with non-degenerate Brownian dynamics, possibly including common noise, those authors establish convergence to the mean field game limit, in the sense of convergence of value functions as well as propagation of chaos for the optimal state trajectories, for arbitrary time horizon provided the so-called master equation associated with the mean field game possesses a unique sufficiently regular solution. The master equation arises as the formal limit of the Hamilton-Jacobi-Bellman systems determining the Markov feedback Nash equilibria. It yields, if well-posed, the optimal value in the mean field game as a function of initial time, state and distribution. It thus also provides the optimal control action, again as a function of time, state, and measure variable. This allows, in particular, to compare the prelimit Nash equilibria to the solution of the limit model through coupling arguments.If the master equation possesses a unique regular solution, which is guaranteed under the Lasry-Lions monotonicity conditions, then the convergence analysis can be considerably refined. In this case, for games with finite state dynamics, Cecchin and Pelino [11] and, independently, Bayraktar and Cohen [3] obtain a central limit theorem and lar...

Convergence to the mean field game limit: A case study

“…However, to the best of our knowledge, the question which mean field equilibria are limit points of (true) n-player equilibria has not been emphasized as such in the literature. We can mention the parallel work [15] on a two-state game: the game has unique n-player equilibria and these converge to a mean field equilibrium as expected; however, a second, less plausible mean field solution can appear for certain parameter values and this solution is not a limit. Another interesting parallel work [16] studies several approaches of selecting an equilibrium in a linear-quadratic mean field game with multiple equilibria, including the convergence of n-player equilibria.…”

On non-uniqueness in mean field games

“…Proof. It is clear that the function Y (x, t) is C 1 outside the shock curve, and we only need to check the Rankine-Hugoniot condition and the Lax condition (see [6,Proposition 3]). Define…”

“…In combination with the fact that U (t, i, θ) is smooth outside the curveγ(t) = 1 2 , it can be easily seen that V N +1 (t, 1, θ) converges to U (t, 1, θ) if θ = 1 2 (see e.g. [6,Theorem 8] ). Let (ξ j ) j∈N be the i.i.d initial datum of Z j such that P[ξ j = 0] =θ = 1 2 , P[ξ j = 1] = 1 −θ.…”

“…Supposing an anti-monotone running cost (follow the crowd game), [9] poses a conjecture on the nature of the limits of N + 1-player Nash equilibrium. We proceed by using similar techniques to [6], which considers an anti-monotone terminal condition. In particular, we again rely on the entropy solution of the master equation to prove the convergence and show that the limit of N + 1player Nash equilibrium selects the unique mean field equilibrium induced by this entropy solution.…”

An example of multiple mean field limits in ergodic differential games

“…The convergence relies on the construction of a solution to the so-called master equation, a partial differential equation stated in the space of probability measures. The result was later applied and extended to different frameworks, with similar-or closely related-techniques of proof in [2,7,11,12,13,14].…”