The Master Equation and the Convergence Problem in Mean Field Games

Abstract

The paper studies the convergence, as N tends to infinity, of a system of N coupled Hamilton-Jacobi equations, the Nash system. This system arises in differential game theory. We describe the limit problem in terms of the so-called "master equation", a kind of second order partial differential equation stated on the space of probability measures. Our first main result is the well-posedness of the master equation. To do so, we first show the existence and uniqueness of a solution to the "mean field game system with common noise", which consists in a coupled system made of a backward stochastic Hamilton-Jacobi equation and a forward stochastic Kolmogorov equation and which plays the role of characteristics for the master equation. Our second main result is the convergence, in average, of the solution of the Nash system and a propagation of chaos property for the associated "optimal trajectories".

Deep Teams: Decentralized Decision Making With Finite and Infinite Number of Agents

“…In this section, we present quantized solutions for Problems 1 and 2 to reduce the complexity in space for dynamic programs (25) and (42) in Theorems 2 and 4, respectively.…”

“…The optimal strategy of Theorem 2 can be implemented in a distributed manner since every agent can independently compute the dynamic program (25) and observe the deep state. 2 According to (26), for any k ∈ K, i ∈ N k and t ∈ N T , the action (role) of agent i of sub-population k at time t is determined by three factors: (a) global law ψ t that depends on the agents' dynamics, per-step cost, and underlying probability distributions; (b) deep state d t that provides the statistical information on the states of agents, and (c) local state x i t that is private information for agent i and unknown to others.…”

“…To numerically solve the dynamic program (25), at each step t ∈ N T , it is required to store | E | 2 |G| entries corresponding to the transition probability matrix of deep state (24), and to run |W| iterations to compute each entry, where each entry occupies a certain amount of space and each iteration takes a certain amount of time. In addition, to compute the minimization in (25), | E ||G| iterations are needed for searching over all possible cases. On the other hand, from (27), | E | and |W| are bounded by k∈K (|N k | + 1) |X k | and k∈K (|N k | + 1) |W k | , respectively.…”

“…The computational complexity of solving the dynamic program(25) in both space and time is polynomial with respect to the number of agents in each sub-population |N k |, k ∈ K, and is linear with respect to the control horizon T ∈ N.…”

Sequential Decomposition of Graphon Mean Field Games

“…In such games, the individual agents have minimal impact of the overall outcome of the game and so the agents track a mean distribution of states of other agents rather than their actual states. MFGs is an excellent and a tractable model to study large population dynamic games of incomplete information, and has been shown to be a good approximation of Nash equilibrium (or MPE) of the original game as the number of players grow large (for instance see [7], [8], [9], [10], [11] and references therein).…”