Cemracs 2017: numerical probabilistic approach to MFG

Abstract

This project investigates numerical methods for solving fully coupled forward-backward stochastic differential equations (FBSDEs) of McKean-Vlasov type. Having numerical solvers for such mean field FBSDEs is of interest because of the potential application of these equations to optimization problems over a large population, say for instance mean field games (MFG) and optimal mean field control problems. Theory for this kind of problems has met with great success since the early works on mean field games by Lasry and Lions, see [27], and by Huang, Caines, and Malhamé, see [24]. Generally speaking, the purpose is to understand the continuum limit of optimizers or of equilibria (say in Nash sense) as the number of underlying players tends to infinity. When approached from the probabilistic viewpoint, solutions to these control problems (or games) can be described by coupled mean field FBSDEs, meaning that the coefficients depend upon the own marginal laws of the solution. In this note, we detail two methods for solving such FBSDEs which we implement and apply to five benchmark problems. The first method uses a tree structure to represent the pathwise laws of the solution, whereas the second method uses a grid discretization to represent the time marginal laws of the solutions. Both are based on a Picard scheme; importantly, we combine each of them with a generic continuation method that permits to extend the time horizon (or equivalently the coupling strength between the two equations) for which the Picard iteration converges.

A fast iterative PDE-based algorithm for feedback controls of nonsmooth mean-field control problems

“…We emphasize that, a key feature of the FIPDE method is that it computes the functions (u m , v m ) for each (t, x) ∈ [0, T ] × R d based on a PDE formulation, in contrast to the pure data-driven algorithms in [6,20,27,42], which solve (u m , v m ) merely along the trajectories of X m . Hence, the FIPDE method leads to a more accurate approximation of the optimal feedback control, especially outside the support of the optimal state process of (1.1)-(1.2).…”

“…The probabilistic formulation represents an optimal control α ⋆ ∈ H 2 (R k ) as α ⋆ t = α(t, X α ⋆ t , L X α ⋆ t , Y α ⋆ t , Z α ⋆ ) for dt ⊗ dP-a.e., with α being the pointwise minimizer of an associated Hamiltonian H, and (X α ⋆ , Y α ⋆ , Z α ⋆ ) being the solution to a coupled forward-backward stochastic differential equation (FBSDE) depending on α. The coupled FBSDE can be solved by first representing the solution in terms of grid functions [6,42], binomial trees [6] or neural networks [25,20,27], and then employing regression methods to obtain the optimal approximation. Similarly, the deterministic formulation represents an optimal feedback control φ ⋆ as φ ⋆ (t, x) = ᾱ(t, x, µ ⋆ t , (∇ x v)(t, x), (Hess x v)(t, x)) for all (t, x) ∈ [0, T ] × R d .…”

“…This is particularly relevant for high-dimensional MFC problems with degenerate diffusion coefficients or irregular initial distribution (see Section 5). Secondly, by exploiting the PDE formulation of the gradient evaluation, our algorithm recovers the optimal feedback control on the entire computational domain, rather than merely along the trajectories of the optimal state (cf., the probabilistic methods in [6,20,27,42]). This allows us to capture important structures of the optimal control and achieve a robust performance with parameter uncertainty (see Section 5).…”

Mean Field Games and Applications: Numerical Aspects

“…In this paragraph, we consider a recursive method introduced by Chassagneux, Crisan and Delarue in [47] and further studied in [16]. It is based on the following idea.…”

“…In [47], Chassagneux et al introduced a version based on FBSDEs and proved, under suitable regularity assumptions on the decoupling field, the convergence of the algorithm, with a complexity that is exponential in K. The method has been further tested in [16] with implementations relying on trees and grids to discretize the evolution of the state process.…”