Policy iteration method for time-dependent Mean Field Games systems with non-separable Hamiltonians

Abstract

We introduce two algorithms based on a policy iteration method to numerically solve time-dependent Mean Field Game systems of partial differential equations with non-separable Hamiltonians. We prove the convergence of such algorithms in sufficiently small time intervals with Banach fixed point method. Moreover, we prove that the convergence rates are linear. We illustrate our theoretical results by numerical examples, and we discuss the performance of the proposed algorithms.

Rates of convergence for the policy iteration method for Mean Field Games systems

“…Some numerical examples have been considered in [19] for comparing the policy iteration method and the Newton method for solving MFGs. In many of these the policy iteration method turns out to be more efficient in terms of computing time.…”

“…Despite the previous limitations, however, the policy iteration method retains the advantage of replacing the resolution of a strongly coupled nonlinear system with a sequence of decoupled linear problems. Moreover, in a neighborhood of the solution, the rapid convergence of the value function u (n) is also reflected in an equally rapid convergence of the distribution m (n) (see [9] and [19] for some numerical simulations).…”

“…In this paper we restrict the discussion to MFGs with separable Hamiltonians. Recently in [19] the authors considered the convergence rate of policy iteration algorithms for MFGs with non-separable Hamiltonians using contraction fixed point method. The key difference is that here we do not impose the short time assumption, which is essential for the reasoning in [19].…”

Optimal bailout strategies resulting from the drift controlled supercooled Stefan problem

“…Among the numerical methods proposed for mean-field control problems, we refer to [21] for a policy gradient-type method where feedback controls are approximated by neural networks and optimised for a given objective function; to [32] and again to [21] for a mean-field FBSDE method, generalising the deep BSDE method to mean-field dependence and in the former case to delayed effects; to [48] for a hybrid model where the mean-field distribution is approximated by a particle system and the control is obtained by numerical approximation of a PDE; and to [6] for a survey of methods for the coupled PDE systems, mainly in the spirit of the seminal works [3,2,5]; see also a related semi-Lagrangian scheme in [16]; a gradient method and penalisation approach in [44]; and a recent analysis of policy iteration in [38].…”

“…A similar iterative procedure has been analysed concurrently with this work in [38] for the classical mean-field game setting. The authors there propose a policy iteration where iterative approximation of the optimal control, as in the context of standard control problems, is intertwined with the iteration over the forward density, and prove convergence locally in time.…”