Policy Iteration Method for Time-Dependent Mean Field Games Systems with Non-separable Hamiltonians

Laurière, Mathieu; Song, Jiahao; Tang, Qing

doi:10.1007/s00245-022-09925-5

Cited by 7 publications

(4 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Because the sampling method is more tractable than the finite-difference method, the policy iteration method may allow high-dimensional ML-POSC to be solved. Furthermore, the policy iteration method has recently been studied in MFSC [51][52][53]. However, its convergence is not guaranteed except for special cases in MFSC.…”

Section: Discussionmentioning

confidence: 99%

Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control

Tottori

Kobayashi

2023

Entropy

View full text Add to dashboard Cite

Memory-limited partially observable stochastic control (ML-POSC) is the stochastic optimal control problem under incomplete information and memory limitation. To obtain the optimal control function of ML-POSC, a system of the forward Fokker–Planck (FP) equation and the backward Hamilton–Jacobi–Bellman (HJB) equation needs to be solved. In this work, we first show that the system of HJB-FP equations can be interpreted via Pontryagin’s minimum principle on the probability density function space. Based on this interpretation, we then propose the forward-backward sweep method (FBSM) for ML-POSC. FBSM is one of the most basic algorithms for Pontryagin’s minimum principle, which alternately computes the forward FP equation and the backward HJB equation in ML-POSC. Although the convergence of FBSM is generally not guaranteed in deterministic control and mean-field stochastic control, it is guaranteed in ML-POSC because the coupling of the HJB-FP equations is limited to the optimal control function in ML-POSC.

show abstract

Section: Discussionmentioning

confidence: 99%

Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control

Tottori

Kobayashi

2023

Entropy

View full text Add to dashboard Cite

show abstract

“…In this regard, a significant part of it is dedicated to the study of numerical methods and algorithms for the computation of the solution to the MFG model, both in the formulation as a PDEs system and as an optimal control problem of a PDE. Such approaches, just to mention a few, include finite differences, semi-Lagrangian methods and Fourier expansions with regard to the approximation methods (see [1,2,6,7,11,13,14,28,29,31,33,34]). Many of these methods exploit the variational structure of the problem, concerns the case in which the coupling term involving the distribution of the population is separated from the Hamiltonian, while relatively few works have been dedicated to the so-called non-separable case…”

Section: Introductionmentioning

confidence: 99%

“…To design implicit finite difference schemes, iterative methods are needed to reduce the problem to a sequence of linear systems. Iterative methods employed in solving MFGs include Newton's method [6,7,9,28], fixed point iteration, fictitious play, policy iteration [15,29], smoothed policy iteration [34], etc. In particular, numerical solution of MFGs with non-separable Hamiltonian have been discussed in, e.g., [6,7,23,28,29].…”

Section: Introductionmentioning

confidence: 99%

On the Quadratic Convergence of Newton’s Method for Mean Field Games with Non-separable Hamiltonian

Camilli,

Tang

2024

Dyn Games Appl

Self Cite

View full text Add to dashboard Cite

We analyze asymptotic convergence properties of Newton’s method for a class of evolutive Mean Field Games systems with non-separable Hamiltonian arising in mean field type models with congestion. We prove the well posedness of the Mean Field Game system with non-separable Hamiltonian and of the linear system giving the Newton iterations. Then, by forward induction and assuming that the initial guess is sufficiently close to the solution of problem, we show a quadratic rate of convergence for the approximation of the Mean Field Game system by Newton’s method. We also consider the case of a nonlocal coupling, but with separable Hamiltonian, and we show a similar rate of convergence.

show abstract

“…In this regard, a significant part of it is dedicated to the study of numerical methods and algorithms for the computation of the solution to the MFG model, both in the formulation as a PDEs system and as an optimal control problem of a PDE. Such approaches, just to mention a few, include finite differences, semi-Lagrangian methods and Fourier expansions with regard to the approximation methods and policy iteration, Newton method, fictitious play, convex programming for the algorithms (see [1,2,6,5,9,11,12,26,27,29,31,32]). Most of the convergence results for numerical methods, which often exploit the variational structure of the problem, concerns the case in which the coupling term involving the distribution of the population is separated from the Hamiltonian, while relatively few works have been dedicated to the so-called non-separable case…”

Section: Introductionmentioning

confidence: 99%

Deep learning for Mean Field Games with non-separable Hamiltonians

Assouli¹,

Missaoui²

2023

Chaos, Solitons & Fractals

View full text Add to dashboard Cite

Policy Iteration Method for Time-Dependent Mean Field Games Systems with Non-separable Hamiltonians

Cited by 7 publications

References 41 publications

Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control

Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control

On the Quadratic Convergence of Newton’s Method for Mean Field Games with Non-separable Hamiltonian

Deep learning for Mean Field Games with non-separable Hamiltonians

Contact Info

Product

Resources

About