On the Convergence of Finite Element Methods for Hamilton--Jacobi--Bellman Equations

Abstract

In this note we study the convergence of monotone P1 finite element methods on unstructured meshes for fully nonlinear Hamilton-Jacobi-Bellman equations arising from stochastic optimal control problems with possibly degenerate, isotropic diffusions. Using elliptic projection operators we treat discretisations which violate the consistency conditions of the framework by Barles and Souganidis. We obtain strong uniform convergence of the numerical solutions and, under non-degeneracy assumptions, strong L 2 convergence of the gradients. I IntroductionHamilton-Jacobi-Bellman (HJB) equations, which are of the formwhere the L α are linear first-or second order operators and d α ∈ L 2 , characterise the value function of optimal control problems. Indeed, one possibility to introduce the notion of solution of (1) is via the underlying optimal control structure. An alternative approach is to use the monotonicity properties of the operator which leads to the concept of viscosity solutions. While these perceptions are essentially equivalent [17, p.72] both views have been instructive for the design and analysis of numerical methods.The former approach, based on the discretisation of the optimal control problem before employing the Dynamic Programming Principle, has been proposed in the setting of finite elements in [26,7,8], see also the review article [22] and the references therein. Regarding finite difference methods we refer to the book [23]. The latter approach, which is also adopted in this note, was firmly established with the contribution [3] by Barles and Souganidis in 1991, providing an abstract framework for the convergence to viscosity solutions. Starting with [20,21] techniques were developed to quantify the rate of convergence; more recent works are [1,13]. A third direction was opened by the method of vanishing moments which neither enforces discrete maximum principles nor makes use of the underlying optimal control structure but relies on a higher order regularisation [16]. For a more comprehensive review of the state-of-the-art in the numerical solution of fully non-linear second order equations we refer to [15].In the traditional finite element analysis the multiplicative testing with hat functions is viewed as the discrete analogue of the multiplicative testing procedure to define weak solutions of the (variational) differential equation. While elements of this viewpoint are implicitly used in Section VII on gradient convergence, we would like to stress a second interpretation: multiplication with hat functions as regularisation of the residual. Consider for a moment the linear problem −a(x)∆u(x) = f (x) with smooth functions a and u as well as a hat function φ at the node y ℓ . Let P be the orthogonal projection onto the approximation space with respect to the scalar product 〈v, w〉 = ∇v · ∇w dx (given suitable boundary

Convergent Semi-Lagrangian Methods for the Monge--Ampère Equation on Unstructured Grids

“…Well-posedness of the discrete equations. A common technique to show the well-posedness of a nonlinear system such as (21) is to formulate a fixed point argument akin to a pseudo-time Euler scheme [13,29]. However, to take advantage of the monotone discretization of the Bellman equation, we use instead Howard's algorithm [7,20] to establish the existence and uniqueness of numerical solutions.…”

“…We deviate from the established Barles-Souganidis framework in the treatment of the boundary conditions to address challenges arising from consistency and comparison. The proposed approach also bridges the gap between advances on numerical methods for these two classes of second order fully nonlinear PDEs, see for instance [6,10,11,13,21,25,30] and the references therein for the numerical literature on Bellman equations.…”

Optimal feedback control for undamped wave equations by solving a HJB equation

“…Another recent reference [15] gives an overview on semi-Lagrangian schemes and the features of this class of methods. For a finite element approach we refer to [28], and for discontinuous Galerkin methods to [7]. Another class of antidiffusive methods have also been studied in [8,12].…”

Preface: DGAA Special Issue on Numerical Methods for Dynamic Games

“…For N players one gets a system of N elliptic partial differential equations that provides a Nash equilibrium in feedback form for N -person stochastic differential games, and it can be proved that in the limit for N going to infinity one gets a system of two PDEs, where a classical Hamilton-JacobiBellman equation is coupled with a Kolmogorov-Fokker-Planck equation for the density of the players. Numerical approaches to determine equilibria of dynamic games typically rely on dynamic programming with time discretization (e.g., [2], or [1]), collocation schemes (e.g., [5]) or the application of standard approaches, like finite elements or finite differences, for solving the associated Hamilton-Jacobi-Bellman equations [3,4].…”