A HJB-POD approach for the control of nonlinear PDEs on a tree structure

Abstract

The Dynamic Programming approach allows to compute a feedback control for nonlinear problems, but suffers from the curse of dimensionality. The computation of the control relies on the resolution of a nonlinear PDE, the Hamilton-Jacobi-Bellman equation, with the same dimension of the original problem. Recently, a new numerical method to compute the value function on a tree structure has been introduced. The method allows to work without a structured grid and avoids any interpolation.Here, we aim at testing the algorithm for nonlinear two dimensional PDEs. We apply model order reduction to decrease the computational complexity since the tree structure algorithm requires to solve many PDEs. Furthermore, we prove an error estimate which guarantees the convergence of the proposed method. Finally, we show efficiency of the method through numerical tests.

Approximating optimal feedback controllers of finite horizon control problems using hierarchical tensor formats

“…Both equations can be approximated by the policy iteration [How60], which is a widely used tool in optimal feedback control, see e.g. [KK18; AFK15] Other popular methods for solving the HJB equation are semi-Lagrangian methods [TAK17; DJ14; Fal87], Domain splitting algorithms [FLS94], variational iterative methods [KDK13], data based methods with Neural Networks [Luo+14], actor-critic methods [ZHL21], tree-based methods [AS20] and tropical algorithms [AGL09;AF18].…”

Dynamical low-rank approximations of solutions to the Hamilton-Jacobi-Bellman equation

“…In both cases, many methods rely on a fixed point iteration of the equation, which in the OC and Reinforcement Learning (RL) literature is called policy iteration [How60]. Alternatives are domain splitting algorithms [FLS94], semi-Lagrangian methods [Fal87; FK14; TAK17], data-based methods using Neural Networks [Luo+14], variational iterative methods [KDK13], actor-critic methods [ZH21], tree-based methods [AS19] and tropical methods [AF18; AGL08].…”

Guaranteed Optimal Reachability Control of Reaction-Diffusion Equations Using One-Sided Lipschitz Constants and Model Reduction

“…[26] for application to semilinear parabolic PDEs). The MOR technique of "Proper Orthogonal Decomposition (POD)" was coupled to an HJB approach in [1,2,30]. Here, we couple our HJBbased method to a simple ad hoc reduction method (see Section 2.5).…”