Robust Feedback Control of Nonlinear PDEs by Numerical Approximation of High-Dimensional Hamilton--Jacobi--Isaacs Equations

Abstract

We propose an approach for the synthesis of robust and optimal feedback controllers for nonlinear PDEs. Our approach considers the approximation of infinite-dimensional control systems by a pseudospectral collocation method, leading to high-dimensional nonlinear dynamics. For the reduced-order model, we construct a robust feedback control based on the H∞ control method, which requires the solution of an associated high-dimensional Hamilton-Jacobi-Isaacs nonlinear PDE. The dimensionality of the Isaacs PDE is tackled by means of a separable representation of the control system, and a polynomial approximation ansatz for the corresponding value function. Our method proves to be effective for the robust stabilization of nonlinear dynamics up to dimension d ≈ 12. We assess the robustness and optimality features of our design over a class of nonlinear parabolic PDEs, including nonlinear advection and reaction terms. The proposed design yields a feedback controller achieving optimal stabilization and disturbance rejection properties, along with providing a modelling framework for the robust control of PDEs under parametric uncertainties.

A Neural Network-Based Policy Iteration Algorithm with Global $$H^2$$-Superlinear Convergence for Stochastic Games on Domains

“…[9]) and high-order polynomial spaces or kernel-function spaces used in global spectral methods (see e.g. [4,5,13,28,29]). We now demonstrate that (H.3) can also be easily satisfied by the sets of multilayer feedforward neural networks, which provides effective trial functions for high-dimensional problems.…”

“…For trial functions with linear architecture, e.g. if {F M } M∈N are finite element spaces, high-order polynomial spaces, and kernelfunction spaces (see [9,13,28,29]), one may evaluate the norms by applying high-order quadrature rules to the basis functions involved.…”

“…Specifically, [4,5,28] approximate the solution to each linear equation via a separable polynomial ansatz (without concluding any convergence rate), while [30] assumes each linear equation is solved sufficiently accurately (without specifying a numerical method), and deduces pointwise linear convergence. The continuous policy iteration in [28] has also been applied to solve HJBI equations on R n in [29], which is a direct extension of Algorithm Ho-3 in [8] and still requires to solve a nonlinear HJB subproblem at each iteration. In this paper, we propose an easily implementable accuracy criterion for the numerical solutions of the linear PDEs which ensures the numerical solutions converge superlinearly in a suitable function space for nonconvex HJBI equations from an arbitrary initial guess.…”

State-dependent Riccati equation feedback stabilization for nonlinear PDEs

“…Unfortunately, the dynamic programming approach is not suitable for systems described by large-scale dynamics, as the computational complexity of approximating the associated high-dimensional HJB PDEs goes beyond the reach of traditional computational methods. Only very recently, the use of effective computational approaches such as sparse grids [19,31], tree structure algorithms [2], polynomial approximation [28,29,4] tensor decomposition methods [47,21,18,39], and representation formulas [13,12] have addressed the solution of highdimensional HJB PDEs. Recent works making use of deep learning [24,17,38,26,34,37,30] anticipate that the synthesis of optimal feedback laws for largescale dynamics can be a viable path in the near future.…”

Tensor Decomposition Methods for High-dimensional Hamilton--Jacobi--Bellman Equations

“…While the rigorous design of numerical methods for the solution of very high-dimensional HJB PDEs remains largely an open problem, encouraging results have been obtained over the last years. A non-exhaustive list includes the use of machine learning techniques [59,22,31,35], approximate dynamic programming in the context of reinforcement learning [11,56], causality-free methods and convex optimization [40,17], max-plus algebra methods [47,1], polynomial approximation [39,38], tree structure algorithms [4], and sparse grids [14,25]. A very recent stream of works [22,59,35,55,36] has explored the use of machine learning techniques to approximate high-dimensional nonlinear PDEs.…”