Tensor Decomposition Methods for High-dimensional Hamilton--Jacobi--Bellman Equations

Abstract

A tensor decomposition approach for the solution of high-dimensional, fully nonlinear Hamilton-Jacobi-Bellman equations arising in optimal feedback control of nonlinear dynamics is presented. The method combines a tensor train approximation for the value function together with a Newton-like iterative method for the solution of the resulting nonlinear system. The tensor approximation leads to a polynomial scaling with respect to the dimension, partially circumventing the curse of dimensionality. A convergence analysis for the linear-quadratic optimal control problem is presented. For nonlinear dynamics, the effectiveness of the high-dimensional control synthesis method is assessed in the optimal feedback stabilization of the Allen-Cahn and Fokker-Planck equations with a hundred of variables.

A HJB-POD approach for the control of nonlinear PDEs on a tree structure

“…The solution of ( 13) is obtained by the SVD of the snapshots matrix Y = ΨΣV T , where we consider the first −columns {ψ i } i=1 of the orthogonal matrix Ψ. The selection of the rank of POD basis is based on the error computed in (13) which is related to the singular values neglected. We will choose such that E( ) ≈ 0.999, with…”

“…Other approaches to mitigate the curse of dimensionality are built upon the sparse grid method (see e.g. [16]), the spectral elements method (see [19]) and, more recently, a tensor decomposition (see [13]). For the sake of completeness, we mention that the control of PDEs can be solved with other methods such as, among others, open loop techniques (see e.g [22]) and model predictive control (see e.g [17]).…”

Optimizing semilinear representations for State-dependent Riccati Equation-based feedback control

“…This poses a formidable computational computational challenge, as the HJB PDE is a nonlinear first order PDE cast over the state space of the system, with a dimension which can be arbitrarily high. Over the last years, the solution of large-scale optimal feedback synthesis problems has witnessed a tremendous development, in parallel with the development of sophisticated techniques for high-dimensional problems, such as tensor decomposition techniques Dolgov et al (2021), representation formulas for the HJB PDE Chow et al (2019), tree-structured algorithms Alla and Saluzzi (2020) and data-driven methods Han et al (2018); Azmi et al (2021); Nakamura-Zimmerer et al (2021), among others. An alternative approach, which can be interpreted as a relaxed dynamic programming, is provided by Nonlinear Model Predictive Control (NMPC) Grüne and Rantzer (2008).…”

Semiglobal optimal feedback stabilization of autonomous systems via deep neural network approximation

“…With the aim of increasing the dimension of computable HJB equations a sparse grid approach was presented in [18]. More recently significant progress was made in solving high dimensional HJB equations by the of use policy iterations (Newton method applied to HJB) combined with tensor calculus techniques, [12,22]. The use of Hopf formulas was proposed in e.g.…”