An Efficient Policy Iteration Algorithm for Dynamic Programming Equations

Alla, Alessandro; Falcone, Maurizio; Kalise, Dante

doi:10.1137/130932284

Cited by 66 publications

(85 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another improvement can be obtained using efficient acceleration methods for the computation of the value function in every subdomain. In the framework of optimal control problems an efficient acceleration technique based on the coupling between value and policy iterations has been recently proposed and studied in [2]. The construction of a DP algorithm for time dependent problems has been addressed in [13] where also a-priori error estimates have been studied.…”

mentioning

confidence: 99%

An Efficient DP Algorithm on a Tree-Structure for Finite Horizon Optimal Control Problems

Alla¹,

Falcone²,

Saluzzi³

2019

SIAM J. Sci. Comput.

Self Cite

View full text Add to dashboard Cite

The classical Dynamic Programming (DP) approach to optimal control problems is based on the characterization of the value function as the unique viscosity solution of a Hamilton-Jacobi-Bellman (HJB) equation. The DP scheme for the numerical approximation of viscosity solutions of Bellman equations is typically based on a time discretization which is projected on a fixed state-space grid. The time discretization can be done by a one-step scheme for the dynamics and the projection on the grid typically uses a local interpolation. Clearly the use of a grid is a limitation with respect to possible applications in high-dimensional problems due to the curse of dimensionality.Here, we present a new approach for finite horizon optimal control problems where the value function is computed using a DP algorithm with a tree structure algorithm (TSA) constructed by the time discrete dynamics. In this way there is no need to build a fixed space triangulation and to project on it. The tree will guarantee a perfect matching with the discrete dynamics and drop off the cost of the space interpolation allowing for the solution of very high-dimensional problems. Numerical tests will show the effectiveness of the proposed method.

show abstract

mentioning

confidence: 99%

An Efficient DP Algorithm on a Tree-Structure for Finite Horizon Optimal Control Problems

Alla¹,

Falcone²,

Saluzzi³

2019

SIAM J. Sci. Comput.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Therefore, the algorithm is a way to enhance PI with both efficiency and robustness features. We refer to [1] for a detailed description of the algorithm. Finally, we note that in both algorithms we penalize the value function outside of the numerical domain to impose state constraints boundary conditions.…”

Section: Numerical Methods For Dynamic Programming Equationsmentioning

confidence: 99%

Feedback control of parametrized PDEs via model order reduction and dynamic programming principle

2020

Self Cite

View full text Add to dashboard Cite

In this paper we investigate infinite horizon optimal control problems for parametrized partial differential equations. We are interested in feedback control via dynamic programming equations which is well-known to suffer from the curse of dimensionality. Thus, we apply parametric model order reduction techniques to construct low-dimensional subspaces with suitable information on the control problem, where the dynamic programming equations can be approximated. To guarantee a low number of basis functions, we combine recent basis generation methods and parameter partitioning techniques. Furthermore, we present a novel technique to construct nonuniform grids in the reduced domain, which is based on statistical information. Finally, we discuss numerical examples to illustrate the effectiveness of the proposed methods for PDEs in two space dimensions.

show abstract

“…To avoid this issue the Empirical Interpolation Method (EIM, [7]) and Discrete Empirical Interpolation Method (DEIM, [12]) were introduced. The computation of the POD basis functions for the nonlinear part is related to the set of the snapshots F (t j , y(t j )), where y(t j ) are already computed from (1). We denote by Φ ∈ R d×k the POD basis functions of rank k min{d, N + 1} of the nonlinear part.…”

Section: Pod For the State Equationmentioning

confidence: 99%

A HJB-POD approach for the control of nonlinear PDEs on a tree structure

Alla

Saluzzi

2020

Applied Numerical Mathematics

Self Cite

View full text Add to dashboard Cite

The Dynamic Programming approach allows to compute a feedback control for nonlinear problems, but suffers from the curse of dimensionality. The computation of the control relies on the resolution of a nonlinear PDE, the Hamilton-Jacobi-Bellman equation, with the same dimension of the original problem. Recently, a new numerical method to compute the value function on a tree structure has been introduced. The method allows to work without a structured grid and avoids any interpolation.Here, we aim at testing the algorithm for nonlinear two dimensional PDEs. We apply model order reduction to decrease the computational complexity since the tree structure algorithm requires to solve many PDEs. Furthermore, we prove an error estimate which guarantees the convergence of the proposed method. Finally, we show efficiency of the method through numerical tests.

show abstract

An Efficient Policy Iteration Algorithm for Dynamic Programming Equations

Cited by 66 publications

References 28 publications

An Efficient DP Algorithm on a Tree-Structure for Finite Horizon Optimal Control Problems

An Efficient DP Algorithm on a Tree-Structure for Finite Horizon Optimal Control Problems

Feedback control of parametrized PDEs via model order reduction and dynamic programming principle

A HJB-POD approach for the control of nonlinear PDEs on a tree structure

Contact Info

Product

Resources

About