Constrained Differential Dynamic Programming Revisited

Abstract

Differential Dynamic Programming (DDP) has become a well established method for unconstrained trajectory optimization. Despite its several applications in robotics and controls however, a widely successful constrained version of the algorithm has yet to be developed. This paper builds upon penalty methods and active-set approaches, towards designing a Dynamic Programming-based methodology for constrained optimal control. Regarding the former, our derivation employs a constrained version of Bellman's principle of optimality, by introducing a set of auxiliary slack variables in the backward pass. In parallel, we show how Augmented Lagrangian methods can be naturally incorporated within DDP, by utilizing a particular set of penalty-Lagrangian functions that preserve second-order differentiability. We demonstrate experimentally that our extensions (individually and combinations thereof) enhance significantly the convergence properties of the algorithm, and outperform previous approaches on a large number of simulated scenarios.

Distributed Differential Dynamic Programming Architectures for Large-Scale Multi-Agent Control

“…) , and w i,k , β i,k are the Lagrange multipliers and penalty parameters for the constraint 29) is solved with DDP. Details on the form of P (•) and on how w i,k , β i,k are updated, can be found in [29]. The backward pass rules for DDP will obtain the following form…”

“…This iterative procedure continues until a predefined termination criterion is satisfied. While the original DDP method was developed for unconstrained optimal control problems, several variations have been proposed for handling control and/or state constraints such as [23]- [29]. Out of them, methods that incorporate constraints through an AL on the cost have been shown to be particularly effective [28], [29].…”

“…While the original DDP method was developed for unconstrained optimal control problems, several variations have been proposed for handling control and/or state constraints such as [23]- [29]. Out of them, methods that incorporate constraints through an AL on the cost have been shown to be particularly effective [28], [29]. In this paper, we will use the recently suggested AL-DDP combination [29] as a baseline for addressing constrained optimal control problems with DDP.…”

“…Out of them, methods that incorporate constraints through an AL on the cost have been shown to be particularly effective [28], [29]. In this paper, we will use the recently suggested AL-DDP combination [29] as a baseline for addressing constrained optimal control problems with DDP.…”

Constraint Handling in Continuous-Time DDP-Based Model Predictive Control

“…Other works have adopted an augmented-Lagrangian (AL) approach [19] to transform the generic constrained-DDP problem into an unconstrained one [23], [24]. Certain authors have attempted to combine different optimization-based notions in a single framework: Lantoine et al [25] use an active-set method along with an augmented-Lagrangian formulation to handle hard and soft inequality constraints, respectively; while in [26], the solver switches between AL-DDP and a primaldual interior point method to exploit the benefits carried by both approaches.…”