Primal-Dual Active-Set Methods for Large-Scale Optimization

Abstract

In this paper, we introduce two primal-dual active-set methods for solving large-scale constrained optimization problems. The first method minimizes a sequence of primal-dual augmented Lagrangian functions subject to bounds on the primal variables and artificial bounds on the dual variables. The basic structure is similar to the well-known optimization package Lancelot (Conn, et al. in SIAM J Numer Anal 28:545-572, 1991), which uses the traditional primal augmented Lagrangian function. Like Lancelot, our algorithm may use gradient projection-based methods enhanced by subspace acceleration techniques to solve each subproblem and therefore may be implemented matrix-free. The artificial bounds on the dual variables are a unique feature of our method and serve as a form of dual regularization. Our second algorithm is a two-phase method. The first phase computes iterates using our primal-dual augmented Lagrangian algorithm, which benefits from using cheap gradient projections and matrix-free linear CG calculations. The final iterate produced during this phase is then used as input for phase two, which is a stabilized sequential quadratic programming method (Gill and Robinson in SIAM J Opt 1-45, 2013). Obtaining superlinear local convergence under weak assumptions is an important benefit of the transition to a stabilized sequential quadratic programming algorithm. Interestingly, the boundconstrained subproblem used in phase one is equivalent to the stabilized subproblem used in phase two under certain assumptions. This fact makes our choice of algorithms a natural one.

Comments on: Critical Lagrange multipliers: what we currently know about them, how they spoil our lives, and what we can do about it

“…It therefore seems that commonly used two-phase approaches, although successfully used on many problems, may never reliably produce superlinearly convergent iterates in practice, as predicted by the local convergence theory. This observation leads me to conclude that efficient and reliable globally convergent sSQP methods will either be single-phase approaches or two-phase approaches in which the first-phase also uses some form of dual stabilization (Robinson 2015).…”