Constrained composite optimization and augmented Lagrangian methods

Abstract

We investigate and develop numerical methods for finite dimensional constrained structured optimization problems. Offering a comprehensive yet simple and expressive language, this problem class provides a modeling framework for a variety of applications. A general and flexible algorithm is proposed that interlaces proximal methods and safeguarded augmented Lagrangian schemes. We provide a theoretical characterization of the algorithm and its asymptotic properties, deriving convergence results for fully nonconvex problems. Adopting a proximal gradient method with an oracle as a formal tool, it is demonstrated how the inner subproblems can be solved by off-the-shelf methods for composite optimization, without introducing slack variables and despite the possibly set-valued projections. Finally, we describe our open-source matrix-free implementation ALPS of the proposed algorithm and test it numerically. Illustrative examples show the versatility of constrained structured programs as a modeling tool, expose difficulties arising in this vast problem class and highlight benefits of the implicit approach developed.

Constrained composite optimization and augmented Lagrangian methods

“…Most of these problem classes have been extensively studied in the convex setting, while nonconvexity has been mostly considered for structured twoterm problems f + g only [40]. Constrained structured optimization [15] entails the minimization of f (x) + g(x) over x ∈ R n subject to set-membership constraints c(x) ∈ D, where c is a smooth mapping and D a nonempty closed set. As such, while more general than nonlinear programming [4] and disjunctive optimization [2], it is covered by the template (P).…”

“…Problem data f and g are allowed to be nonconvex mappings, the nonsmooth cost g is not necessarily continuous nor real-valued, and the mapping c is potentially nonlinear. The composition of a nonsmooth term g with a smooth mapping c results in great modeling flexibility and encompasses a broad spectrum of problems; both regularizers and constraints can be encoded, as well as combinations thereof [9,7,13,15]. Prominent special cases of (P) are nonlinear programming [4], disjunctive [2] and structured optimization [32,39,15].…”

“…A fundamental technique for solving constrained optimization problems is the augmented Lagrangian (AL) framework [35,4,6], which we embrace as it can effortlessly handle nonsmoothness [18,19,15]. Moreover, we work under the practical assumption that (only) the following computational oracles are available or simple to evaluate:…”

Proximal Gradient Algorithms Under Local Lipschitz Gradient Continuity

“…Therefore, we seek relaxed (sub)optimality concepts for the evaluation of the proximal mapping. This viewpoint will ultimately highlight how additionally to being used as a solver within ALMs, as in [12,21,29], PANOC + can operate as an ALMtype solver itself.…”

“…These findings will significantly impact on PANOC (+) both in performance and applicability, propagating to all its dependencies, e.g.,by removing stringent assumptions of general purpose optimization solvers such as OpEn [29]. Indeed, the significance and effectiveness of PANOC + have already been demonstrated in [12,21]. As part of the open-source Julia package ProximalAlgorithms.jl [31], our implementation PANOCplus of PANOC + is publicly available.…”