Constrained composite optimization and augmented Lagrangian methods

Abstract

We investigate finite-dimensional constrained structured optimization problems, featuring composite objective functions and set-membership constraints. Offering an expressive yet simple language, this problem class provides a modeling framework for a variety of applications. We study stationarity and regularity concepts, and propose a flexible augmented Lagrangian scheme. We provide a theoretical characterization of the algorithm and its asymptotic properties, deriving convergence results for fully nonconvex problems. It is demonstrated how the inner subproblems can be solved by off-the-shelf proximal methods, notwithstanding the possibility to adopt any solvers, insofar as they return approximate stationary points. Finally, we describe our matrix-free implementation of the proposed algorithm and test it numerically. Illustrative examples show the versatility of constrained composite programs as a modeling tool and expose difficulties arising in this vast problem class.

Inexact penalty decomposition methods for optimization problems with geometric constraints

“…holds. Now, setting µ = µ * , λ = λ * , ν x i = αs ρ i − α x i (i ∈ I 0 (x * )) and ν y i = αx * i − α y i (i ∈ I 0 (y * )) for an arbitrary α > 0, we see that ( 13) is a direct consequence of (12). Moreover, for…”

“…The class of nonconvex approximation schemes usually replaces the ℓ 0 -term in (SPO) by a nonconvex penalty function. One possibility is to use the ℓ p -quasi-norm for p ∈ (0, 1), see, e.g., [11,12], which has nicer properties than the ℓ 0 -norm, e.g., it is continuous. However, despite its nonconvexity, it also fails to be Lipschitz continuous.…”

Implicit augmented Lagrangian and generalized optimization

“…Nonsmooth composite problems like (P) have been extensively studied in the past decades, but most often with convex (or even polyhedral) nonsmooth term g; see [22,35,48]. Prominent special cases of (P) are nonlinear [4] and disjunctive [2] programming, structured optimization [39,47], and constrained structured optimization [17]. The abstract problem (P) is also referred to as "extended nonlinear programming" in [43], promoting better representation of structures found in applications by featuring a composite term, beyond the conventional format of nonlinear programming.…”

“…subject to c(x) − z = 0, (P S ) which has a simpler, separable objective function but is subject to some (explicit) constraints. It has been observed in [17,Lemma 3.1] that incorporating auxiliary variables in this manner does not affect minimizers and stationary points when compared to the original (P) -somewhat remarkably in light of [3]. A fundamental technique for solving constrained optimization problems such as (P S ) is the augmented Lagrangian (AL) framework [4,6,42], which can also effortlessly handle nonsmoothness [17,18,21,23].…”

Convergence Analysis of the Proximal Gradient Method in the Presence of the Kurdyka-Łojasiewicz Property without Global Lipschitz Assumptions

“…for some penalty parameter ρ k > 0. Since the squared distance function y → dist 2 (y, C) is continuously differentiable by convexity of C, see [7,Corollary 12.30], this subproblem has exactly the structure of the composite optimization problem (P) and can therefore, in principle, be solved by a proximal gradient method, see [21,24,27,28] for suitable realizations of this approach.…”