Implicit augmented Lagrangian and generalized optimization

Abstract

Generalized nonlinear programming is considered without any convexity assumption, capturing a variety of problems that include nonsmooth objectives, combinatorial structures, and set-membership nonlinear constraints. We extend the augmented Lagrangian framework to this broad problem class, preserving an implicit formulation and introducing auxiliary variables merely as a formal device. This, however, gives rise to a generalized augmented Lagrangian function that lacks regularity. Based on parametric optimization, we develop a tailored stationarity concept to better qualify the iterates, generated as approximate solutions to a sequence of subproblems. Using this variational characterization and the lifted representation, asymptotic properties and convergence guarantees are established for a safeguarded augmented Lagrangian scheme. Numerical examples showcase the modelling versatility gained by dropping convexity assumptions and the practical benefits of the advocated implicit approach.

Local properties and augmented Lagrangians in fully nonconvex composite optimization

“…Contributions With this work, we extend recent results by Rockafellar from [53,54], where composite optimization problems with convex 𝑔 are considered, to the more general setting. In particular, we study the implicit AL method from [16] and characterize its local convergence behavior. Particularly, under suitable conditions, we show convergence of the full sequence with linear or superlinear rate in Theorem 4.12.…”

“…in [17,35]. We are also interested in local properties now, with a focus on the numerical method proposed in [16], which favorably avoids the use of slack variables, see [6] for a recent study.…”

“…Furthermore, we comment on a reasonable choice for an AL function and investigate error bounds for a system of necessary optimality conditions associated with (P). In Section 4, we first introduce the AL method of our interest before presenting some global convergence results which complement the analysis provided in [16]. Then, local convergence results are presented.…”

“…A fundamental technique for solving constrained optimization problems is the augmented Lagrangian (AL) framework, which can effortlessly handle nonsmooth objectives, see e.g. [12,14,16,17,20,28,29,53,54,56] for some recent contributions, and [10,11,15] for some fundamental literature which addresses the setting of standard nonlinear programming. Particularly, Rockafellar extended the approach in [53,54] to the broad setting of (P) with 𝑔 convex, relying on some local duality to build a connection with the proximal point algorithm (PPA), see [49,50].…”

“…Here, we seek a better understanding of the variational properties of (P) and the convergence guarantees of AL methods for this class of problems. Building upon the global characterization in [16], we will focus on second-order optimality conditions and local analysis, portraying a convergence theory for the fully nonconvex setting including rates-of-convergence results.…”