An adaptive superfast inexact proximal augmented Lagrangian method for smooth nonconvex composite optimization problems

Abstract

This work presents an adaptive superfast proximal augmented Lagrangian (AS-PAL) method for solving linearly-constrained smooth nonconvex composite optimization problems. At each iteration, AS-PAL inexactly solves a possibly nonconvex proximal augmented Lagrangian subproblem with prox stepsize chosen aggressively large so as to speed up its termination. An adaptive ACG variant of FISTA, namely R-FISTA, is then used to possibly solve the above subproblem. AS-PAL then adaptively updates the prox stepsizes based off the possible failures of R-FISTA. An interesting feature of AS-PAL is that it does not require knowledge of the parameters (e.g., size of constraint matrix, objective function curvatures, etc) underlying the problem. We demonstrate the speed and efficiency of our method through extensive computational experiments which show that AS-PAL can be more than 10 times faster than all the other state-ofthe-art codes on many instances (particularly when high accuracy is required).

Single-Loop Switching Subgradient Methods for Non-Smooth Weakly Convex Optimization with Non-Smooth Convex Constraints

“…In recent literature, the prevalent classes of algorithms for non-convex constrained optimization are the augmented Lagrangian method (ALM) and the penalty method [35,80,81,82,37,38,41,44,57,36,56,46,71,45,53,65,54,52]. Another actively studied class of algorithms is the sequential quadratic programming method [10,33,5,11,23,9,8,59,7,22].…”

“…However, most of the existing algorithms and their theoretical complexity analysis are developed by assuming f and g i 's are all smooth or can be written as the sum of a smooth and a simple non-smooth functions. A non-exhaustive list of the works under such an assumption includes [10,33,81,82,38,41,57,36,56,46,71,45,53,65,54,52,11,14,23,8,59,22]. Their results cannot be applied to (1) due to non-smoothness in the problem.…”