An Inexact Augmented Lagrangian Framework for Nonconvex Optimization with Nonlinear Constraints

Sahin, Mehmet Fatih; Eftekhari, Armin; Alacaoglu, Ahmet; Latorre, Fabian; Cevher, Volkan

doi:10.48550/arxiv.1906.11357

Cited by 8 publications

(19 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Namely, we need to show that ALM converges to an AFAC point in polynomial time. A step in this direction was recently given by Sahin et al [31]. They proved that ALM computes a point satisfying the 2nd-order condition for the augmented Lagrangian function.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Polynomial time guarantees for the Burer-Monteiro method

Cifuentes¹,

Moitra²

2019

Preprint

View full text Add to dashboard Cite

The Burer-Monteiro method is one of the most widely used techniques for solving large-scale semidefinite programs (SDP). The basic idea is to solve a nonconvex program in Y , where Y is an n×p matrix such that X = Y Y T . In this paper, we show that this method can solve SDPs in polynomial time in an smoothed analysis setting. More precisely, we consider an SDP whose domain satisfies some compactness and smoothness assumptions, and slightly perturb the cost matrix and the constraints. We show that if p 2(1+η)m, where m is the number of constraints and η > 0 is any fixed constant, then the Burer-Monteiro method can solve SDPs to any desired accuracy in polynomial time, in the setting of smooth analysis. Our bound on p approaches the celebrated Barvinok-Pataki bound in the limit as η goes to zero, beneath which it is known that the nonconvex program can be suboptimal.Previous analyses were unable to give polynomial time guarantees for the Burer-Monteiro method, since they either assumed that the criticality conditions are satisfied exactly, or ignored the nontrivial problem of computing an approximately feasible solution. We address the first problem through a novel connection with tubular neighborhoods of algebraic varieties. For the feasibility problem we consider a least squares formulation, and provide the first guarantees that do not rely on the restricted isometry property.

show abstract

Section: Discussionmentioning

confidence: 99%

“…As for approximately 2-critical points, we are only aware of [17,31]. But both papers use a different 2nd-order condition, which is not easy to translate into our setting.…”

Section: Critical Points In Nonlinear Programmingmentioning

confidence: 99%

Polynomial time guarantees for the Burer-Monteiro method

Cifuentes¹,

Moitra²

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Paper [16] extends this previous work by presenting a hybrid penalty/AL-based method that also obtains the same aforementioned complexity. Finally, papers [26] and [15] respectively present O(ε −3 log ε −1 ) and O(ε −5/2 log ε −1 ) iteration complexities of some AL-based methods that perform under-relaxed Lagrange multiplier updates only when the penalty parameter is updated. It is worth noting that when these methods initialize the penalty parameter on the order of O(1), they perform a O(log ε −1 ) number of multiplier updates of the form (4) with θ = 0, χ = χ k , and χ k approaching 0.…”

Section: Introductionmentioning

confidence: 99%

An Accelerated Inexact Dampened Augmented Lagrangian Method for Linearly-Constrained Nonconvex Composite Optimization Problems

Kong¹,

Monteiro

2021

Preprint

View full text Add to dashboard Cite

This paper proposes and analyzes an accelerated inexact dampened augmented Lagrangian (AIDAL) method for solving linearly-constrained nonconvex composite optimization problems. Each iteration of the AIDAL method consists of: (i) inexactly solving a dampened proximal augmented Lagrangian (AL) subproblem by calling an accelerated composite gradient (ACG) subroutine; (ii) applying a dampened and under-relaxed Lagrange multiplier update; and (iii) using a novel test to check whether the penalty parameter of the AL function should be increased. Under several mild assumptions involving the dampening factor and the under-relaxation constant, it is shown that the AIDAL method generates an approximate stationary point of the constrained problem in O(ε −5/2 log ε −1 ) iterations of the ACG subroutine, for a given tolerance ε > 0. Numerical experiments are also given to show the computational efficiency of the proposed method.

show abstract

“…For the case where each component of g is convex and K = {0} × ℜ k ++ , i.e., the constraint is of the form g(x) = 0 and/or g(x) ≤ 0, papers [16,29] present PAL methods that perform Lagrange multiplier updates only when the penalty parameter is updated. Hence, if the penalty parameter is never updated (which usually happens when the initial penalty parameter is chosen to be sufficiently large), then these methods never perform Lagrange multiplier updates, and thus they behave more like penalty methods.…”

Section: Introductionmentioning

confidence: 99%

“…It is now worth discussing how NL-IAPIAL compares with the works [21,9,29,16,7,31]. First, it extends the IAPIAL method of [21] to the case where g is K-convex rather than K = {0} and g being affine.…”

Section: Introductionmentioning

confidence: 99%

Iteration-complexity of a proximal augmented Lagrangian method for solving nonconvex composite optimization problems with nonlinear convex constraints

Kong¹,

Melo²,

Monteiro³

2020

Preprint

View full text Add to dashboard Cite

This paper proposes and analyzes a proximal augmented Lagrangian (NL-IAPIAL) method for solving smooth nonconvex composite optimization problems with nonlinear K-convex constraints, i.e., the constraints are convex with respect to the order given by a closed convex cone K. Each NL-IAPIAL iteration consists of inexactly solving a proximal augmented Lagrangian subproblem by an accelerated composite gradient (ACG) method followed by a Lagrange multiplier update. Under some mild assumptions, it is shown that NL-IAPIAL generates an approximate stationary solution of the aforementioned problem in at most O(log(1/ρ)/ρ 3 ) inner iterations, where ρ > 0 is a given tolerance.

show abstract

An Inexact Augmented Lagrangian Framework for Nonconvex Optimization with Nonlinear Constraints

Cited by 8 publications

References 0 publications

Polynomial time guarantees for the Burer-Monteiro method

Polynomial time guarantees for the Burer-Monteiro method

An Accelerated Inexact Dampened Augmented Lagrangian Method for Linearly-Constrained Nonconvex Composite Optimization Problems

Iteration-complexity of a proximal augmented Lagrangian method for solving nonconvex composite optimization problems with nonlinear convex constraints

Contact Info

Product

Resources

About