An Accelerated Inexact Dampened Augmented Lagrangian Method for Linearly-Constrained Nonconvex Composite Optimization Problems

Abstract

This paper proposes and analyzes an accelerated inexact dampened augmented Lagrangian (AIDAL) method for solving linearly-constrained nonconvex composite optimization problems. Each iteration of the AIDAL method consists of: (i) inexactly solving a dampened proximal augmented Lagrangian (AL) subproblem by calling an accelerated composite gradient (ACG) subroutine; (ii) applying a dampened and under-relaxed Lagrange multiplier update; and (iii) using a novel test to check whether the penalty parameter of the AL function should be increased. Under several mild assumptions involving the dampening factor and the under-relaxation constant, it is shown that the AIDAL method generates an approximate stationary point of the constrained problem in O(ε −5/2 log ε −1 ) iterations of the ACG subroutine, for a given tolerance ε > 0. Numerical experiments are also given to show the computational efficiency of the proposed method.

Global Complexity Bound of a Proximal ADMM for Linearly-Constrained Nonseperable Nonconvex Composite Programming

“…Lemma 3.6. Let δ k , B δ , and β λ (•) be as in Lemma 3.4,(17), and (18), respectively. Then, it holds that…”

“…This subsection presents some specialized Lagrange multiplier bounds and generalizes the analysis in [17].…”

“…Lemma 3.8. Let κ 2 and τ 1 (•) be as in (17) and (18), respectively. Then, the following statements hold:…”

“…Lemma 3.10. Let (κ 3 , κ 4 ) and (τ 2 , τ 4 ) be as in (17) and (18), respectively. If ξ k ≤ ξ and k ≥ 2, then the following statements hold…”

“…For any S ∈ R m×n and u ∈ S(R m×n ), we have σ + S u ≤ Su .The proof of the following result can be found in[17, Lemma B.2]. For any (τ, θ) ∈ [0, 1] 2 satisfying τ ≤ θ 2 and any a, b ∈ R n , we have thata − (1 − θ)b 2 − τ a 2 ≥ (1 − τ ) − (1 − θ) 2 (38)Finally, the proof of the next result can be found in[18, Lemma 4.7].…”