A Majorized ADMM with Indefinite Proximal Terms for Linearly Constrained Convex Composite Optimization

Abstract

This paper presents a majorized alternating direction method of multipliers (ADMM) with indefinite proximal terms for solving linearly constrained 2-block convex composite optimization problems with each block in the objective being the sum of a non-smooth convex function (p(x) or q(y)) and a smooth convex function (f (x) or g(y)), i.e., min x∈X , y∈Y {p(x)+f (x)+q(y)+g(y) | A * x + B * y = c}. By choosing the indefinite proximal terms properly, we establish the global convergence, and the iteration-complexity in both the non-ergodic and ergodic senses of the proposed method for the step-length τ ∈ (0, (1 + √ 5)/2). The computational benefit of using indefinite proximal terms within the ADMM framework instead of the current requirement of positive semidefinite ones is also demonstrated numerically. This opens up a new way to improve the practical performance of the ADMM and related methods.

On the equivalence of inexact proximal ALM and ADMM for a class of convex composite programming

“…The majorized augmented Lagrangian function associated with problem (3.1) is defined by In the following, we propose an inexact majorized indefinite-proximal ALM in Algorithm iPALM for solving problem (3.1). This algorithm is an extension of the proximal method of multipliers developed by Rockafellar [45], with new ingredients added based on the recent progress on using proximal terms which are not necessarily positive definite [16,29,57] and the implementable inexact minimization criteria studied in [6]. For the convenience of later convergence analysis, we make the following blanket assumption.…”

“…where τ ∈ 0, (1 + √ 5)/2 was allowed in [6]. As one can observe from (1.5) and (1.6), the quadratic majorization technique in Li et al [29] was used to replace the original augmented Lagrangian function by the majorized augmented Lagrangian function. This in turn enables us to employ the inexact block sGS decomposition technique in Li et al [32] to sequentially update the sub-blocks of y individually.…”

“…This question was first resolved in[48] when the initial multiplier x 0 satisfies Gx 0 − b = 0 and all the subproblems are solved exactly 2. One may refer to[29] for the details that motivating the use of indefinite proximal terms in the 2-block majorized proximal ADMM, especially [29, Section 6] on their computational merits, as well as[57] for the similar results in multi-block cases.…”

The augmented Lagrangian method with full Jacobian decomposition and logarithmic-quadratic proximal regularization for multiple-block separable convex programming

“…However, it was shown in [6] that the scheme (1.6) is not necessarily convergent. The convergence rate of ADMM and its extension are analysed in [9,28,30,33,32].…”

A linearly convergent majorized ADMM with indefinite proximal terms for convex composite programming and its applications

“…It is well known that if S = 0 and T = 0, the iterative scheme (1.2) is exactly the classic ADMM designed by Glowinski and Marroco [22] and Gabay and Mercier [21]; if S 0, T 0 and τ = 1, iterative scheme (1.2) reduces to the method of the proximal ADMM introduced by Eckstein [14]; if both S and T are self-adjoint positive semidefinite linear operators, τ ∈ (0, (1 + √ 5)/2), iterative scheme (1.2) is known as the semiproximal ADMM (sPADMM) which is proposed by Fazel et al [17]. To know more about the above mentioned works and their relationships with well known methods, such as proximal point algorithm (PPA) and Douglas-Rachford (DR) splitting method, we refer the readers to [22,18,16,15,14,23,7,24,34,27].…”