A Convergent 3-Block Semi-Proximal ADMM for Convex Minimization Problems with One Strongly Convex Block

Abstract

In this paper, we present a semi-proximal alternating direction method of multipliers (ADMM) for solving 3-block separable convex minimization problems with the second block in the objective being a strongly convex function and one coupled linear equation constraint. By choosing the semi-proximal terms properly, we establish the global convergence of the proposed semiproximal ADMM for the step-length τ ∈ (0, (1+ √ 5)/2) and the penalty parameter σ ∈ (0, +∞). In particular, if σ > 0 is smaller than a certain threshold and the first and third linear operators in the linear equation constraint are injective, then all the three added semi-proximal terms can be dropped and consequently, the convergent 3-block semi-proximal ADMM reduces to the directly extended 3-block ADMM with τ ∈ (0, (1 + √ 5)/2).

Convergence analysis of the direct extension of ADMM for multiple-block separable convex minimization

“…Therefore, to discuss the more difficult case where only some of the functions θ i 's are assumed to be strongly convex, we shall also restrict the penalty parameter into certain intervals when discussing the the convergence of e-ADMM (1.3a)-(1.3e) with m ≥ 3. Indeed, as we shall elucidate, the range of β, which is eligible to the case with a generic m, is even larger than those in [1,16] when it reduces to the special case of m = 3. That is, we shall prove the convergence for the e-ADMM (1.3) by requiring only (m − 2) strongly convex functions and a larger range of β for m ≥ 3.…”

“…Moreover, we shall establish the worst-case O(1/t) convergence rate in the ergodic sense for m ≥ 3, where t is the iteration counter; and explore some stronger conditions that can ensure the asymptotically linear convergence for m ≥ 3. Thus, compared with existing work in the same category such as [1,4,12,16,17,18], the convergence results in this paper are more general and they are proved under weaker conditions. The rest of this paper is organized as follows.…”

“…Still requiring the strong convexity of two functions, the work [17] proves some refined convergence results such as the O(1/t) ergodic convergence rate and o(1/t) non-ergodic convergence rate measured by the iteration complexity, where t is the iteration number. Later, the results in [4,17] were improved in [1,16], in which the convergence of (1.3a)-(1.3e) was obtained with only one strongly convex function for the case m = 3. According to the results in [3], the strong convexity of at least one function seems minimal for the special case of m = 3 of (1.1) to ensure the convergence of (1.3a)-(1.3e); and the work in [1,16] verifies this conclusion positively.…”

“…In addition to the strong convexity of some or all the functions in the objective of (1.1), it is worthwhile to mention that the penalty parameter β in (1.3) should be appropriately restricted to theoretically ensure the convergence, see, e.g., all the work [1,4,12,16,17,18] 1 . According to Theorem 4.1 in [3], even all the functions θ i 's in the objective of (1.1) are strongly convex, the scheme (1.3a)-(1.3e) with m = 3 may be divergent if the penalty parameter β is not well restricted.…”

Recovery of Corrupted Low-Rank Tensors

“…Cai et al [24] and Li et al [25] have proved the convergence of Extended Alternating Direction Method of Multipliers (e-ADMM) with only one strongly convex function for the case m = 3. Assumption 1.…”