Improved Pointwise Iteration-Complexity of A Regularized ADMM and of a Regularized Non-Euclidean HPE Framework

Abstract

This paper describes a regularized variant of the alternating direction method of multipliers (ADMM) for solving linearly constrained convex programs. It is shown that the pointwise iteration-complexity of the new variant is better than the corresponding one for the standard ADMM method and that, up to a logarithmic term, is identical to the ergodic iteration-complexity of the latter method. Our analysis is based on first presenting and establishing the pointwise iteration-complexity of a regularized non-Euclidean hybrid proximal extragradient framework whose error condition at each iteration includes both a relative error and a summable error. It is then shown that the new ADMM variant is a special instance of the latter framework where the sequence of summable errors is identically zero when the ADMM stepsize is less than one or a nontrivial sequence when the stepsize is in the interval [1, (1 + √ 5)/2).2000 Mathematics Subject Classification: 47H05, 47J22, 49M27, 90C25, 90C30, 90C60, 65K10.

An Inexact Spingarn’s Partial Inverse Method with Applications to Operator Splitting and Composite Optimization

“…The first and second identities in (17) give that they also satisfy (10). Altogether, we have that the iteration defined in (17) is generated by Algorithm 1 for solving (5) with T given in (16). On the other hand, it follows from (17) and the assumption σ > 2/5 that…”

“…The HPE method has been used for many authors [3,4,5,6,7,8,9,10,11,12,13,14] as a framework for the design and analysis of several algorithms for monotone inclusion problems, variational inequalities, saddlepoint problems and convex optimization. Its iteration-complexity has been established recently by Monteiro and Svaiter [15] and, as a consequence, it has proved the iteration-complexity of various important algorithms in optimization (which use the HPE method as a framework) including Tseng's forward-backward method, Korpelevich extragradient method and the alternating direction method of multipliers (ADMM) [12,15,16].…”

Iteration-complexity analysis of a generalized alternating direction method of multipliers

“…Since z * is an arbitrary solution of (13), the result follows from the definition of ρ k , d 0 , and η 0 given in (11), (18) and (30), respectively.…”

Cauchy Noise Removal by Nonconvex ADMM with Convergence Guarantees

“…Recently, researchers have discovered some useful convergence properties of the optimization algorithms for solving nonconvex minimization problems [24,47,48,53]. In particular, the paper [48] established the global convergence (to a stationary point) of the alternating direction method of multipliers (ADMM) for nonconvex nonsmooth optimization with linear constraints.…”