Alternating Proximal Gradient Method for Convex Minimization

Ma, Shiqian

doi:10.1007/s10915-015-0150-0

Cited by 62 publications

(56 citation statements)

References 76 publications

(113 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, when there are more than two blocks involved (K ≥ 3), the convergence (or the rate of convergence) of the ADMM method is unknown, and this has been a key open question for several decades. The recent work [38] describes a list of novel applications of the ADMM with K ≥ 3 and motivates strongly for the need to analyze the convergence of the ADMM in the multi-block case. The recent monograph [7] contains more details of the history, convergence analysis and applications of the ADMM and related methods.…”

Section: Alternating Direction Methods Of Multipliers (Admm)mentioning

confidence: 99%

On the linear convergence of the alternating direction method of multipliers

2016

View full text Add to dashboard Cite

We analyze the convergence rate of the alternating direction method of multipliers (ADMM) for minimizing the sum of two or more nonsmooth convex separable functions subject to linear constraints. Previous analysis of the ADMM typically assumes that the objective function is the sum of only two convex functions defined on two separable blocks of variables even though the algorithm works well in numerical experiments for three or more blocks. Moreover, there has been no rate of convergence analysis for the ADMM without strong convexity in the objective function. In this paper we establish the global linear convergence of the ADMM for minimizing the sum of any number of convex separable functions. This result settles a key question regarding the convergence of the ADMM when the number of blocks is more than two or if the strong convexity is absent. It also implies the linear convergence of the ADMM for several contemporary applications including LASSO, Group LASSO and Sparse Group LASSO without any strong convexity assumption. Our proof is based on estimating the distance from a dual feasible solution to the optimal dual solution set by the norm of a certain proximal residual, and by requiring the dual stepsize to be sufficiently small.KEY WORDS: Linear convergence, alternating directions of multipliers, error bound, dual ascent.

show abstract

Section: Alternating Direction Methods Of Multipliers (Admm)mentioning

confidence: 99%

On the linear convergence of the alternating direction method of multipliers

2016

View full text Add to dashboard Cite

show abstract

“…Remark 2: DLM differs from the centralized linearized ADMM in [25], [26] in that the latter linearizes the quadratic term in the augmented Lagrangian in (5)-while DLM linearizes the objective function . The centralized linearized ADMM in [27] applies to objectives of the form and uses linearized versions of these functions in both, the iteration in (5) and the iteration in (6).…”

Section: Algorithm 1 Dlm Algorithm Run By Agentmentioning

confidence: 99%

DLM: Decentralized Linearized Alternating Direction Method of Multipliers

Ling

Shi

et al. 2015

IEEE Trans. Signal Process.

190

153

View full text Add to dashboard Cite

This paper develops the Decentralized Linearized Alternating Direction Method of Multipliers (DLM) that minimizes a sum of local cost functions in a multiagent network. The algorithm mimics operation of the decentralized alternating direction method of multipliers (DADMM) except that it linearizes the optimization objective at each iteration. This results in iterations that, instead of successive minimizations, implement steps whose cost is akin to the much lower cost of the gradient descent step used in the distributed gradient method (DGM). The algorithm is proven to converge to the optimal solution when the local cost functions have Lipschitz continuous gradients. Its rate of convergence is shown to be linear if the local cost functions are further assumed to be strongly convex. Numerical experiments in least squares and logistic regression problems show that the number of iterations to achieve equivalent optimality gaps are similar for DLM and ADMM and both much smaller than those of DGM. In that sense, DLM combines the rapid convergence of ADMM with the low computational burden of DGM. Index Terms-Decentralized optimization, linearized alternating direction method of multipliers, multiagent network.1053-587X

show abstract

“…For multiple convex optimization problems, Y. XU and W. YIN put forward a kind of based on BPG (Block proximal gradient) solving method [16]; S. MA puts forward a kind of based on APG (Alternating Proximal gradient) solving method [17]. In this paper, we will combine the two methods to solve type such as formula (3) …”

Section: Accelerate the Proximal Gradient Methods Based On Blockmentioning

confidence: 99%