Fast Alternating Direction Optimization Methods

Goldstein, Tom; O’Donoghue, Brendan; Setzer, Simon; Baraniuk, Richard G.

doi:10.1137/120896219

Cited by 663 publications

(641 citation statements)

References 44 publications

Supporting

Mentioning

622

Contrasting

Unclassified

Order By: Relevance

“…More recent convergence rate analysis of the ADMM still requires at least one of the component functions (f 1 or f 2 ) to be strongly convex and have a Lipschitz continuous gradient. Under these and additional rank conditions on the constraint matrix E, some linear convergence rate results can be obtained for a subset of primal and dual variables in the ADMM algorithm (or its variant); see [6,12,24]. However, when there are more than two blocks involved (K ≥ 3), the convergence (or the rate of convergence) of the ADMM method is unknown, and this has been a key open question for several decades.…”

Section: Alternating Direction Methods Of Multipliers (Admm)mentioning

confidence: 99%

On the linear convergence of the alternating direction method of multipliers

2016

View full text Add to dashboard Cite

We analyze the convergence rate of the alternating direction method of multipliers (ADMM) for minimizing the sum of two or more nonsmooth convex separable functions subject to linear constraints. Previous analysis of the ADMM typically assumes that the objective function is the sum of only two convex functions defined on two separable blocks of variables even though the algorithm works well in numerical experiments for three or more blocks. Moreover, there has been no rate of convergence analysis for the ADMM without strong convexity in the objective function. In this paper we establish the global linear convergence of the ADMM for minimizing the sum of any number of convex separable functions. This result settles a key question regarding the convergence of the ADMM when the number of blocks is more than two or if the strong convexity is absent. It also implies the linear convergence of the ADMM for several contemporary applications including LASSO, Group LASSO and Sparse Group LASSO without any strong convexity assumption. Our proof is based on estimating the distance from a dual feasible solution to the optimal dual solution set by the norm of a certain proximal residual, and by requiring the dual stepsize to be sufficiently small.KEY WORDS: Linear convergence, alternating directions of multipliers, error bound, dual ascent.

show abstract

Section: Alternating Direction Methods Of Multipliers (Admm)mentioning

confidence: 99%

On the linear convergence of the alternating direction method of multipliers

2016

View full text Add to dashboard Cite

show abstract

“…However, it is possible to accelerate the convergence of both algorithms to reach an optimal O(1/k 2 ) rate of the residuals [25]. Most importantly, these accelerated versions are obtained at a completely negligible cost, except for keeping in memory the fields from the previous iteration.…”

Section: Accelerated Versions Of Augmented Lagrangian Methodsmentioning

confidence: 99%

“…Most importantly, these accelerated versions are obtained at a completely negligible cost, except for keeping in memory the fields from the previous iteration. Following Nesterov's predictor-corrector scheme [25,26], the previous steps (6)-(9) are now replaced by:…”

Section: Accelerated Versions Of Augmented Lagrangian Methodsmentioning

confidence: 99%

“…Convergence of the accelerated ADMM version is ensured only if both functions G and H are strongly convex [25]. In the present case, H is only weakly convex and a restart procedure would need to be implemented to ensure convergence.…”

mentioning

confidence: 98%

“…This has not been done here since we always observed convergence of the accelerated ADMM algorithm, at least for antiplane simulations. Convergence of the accelerated AMA algorithm is however always ensured [25].…”

mentioning

confidence: 99%

See 2 more Smart Citations

Advances in the simulation of viscoplastic fluid flows using interior-point methods

Bleyer

2018

Computer Methods in Applied Mechanics and Engineering

View full text Add to dashboard Cite

We present a primal-dual interior point algorithm for the resolution of steady-state viscoplastic fluid flows formulated as a conic optimization problem. We give a complete description of the algorithm including some advanced aspects such as a predictor-corrector and scaling scheme to improve its efficiency. Our interior-point approach is shown to be largely more efficient than Augmented Lagrangian (AL) approaches which are traditionally used to solve such problems. In particular, the interior-point approach is roughly 5 times faster than the modern accelerated version of AL algorithms. The yield surfaces are shown to be accurately predicted and various examples ranging from channel flows to three-dimensional flows through a porous medium demonstrate its efficiency.

show abstract

Image Denoising Algorithms: From Wavelet Shrinkage to Nonlocal Collaborative Filtering

Pižurica¹

2017

Wiley Encyclopedia of Electrical and Electronics Engineering

View full text Add to dashboard Cite

This article presents an overview of image denoising algorithms ranging from wavelet shrinkage to patch‐based nonlocal processing. The focus is on the suppression of additive Gaussian noise (white and colored). A great attention is devoted to explaining the main underlying ideas and concepts of representative approaches, complemented by illustrative examples, accessible also to nonexperts in the field. A Bayesian perspective of wavelet shrinkage is given, with different instances of spatial context modeling (including local spatial activity indicators, Markov random fields, hidden Markov tree models, and Gaussian scale mixture models). Extensions to other transform domains (curvelets and other generalizations of wavelets) are addressed too, showing the benefits in terms of image quality. Patch‐based image denoising is illustrated with principles of nonlocal means filtering and collaborative filtering, explaining also the connections with dictionary learning. Some general notes on the performance comparison are given, by summarizing the benefits and limitations of various approaches against each other, and pointing to some of the current trends in the field.

show abstract

Fast Alternating Direction Optimization Methods

Cited by 663 publications

References 44 publications

On the linear convergence of the alternating direction method of multipliers

On the linear convergence of the alternating direction method of multipliers

Advances in the simulation of viscoplastic fluid flows using interior-point methods

Image Denoising Algorithms: From Wavelet Shrinkage to Nonlocal Collaborative Filtering

Contact Info

Product

Resources

About