Convergence Rate Analysis for the Alternating Direction Method of Multipliers with a Substitution Procedure for Separable Convex Programming

He, Bingsheng; Tao, Min; Yuan, Xiaoming

doi:10.1287/moor.2016.0822

Cited by 41 publications

(29 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the augmented Lagrangian function given in (9), if we delete the first term which depends on the dual variable µ, we obtain the quadratic function 1 2 Ax − b 2 . Thus if we eliminate the dual variable µ in the update equations of RP-ADMM, we will obtain the update equations for RP-BCD.…”

Section: Randomly Permuted Bcdmentioning

confidence: 99%

On the Efficiency of Random Permutation for ADMM and Coordinate Descent

Sun

Luo

2020

Mathematics of OR

View full text Add to dashboard Cite

Random permutation is observed to be powerful for optimization algorithms: for multi-block ADMM (alternating direction method of multipliers), while the classical cyclic version divergence, the randomly permuted version converges in practice; for BCD (block coordinate descent), the randomly permuted version is typically faster than other versions. In this paper, we provide strong theoretical evidence that random permutation has positive effects on ADMM and BCD, by analyzing randomly permuted ADMM (RP-ADMM) for solving linear systems of equations, and randomly permuted BCD (RP-BCD) for solving unconstrained quadratic problems. First, we prove that RP-ADMM converges in expectation for solving systems of linear equations. The key technical result is that the spectrum of the expected update matrix of RP-BCD lies in (−1/3, 1), instead of the typical range (−1, 1). Second, we establish expected convergence rates of RP-ADMM for solving linear sytems and RP-BCD for solving unconstrained quadratic problems. This expected rate of RP-BCD is O(n) times better than the worst-case rate of cyclic BCD, thus establishing a gap of at least O(n) between RP-BCD and cyclic BCD. To analyze RP-BCD, we propose a conjecture of a new matrix AM-GM (algebraic mean-geometric mean) inequality, and prove a weaker version of it. * This paper is a strengthened version of a previous technical report "On the expected convergence of randomly permuted ADMM" appeared on arxiv on April 2015, with several new results, mainly the ones on the expected convergence rate of RP-CD and RP-ADMM.

show abstract

Section: Randomly Permuted Bcdmentioning

confidence: 99%

On the Efficiency of Random Permutation for ADMM and Coordinate Descent

Sun

Luo

2020

Mathematics of OR

View full text Add to dashboard Cite

show abstract

“…In this section, we introduce several sophisticated modifications of ADMM, Variable splitting ADMM [9], [10], [25], ADMM with Gaussian Back Substitution [21], [26] and Proximal Jacobian ADMM [24], [27], to deal with the multiblock setting.…”

Section: Multi-block Admmmentioning

confidence: 99%

Multi-block ADMM for big data optimization in smart grid

Liu

Han

2015

2015 International Conference on Computing, Networking and Communications (ICNC)

View full text Add to dashboard Cite

Abstract-In this paper, we review the parallel and distributed optimization algorithms based on alternating direction method of multipliers (ADMM) for solving "big data" optimization problem in smart grid communication networks. We first introduce the canonical formulation of the large-scale optimization problem. Next, we describe the general form of ADMM and then focus on several direct extensions and sophisticated modifications of ADMM from 2-block to N -block settings to deal with the optimization problem. The iterative schemes and convergence properties of each extension/modification are given, and the implementation on large-scale computing facilities is also illustrated. Finally, we numerate several applications in power system for distributed robust state estimation, network energy management and security constrained optimal power flow problem.

show abstract

“…III. MULTI-BLOCK ADMM In this section, we introduce several sophisticated modifications of ADMM, Variable splitting ADMM [9], [10], [25], ADMM with Gaussian Back Substitution [21], [26] and Proximal Jacobian ADMM [24], [27], to deal with the multiblock setting.…”

Section: Algorithm 3 Jacobian Multi-block Admmmentioning

confidence: 99%

Multi-Block ADMM for Big Data Optimization in Modern Communication Networks

Liu¹,

Han²

2015

jcm

View full text Add to dashboard Cite

Abstract-In this paper, we review the parallel and distributed optimization algorithms based on the alternating direction method of multipliers (ADMM) for solving "big data" optimization problems in modern communication networks. We first introduce the canonical formulation of the large-scale optimization problem. Next, we describe the general form of ADMM and then focus on several direct extensions and sophisticated modifications of ADMM from 2-block to N -block settings to deal with the optimization problem. The iterative schemes and convergence properties of each extension/modification are given, and the implementation on large-scale computing facilities is also illustrated. Finally, we numerate several applications in communication networks, such as the security constrained optimal power flow problem in smart grid networks and mobile data offloading problem in software defined networks (SDNs).

show abstract

Convergence Rate Analysis for the Alternating Direction Method of Multipliers with a Substitution Procedure for Separable Convex Programming

Cited by 41 publications

References 42 publications

On the Efficiency of Random Permutation for ADMM and Coordinate Descent

On the Efficiency of Random Permutation for ADMM and Coordinate Descent

Multi-block ADMM for big data optimization in smart grid

Multi-Block ADMM for Big Data Optimization in Modern Communication Networks

Contact Info

Product

Resources

About