Weighted ADMM for Fast Decentralized Network Optimization

Alternating direction method of multiplier (ADMM) is a powerful method to solve decentralized convex optimization problems. In distributed settings, each node performs computation with its local data and the local results are exchanged among neighboring nodes in an iterative fashion. During this iterative process the leakage of data privacy arises and can accumulate significantly over many iterations, making it difficult to balance the privacy-accuracy tradeoff. We propose Recycled ADMM (R-ADMM), where a linear approximation is applied to every even iteration, its solution directly calculated using only results from the previous, odd iteration. It turns out that under such a scheme, half of the updates incur no privacy loss and require much less computation compared to the conventional ADMM. Moreover, R-ADMM can be further modified (MR-ADMM) such that each node independently determines its own penalty parameter over iterations. We obtain a sufficient condition for the convergence of both algorithms and provide the privacy analysis based on objective perturbation. It can be shown that the privacyaccuracy tradeoff can be improved significantly compared with conventional ADMM.

show abstract

“…Lemma IV.1. [First-order Optimality Condition [16]] Under Assumptions 1 and 2, the following two statements are equivalent:…”

Section: Convergence Of Non-private Mr-admmmentioning

confidence: 99%

“…Existing approaches to decentralizing the above problem primarily consist of subgradient-based algorithms [7]- [9] and ADMM-based algorithms [10]- [16]. It has been shown that ADMM-based algorithms can converge at the rate of O( 1 k )…”

Section: Introductionmentioning

confidence: 99%

Recycled ADMM: Improving the Privacy and Accuracy of Distributed Algorithms

Zhang

Khalili

Liu

2020

IEEE Trans.Inform.Forensic Secur.

View full text Add to dashboard Cite

show abstract

“…A distributed algorithm combining a linearization approach with ADMM has been proposed in [109], while quadratic approximations have been explored in [110]. A fast distributed ADMM algorithm for quadratic problems is devised in [111]. A more general ADMM framework is considered in [112], where an explicit converge rate has been provided.…”

Section: Discussion and Referencesmentioning

confidence: 99%

Distributed Optimization for Smart Cyber-Physical Networks

Notarstefano¹,

Notarnicola²,

Camisa³

2019

View full text Add to dashboard Cite

The presence of embedded electronics and communication capabilities as well as sensing and control in smart devices has given rise to the novel concept of cyber-physical networks, in which agents aim at cooperatively solving complex tasks by local computation and communication. Numerous estimation, learning, decision and control tasks in smart networks involve the solution of large-scale, structured optimization problems in which network agents have only a partial knowledge of the whole problem. Distributed optimization aims at designing local computation and communication rules for the network processors allowing them to cooperatively solve the global optimization problem without relying on any central unit. The purpose of this survey is to provide an introduction to distributed optimization methodologies. Principal approaches, namely (primal) consensus-based, duality-based and constraint exchange methods, are formalized. An analysis of the basic schemes is supplied, and state-of-the-art extensions are reviewed.

show abstract

“…This problem has found wide applications in various domains, ranging from rendezvous in multi-agent systems [1], support vector machine [2] and classification [3] in machine learning, source localization in sensor networks [4], to data regression in statistics [5], [6]. To solve the optimization problem (1) in an decentralized manner, different algorithms were proposed in recent years, including the distributed (sub)gradient algorithm [7], augmented Lagrangian methods (ALM) [8], and the alternating direction method of multipliers (ADMM) as well as its variants [8]- [11]. Among existing approaches, ADMM has attracted tremendous attention due to its wide applications [9] and fast convergence rate in both primal and dual iterations [11].…”

Section: Introductionmentioning

confidence: 99%

ADMM Based Privacy-Preserving Decentralized Optimization

Zhang

Ahmad

Wang

2019

IEEE Trans.Inform.Forensic Secur.

157

104

View full text Add to dashboard Cite

This paper considers the problem of privacypreservation in decentralized optimization, in which N agents cooperatively minimize a global objective function that is the sum of N local objective functions. We assume that each local objective function is private and only known to an individual agent. To cooperatively solve the problem, most existing decentralized optimization approaches require participating agents to exchange and disclose estimates to neighboring agents. However, this results in leakage of private information about local objective functions, which is undesirable when adversaries exist and try to steal information from participating agents. To address this issue, we propose a privacy-preserving decentralized optimization approach based on proximal Jacobian ADMM via function decomposition. Numerical simulations confirm the effectiveness of the proposed approach.

show abstract

Weighted ADMM for Fast Decentralized Network Optimization

Cited by 41 publications

References 36 publications

Recycled ADMM: Improving the Privacy and Accuracy of Distributed Algorithms

Recycled ADMM: Improving the Privacy and Accuracy of Distributed Algorithms

Distributed Optimization for Smart Cyber-Physical Networks

ADMM Based Privacy-Preserving Decentralized Optimization

Contact Info

Product

Resources

About