ADMM Based Privacy-Preserving Decentralized Optimization

Zhang, Chunlei; Ahmad, Muaz; Wang, Yongqiang

doi:10.1109/tifs.2018.2855169

Cited by 157 publications

(108 citation statements)

References 69 publications

Supporting

Mentioning

104

Contrasting

Order By: Relevance

“…Some of them employ cryptography-based methods in the protocol to hide the private information [32]- [35]. A recent work [34] uses partially homomorphic cryptography in ADMM-based distributed learning to preserve data privacy but the proposed approach cannot protect the information leakage of the private user data from the final learned models. In contrast, our approach provides differential privacy in the final trained machine learning models.…”

Section: Related Workmentioning

confidence: 99%

DP-ADMM: ADMM-Based Distributed Learning With Differential Privacy

Huang

Guo

et al. 2020

IEEE Trans.Inform.Forensic Secur.

155

176

View full text Add to dashboard Cite

Alternating direction method of multipliers (ADMM) is a widely used tool for machine learning in distributed settings, where a machine learning model is trained over distributed data sources through an interactive process of local computation and message passing. Such an iterative process could cause privacy concerns of data owners. The goal of this paper is to provide differential privacy for ADMM-based distributed machine learning. Prior approaches on differentially private ADMM exhibit low utility under high privacy guarantee and assume the objective functions of the learning problems to be smooth and strongly convex. To address these concerns, we propose a novel differentially private ADMM-based distributed learning algorithm called DP-ADMM, which combines an approximate augmented Lagrangian function with time-varying Gaussian noise addition in the iterative process to achieve higher utility for general objective functions under the same differential privacy guarantee. We also apply the moments accountant method to analyze the end-to-end privacy loss. The theoretical analysis shows that DP-ADMM can be applied to a wider class of distributed learning problems, is provably convergent, and offers an explicit utility-privacy tradeoff. To our knowledge, this is the first paper to provide explicit convergence and utility properties for differentially private ADMM-based distributed learning algorithms. The evaluation results demonstrate that our approach can achieve good convergence and model accuracy under high end-to-end differential privacy guarantee.

show abstract

Section: Related Workmentioning

confidence: 99%

DP-ADMM: ADMM-Based Distributed Learning With Differential Privacy

Huang

Guo

et al. 2020

IEEE Trans.Inform.Forensic Secur.

155

176

View full text Add to dashboard Cite

show abstract

“…Theorem III.1. [Sufficient Condition] Consider the modified ADMM defined by (20) (21). Let {f (t), Λ(t)} be outputs in each iteration and {f * , Λ * } a pair satisfying (22)(23).…”

Section: B Convergence Analysismentioning

confidence: 99%

“…Existing approaches to decentralizing the above problem primarily consist of subgradient-based algorithms [7]- [10] and ADMM-based algorithms [11]- [21]. It has been shown that ADMM-based algorithms can converge at the rate of O( 1 k ) while subgradient-based algorithms typically converge at the rate of O( 1 √ k ), where k is the number of iterations [16].…”

Section: Introductionmentioning

confidence: 99%

Recycled ADMM: Improve Privacy and Accuracy with Less Computation in Distributed Algorithms

Zhang

Khalili

Liu

2018

2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton)

View full text Add to dashboard Cite

Alternating direction method of multiplier (ADMM) is a powerful method to solve decentralized convex optimization problems. In distributed settings, each node performs computation with its local data and the local results are exchanged among neighboring nodes in an iterative fashion. During this iterative process the leakage of data privacy arises and can accumulate significantly over many iterations, making it difficult to balance the privacy-utility tradeoff. In this study we propose Recycled ADMM (R-ADMM), where a linear approximation is applied to every even iteration, its solution directly calculated using only results from the previous, odd iteration. It turns out that under such a scheme, half of the updates incur no privacy loss and require much less computation compared to the conventional ADMM. We obtain a sufficient condition for the convergence of R-ADMM and provide the privacy analysis based on objective perturbation.

show abstract

“…Hence, these cryptographic approaches are not suitable for distributed optimization which typically needs many iterations to converge. Indeed, [44] is the only work that we are aware of studying privacy-preserving decentralized optimization. Different from their proposed algorithm that based on ADMM and partially homomorphic cryptography, our construction of the privacypreserving part is non-cryptographic and simpler.…”

Section: B Numerical Resultsmentioning

confidence: 99%

A Market-Based Framework for Multi-Resource Allocation in Fog Computing

Nguyen

Bhargava

2019

IEEE/ACM Trans. Networking

View full text Add to dashboard Cite

Fog computing is transforming the network edge into an intelligent platform by bringing storage, computing, control, and networking functions closer to end-users, things, and sensors. How to allocate multiple resource types (e.g., CPU, memory, bandwidth) of capacity-limited heterogeneous fog nodes to competing services with diverse requirements and preferences in a fair and efficient manner is a challenging task. To this end, we propose a novel market-based resource allocation framework in which the services act as buyers and fog resources act as divisible goods in the market. The proposed framework aims to compute a market equilibrium (ME) solution at which every service obtains its favorite resource bundle under the budget constraint while the system achieves high resource utilization. This work extends the General Equilibrium literature by considering a practical case of satiated utility functions. Also, we introduce the notions of non-wastefulness and frugality for equilibrium selection, and rigorously demonstrate that all the non-wasteful and frugal ME are the optimal solutions to a convex program. Furthermore, the proposed equilibrium is shown to possess salient fairness properties including envy-freeness, sharing-incentive, and proportionality. Another major contribution of this work is to develop a privacy-preserving distributed algorithm, which is of independent interest, for computing an ME while allowing market participants to obfuscate their private information. Finally, extensive performance evaluation is conducted to verify our theoretical analyses.

show abstract

ADMM Based Privacy-Preserving Decentralized Optimization

Cited by 157 publications

References 69 publications

DP-ADMM: ADMM-Based Distributed Learning With Differential Privacy

DP-ADMM: ADMM-Based Distributed Learning With Differential Privacy

Recycled ADMM: Improve Privacy and Accuracy with Less Computation in Distributed Algorithms

A Market-Based Framework for Multi-Resource Allocation in Fog Computing

Contact Info

Product

Resources

About