CodedPrivateML: A Fast and Privacy-Preserving Framework for Distributed Machine Learning

So, Jinhyun; Güler, Başak; Avestimehr, A. Salman

doi:10.1109/jsait.2021.3053220

Cited by 53 publications

(23 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Secure Machine Learning. There has been a flurry of recent results ( [61], [75], [57], [25], [44]) in the area of secure machine learning, both in the 2-party [17], [67], [34], [55], [49], [58], [84], as well as in the 3-party setting [72], [60], [79], [12]. The most relevant to our work are ABY 3 [60] and SecureNN [79] that both provide 3-party semi-honest secure computation protocols for a variety of neural network inference and training algorithms, with somewhat similar performance guarantees.…”

Section: Related Workmentioning

confidence: 99%

CrypTFlow: Secure TensorFlow Inference

Kumar

Rathee

Chandran

et al. 2020

2020 IEEE Symposium on Security and Privacy (SP)

150

149

View full text Add to dashboard Cite

We present CRYPTFLOW, a first of its kind system that converts TensorFlow inference code into Secure Multi-party Computation (MPC) protocols at the push of a button. To do this, we build three components. Our first component, Athos, is an end-to-end compiler from TensorFlow to a variety of semihonest MPC protocols. The second component, Porthos, is an improved semi-honest 3-party protocol that provides significant speedups for TensorFlow like applications. Finally, to provide malicious secure MPC protocols, our third component, Aramis, is a novel technique that uses hardware with integrity guarantees to convert any semi-honest MPC protocol into an MPC protocol that provides malicious security. The malicious security of the protocols output by Aramis relies on integrity of the hardware and semi-honest security of MPC. Moreover, our system matches the inference accuracy of plaintext TensorFlow.We experimentally demonstrate the power of our system by showing the secure inference of real-world neural networks such as RESNET50 and DENSENET121 over the ImageNet dataset with running times of about 30 seconds for semi-honest security and under two minutes for malicious security. Prior work in the area of secure inference has been limited to semi-honest security of small networks over tiny datasets such as MNIST or CIFAR.

show abstract

Section: Related Workmentioning

confidence: 99%

CrypTFlow: Secure TensorFlow Inference

Kumar

Rathee

Chandran

et al. 2020

2020 IEEE Symposium on Security and Privacy (SP)

150

149

View full text Add to dashboard Cite

show abstract

“…This paper is one of the first attempts that propose a privacy-preserving machine learning solution using encryption and covers various architectural issues and problems such as the polynomial approximations or the fixed-depth of circuits. In CodedPrivateML [105], the authors train machine learning models (i.e., linear and logistic regression), using Shamir's secret sharing and speed it up with the Lagrange Coding. This paper proposes a solution based on cloud computing by distributing the workload to train the algorithms.…”

Section: Preceding Approachesmentioning

confidence: 99%

SoK: Privacy-Preserving Computation Techniques for Deep Learning

Cabrero-Holgueras

Pastrana

2021

Proceedings on Privacy Enhancing Technologies

View full text Add to dashboard Cite

Deep Learning (DL) is a powerful solution for complex problems in many disciplines such as finance, medical research, or social sciences. Due to the high computational cost of DL algorithms, data scientists often rely upon Machine Learning as a Service (MLaaS) to outsource the computation onto third-party servers. However, outsourcing the computation raises privacy concerns when dealing with sensitive information, e.g., health or financial records. Also, privacy regulations like the European GDPR limit the collection, distribution, and use of such sensitive data. Recent advances in privacy-preserving computation techniques (i.e., Homomorphic Encryption and Secure Multiparty Computation) have enabled DL training and inference over protected data. However, these techniques are still immature and difficult to deploy in practical scenarios. In this work, we review the evolution of the adaptation of privacy-preserving computation techniques onto DL, to understand the gap between research proposals and practical applications. We highlight the relative advantages and disadvantages, considering aspects such as efficiency shortcomings, reproducibility issues due to the lack of standard tools and programming interfaces, or lack of integration with DL frameworks commonly used by the data science community.

show abstract

“…It uses data from different places to effectively train different neural network models, which has attracted a lot of attention. Previous studies [47][48][49][50][51][52][53][54][55][56][57] have improved the performance of distributed learning systems from different perspectives. In order to maximize the overall performance of tasks in the cluster, Zhang et al [47] design a scheduling algorithm to approximate the training performance of deep learning jobs.…”

Section: Distributed Learningmentioning

confidence: 99%

SmartDL: energy-aware decremental learning in a mobile-based federation for geo-spatial system

et al. 2021

Neural Comput & Applic

View full text Add to dashboard Cite

Federated learning is designed to collaboratively train a shared model based on a large number of mobile devices while preserving data privacy, which has been widely adopted to support different geo-spatial systems. However, two critical issues prevent federated learning to be effectively deployed on resource-constrained devices in large scale. First, federated learning causes high energy consumption which can badly hurt the battery lifetime of mobile devices. Second, leakage of sensitive personal information still occurs during the training process. Thus, a system that can effectively protect the sensitive information while improving the energy efficiency is urgently required for a mobile-based federated learning system. This paper proposes SmartDL, an energy-aware decremental learning framework that well balances the energy efficiency and data privacy in an efficient manner. SmartDL improves the energy efficiency from two levels: (1) global layer, which adopts an optimization approach to select a subset of participating devices with sufficient capacity and maximum reward. (2) local layer, which adopts a novel decremental learning algorithm to actively provides the decremental and incremental updates, and can adaptively tune the local DVFS at the same time. We prototyped SmartDL on physical testbed and evaluated its performance using several learning benchmarks with real-world traces. The evaluation results show that compared with the original federated learning, SmartDL can reduce energy consumption by 75.6-82.4% in different datasets. Moreover, SmartDL achieves a speedup of 2-4 orders of magnitude in model convergence while ensuring the accuracy of the model.

show abstract

CodedPrivateML: A Fast and Privacy-Preserving Framework for Distributed Machine Learning

Cited by 53 publications

References 35 publications

CrypTFlow: Secure TensorFlow Inference

CrypTFlow: Secure TensorFlow Inference

SoK: Privacy-Preserving Computation Techniques for Deep Learning

SmartDL: energy-aware decremental learning in a mobile-based federation for geo-spatial system

Contact Info

Product

Resources

About