CrypTen: Secure Multi-Party Computation Meets Machine Learning

Knott, Brian; Venkataraman, Shobha; Hannun, Awni; Sengupta, Shubho; Ibrahim, Mark; Maaten, Laurens van der

doi:10.48550/arxiv.2109.00984

Cited by 5 publications

(15 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All experiments are developed in Python. The experimental settings of FL mainly follow the previous work [62], and the cryptographic protocols are implemented under CrypTen library [34].…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

PRECAD: Privacy-Preserving and Robust Federated Learning via Crypto-Aided Differential Privacy

Gu¹,

Li²,

Xiong³

2021

Preprint

View full text Add to dashboard Cite

Federated Learning (FL) allows multiple participating clients to train machine learning models collaboratively by keeping their datasets local and only exchanging model updates. Existing FL protocol designs have been shown to be vulnerable to attacks that aim to compromise data privacy and/or model robustness. Recently proposed defenses focused on ensuring either privacy or robustness, but not both. In this paper, we develop a framework called PRECAD, which simultaneously achieves differential privacy (DP) and enhances robustness against model poisoning attacks with the help of cryptography. Using secure multi-party computation (MPC) techniques (e.g., secret sharing), noise is added to the model updates by the honest-but-curious server(s) (instead of each client) without revealing clients' inputs, which achieves the benefit of centralized DP in terms of providing a better privacy-utility tradeoff than local DP based solutions. Meanwhile, a crypto-aided secure validation protocol is designed to verify that the contribution of model update from each client is bounded without leaking privacy. We show analytically that the noise added to ensure DP also provides enhanced robustness against malicious model submissions. We experimentally demonstrate that our PRECAD framework achieves higher privacy-utility tradeoff and enhances robustness for the trained models.

show abstract

“…All experiments are developed in Python. The experimental settings of FL mainly follow the previous work [62], and the cryptographic protocols are implemented under CrypTen library [34].…”

Section: Discussionmentioning

confidence: 99%

“…Step 3. The servers can compute the shares [1(y 0)] A and [1(y 0)] B via the comparison gate (refer to [34] for more details). Finally, by exchanging the above two shares with each other, both servers can compute the value Valid(x) = 1(y 0) = [1(y 0)] A + [1(y 0)] B .…”

Section: Secure Validationmentioning

confidence: 99%

PRECAD: Privacy-Preserving and Robust Federated Learning via Crypto-Aided Differential Privacy

Gu¹,

Li²,

Xiong³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…DeepSecure (Rouhani et al, 2018), EzPC (Chandran et al, 2019), and Chamelon (Riazi et al, 2018) use GC for nonlinear operations and secret-sharing for linear operations. Similarly, CryptGPU (Tan et al, 2021) and CrypTen (Knott et al, 2021) use additive secret sharing for linear operations and binary secret sharing for nonlinear operation and rely on trusted-third party (TTP) for generating the beaver triples.…”

Section: Categorization Of Pi Methodsmentioning

confidence: 99%

CryptoNite: Revealing the Pitfalls of End-to-End Private Inference at Scale

Garimella¹,

Jha²,

Ghodsi³

et al. 2021

Preprint

View full text Add to dashboard Cite

The privacy concerns of providing deep learning inference as a service have underscored the need for private inference (PI) protocols that protect users' data and the service provider's model using cryptographic methods. Recently proposed PI protocols have achieved significant reductions in PI latency by moving the computationally heavy homomorphic encryption (HE) parts to an offline/pre-compute phase. Paired with recent optimizations that tailor networks for PI, these protocols have achieved performance levels that are tantalizingly close to being practical. In this paper, we conduct a rigorous end-to-end characterization of PI protocols and optimization techniques and find that the current understanding of PI performance is overly optimistic. Specifically, we find that offline storage costs of garbled circuits (GC), a key cryptographic protocol used in PI, on user/client devices are prohibitively high and force much of the expensive offline HE computation to the online phase, resulting in a 10-1000× increase to PI latency. We propose a modified PI protocol that significantly reduces client-side storage costs for a small increase in online latency. Evaluated end-to-end, the modified protocol outperforms current protocols by reducing the mean PI latency by 4× for ResNet18 on TinyImageNet. We conclude with a discussion of several recently proposed PI optimizations in light of the findings and note many actually increase PI latency when evaluated from an end-to-end perspective.

show abstract

“…Last, we point to some efforts to perform ML training under different notions of privacy and security: the Myelin system makes use of secure hardware to add noise centrally [6], while Crypten [7] does training in a secure multi-party setting, without explicit noise addition.…”

Section: Related Workmentioning

confidence: 99%

Opacus: User-Friendly Differential Privacy Library in PyTorch

Yousefpour¹,

Шилов²,

Sablayrolles³

et al. 2021

Preprint

View full text Add to dashboard Cite

We introduce Opacus, a free, open-source PyTorch library for training deep learning models with differential privacy (hosted at opacus.ai). Opacus is designed for simplicity, flexibility, and speed. It provides a simple and user-friendly API, and enables machine learning practitioners to make a training pipeline private by adding as little as two lines to their code. It supports a wide variety of layers, including multi-head attention, convolution, LSTM, and embedding, right out of the box, and it also provides the means for supporting other user-defined layers. Opacus computes batched per-sample gradients, providing better efficiency compared to the traditional "micro batch" approach. In this paper we present Opacus, detail the principles that drove its implementation and unique features, and compare its performance against other frameworks for differential privacy in ML.

show abstract

CrypTen: Secure Multi-Party Computation Meets Machine Learning

Cited by 5 publications

References 47 publications

PRECAD: Privacy-Preserving and Robust Federated Learning via Crypto-Aided Differential Privacy

PRECAD: Privacy-Preserving and Robust Federated Learning via Crypto-Aided Differential Privacy

CryptoNite: Revealing the Pitfalls of End-to-End Private Inference at Scale

Opacus: User-Friendly Differential Privacy Library in PyTorch

Contact Info

Product

Resources

About