FLASH: Fast and Robust Framework for Privacy-preserving Machine Learning

Byali, Megha; Chaudhari, Harsh; Patra, Arpita; Suresh, Ananda Theertha

doi:10.2478/popets-2020-0036

Cited by 80 publications

(104 citation statements)

References 41 publications

Supporting

Mentioning

104

Contrasting

Order By: Relevance

“…Trident achieves the same result in a 4PC model with further performance improvements. FLASH [17] also proposes a 4PC model that achieves malicious security with guaranteed output delivery. QuantizedNN [14] proposes an efficient PPML framework using the quantization scheme of Jacob et al [54] and provides protocols in all combinations of semi-honest/malicious security and honest majority vs dishonest majority corruptions.…”

Section: Privatementioning

confidence: 99%

Falcon: Honest-Majority Maliciously Secure Framework for Private Deep Learning

Wagh

Tople

Benhamouda³

et al. 2020

Proceedings on Privacy Enhancing Technologies

146

View full text Add to dashboard Cite

We propose Falcon, an end-to-end 3-party protocol for efficient private training and inference of large machine learning models. Falcon presents four main advantages – (i) It is highly expressive with support for high capacity networks such as VGG16 (ii) it supports batch normalization which is important for training complex networks such as AlexNet (iii) Falcon guarantees security with abort against malicious adversaries, assuming an honest majority (iv) Lastly, Falcon presents new theoretical insights for protocol design that make it highly efficient and allow it to outperform existing secure deep learning solutions. Compared to prior art for private inference, we are about 8× faster than SecureNN (PETS’19) on average and comparable to ABY3 (CCS’18). We are about 16 − 200× more communication efficient than either of these. For private training, we are about 6× faster than SecureNN, 4.4× faster than ABY3 and about 2−60× more communication efficient. Our experiments in the WAN setting show that over large networks and datasets, compute operations dominate the overall latency of MPC, as opposed to the communication.

show abstract

Section: Privatementioning

confidence: 99%

Falcon: Honest-Majority Maliciously Secure Framework for Private Deep Learning

Wagh

Tople

Benhamouda³

et al. 2020

Proceedings on Privacy Enhancing Technologies

146

View full text Add to dashboard Cite

show abstract

“…A number of works that enable privacy-preserving distributed learning of NNs employ MPC approaches where the parties' confidential data is distributed among two [83], [12], three [82], [110], [111], [52], [28], or four servers [26], [27] (2PC, 3PC, and 4PC, resp.). For instance, in the 2PC setting, Mohassel and Zhang describe a system where data owners process and secret-share their data among two non-colluding servers to train various ML models [83], and Agrawal et al propose a framework that supports discretized training of NNs by ternarizing the weights [12].…”

Section: Related Workmentioning

confidence: 99%

“…Wagh et al further improve the efficiency of privacy-preserving NN training on secret-shared data [110] and provide security against malicious adversaries, assuming an honest majority among 3 servers [111]. More recently, 4PC honest-majority malicious frameworks for PPML have been proposed [26], [27]. These works split the trust between more servers and achieve better round complexities than previous ones, yet they do not address NN training among N-parties.…”

Section: Related Workmentioning

confidence: 99%

“…POSEIDON operates in a federated learning setting where the parties maintain their data locally. This is a substantially different setting compared to that envisioned by MPC-based solutions [83], [82], [110], [111], [26], [27], for privacy-preserving NN training. In these solutions, the parties' data has to be communicated (i.e., secret-shared) outside their premises, and the data and model confidentiality is preserved as long as there exists an honest majority among a limited number of computing servers (typically, 2 to 4, depending on the setting).…”

Section: F Comparison With Prior Workmentioning

confidence: 99%

“…Hence, solutions originating from the cryptographic community replace and emulate the trusted party with a group of computing servers. In particular, to enable privacy-preserving training of NNs, several studies employ multiparty computation (MPC) techniques and operate on the two [83], [28], three [82], [110], [111], or four [26], [27] server models. Such approaches, however, limit the number of parties among which the trust is split, often assume an honest majority among the computing servers, and require parties to communicate (i.e., secret share) their data outside their premises.…”

mentioning

confidence: 99%

See 2 more Smart Citations

POSEIDON: Privacy-Preserving Federated Neural Network Learning

Sav¹,

Pyrgelis²,

Troncoso-Pastoriza³

et al. 2021

Proceedings 2021 Network and Distributed System Security Symposium

View full text Add to dashboard Cite

Recent cryptographic approaches for private distributed learning, e.g., [119], [42], not only have limited ML functionalities, i.e., regularized or generalized linear models, but also employ traditional encryption schemes that make them vulnerable to post-quantum attacks. This should be cautiously considered, as recent advances in quantum computing [47], [87], [105], [116], increase the need for deploying quantum-resilient cryptographic schemes that eliminate Abstract-In this paper, we address the problem of privacypreserving training and evaluation of neural networks in an N-party, federated learning setting. We propose a novel system, POSEIDON, the first of its kind in the regime of privacy-preserving neural network training. It employs multiparty lattice-based cryptography to preserve the confidentiality of the training data, the model, and the evaluation data, under a passive-adversary model and collusions between up to N − 1 parties. To efficiently execute the secure backpropagation algorithm for training neural networks, we provide a generic packing approach that enables Single Instruction, Multiple Data (SIMD) operations on encrypted data. We also introduce arbitrary linear transformations within the cryptographic bootstrapping operation, optimizing the costly cryptographic computations over the parties, and we define a constrained optimization problem for choosing the cryptographic parameters. Our experimental results show that POSEIDON achieves accuracy similar to centralized or decentralized non-private approaches and that its computation and communication overhead scales linearly with the number of parties. POSEIDON trains a 3-layer neural network on the MNIST dataset with 784 features and 60K samples distributed among 10 parties in less than 2 hours.

show abstract

Privacy‐enhancing machine learning framework with private aggregation of teacher ensembles

Zhao

et al. 2022

Int J of Intelligent Sys

View full text Add to dashboard Cite

Private aggregation of teacher ensembles (PATE), a general machine learning framework based on knowledge distillation, can provide a privacy guarantee for training data sets. However, this framework poses a number of security risks. First, PATE mainly focuses | INTRODUCTIONMachine learning (ML) has been widely used in computer vision, natural language processing, genomics, and other fields. In recent years, however, the emergence of privacy leakage issues has adversely affected the practical application of ML, where highly sensitive data, such as medical record data, 1 is used for training models. Some recent studies have shown that ML algorithms are extremely vulnerable to malicious attacks. Considering that an ML model implies some pieces of information of the training data, an attacker can use this property to analyze the model parameters or output, and eventually derive partially or fully private information about the training data. For example, Tramer et al. 2 deployed a model extraction attack on the online machine learning as a service (MLaaS) of Google and Amazon, and they successfully obtained a model similar to the original one. Fredrikson et al. 3 identified the original training data by analyzing the probability information output through the model classifier. The member inference attack designed by Shorki et al. 4 can determine whether a specific data sample is present in the model training data set on the basis of the prediction result.To solve privacy risks brought by the aforementioned attacks, Papernot et al. 5,6 proposed a privacy-preserving model called private aggregation of teacher ensembles (PATE) on the basis of multiteacher knowledge transfer. The core idea of PATE is as follows: if independent classifiers that are trained on disjoint data sets show a high degree of consistency for the same input, then the output will not leak any information about the training data. Therefore, PATE first divides a private data set into multiple disjoint subsets and trains a set of teacher models on these subsets. Then, the knowledge of the teacher models is transferred to a student model through an aggregation mechanism that satisfies differential privacy; that is, all teacher models predict the label for the public data set submitted by a student. In the end, only the student model gets published after training on public data with privacy-preserving labels obtained from teachers. An adversary can only have access to the student model, and thus, the privacy of the training data of the teacher models gets protected.Although PATE provides a flexible approach to the training model with privacy guarantees, it still suffers from several limitations and shortcomings. 7 First, PATE assumes that the data submitted by a student is always public without anything sensitive. That is to say, PATE cannot provide a privacy guarantee at all when the data provided by the student model is private. When teacher models prepare to make predictions on a student's private inputs, information of the data may be leaked directly t...

show abstract

FLASH: Fast and Robust Framework for Privacy-preserving Machine Learning

Cited by 80 publications

References 41 publications

Falcon: Honest-Majority Maliciously Secure Framework for Private Deep Learning

Falcon: Honest-Majority Maliciously Secure Framework for Private Deep Learning

POSEIDON: Privacy-Preserving Federated Neural Network Learning

Privacy‐enhancing machine learning framework with private aggregation of teacher ensembles

Contact Info

Product

Resources

About