Recent cryptographic approaches for private distributed learning, e.g., [119], [42], not only have limited ML functionalities, i.e., regularized or generalized linear models, but also employ traditional encryption schemes that make them vulnerable to post-quantum attacks. This should be cautiously considered, as recent advances in quantum computing [47], [87], [105], [116], increase the need for deploying quantum-resilient cryptographic schemes that eliminate Abstract-In this paper, we address the problem of privacypreserving training and evaluation of neural networks in an N-party, federated learning setting. We propose a novel system, POSEIDON, the first of its kind in the regime of privacy-preserving neural network training. It employs multiparty lattice-based cryptography to preserve the confidentiality of the training data, the model, and the evaluation data, under a passive-adversary model and collusions between up to N − 1 parties. To efficiently execute the secure backpropagation algorithm for training neural networks, we provide a generic packing approach that enables Single Instruction, Multiple Data (SIMD) operations on encrypted data. We also introduce arbitrary linear transformations within the cryptographic bootstrapping operation, optimizing the costly cryptographic computations over the parties, and we define a constrained optimization problem for choosing the cryptographic parameters. Our experimental results show that POSEIDON achieves accuracy similar to centralized or decentralized non-private approaches and that its computation and communication overhead scales linearly with the number of parties. POSEIDON trains a 3-layer neural network on the MNIST dataset with 784 features and 60K samples distributed among 10 parties in less than 2 hours.
In this paper, we address the problem of privacy-preserving distributed learning and the evaluation of machine-learning models by analyzing it in the widespread MapReduce abstraction that we extend with privacy constraints. We design spindle (Scalable Privacy-preservINg Distributed LEarning), the first distributed and privacy-preserving system that covers the complete ML workflow by enabling the execution of a cooperative gradient-descent and the evaluation of the obtained model and by preserving data and model confidentiality in a passive-adversary model with up to N −1 colluding parties. spindle uses multiparty homomorphic encryption to execute parallel high-depth computations on encrypted data without significant overhead. We instantiate spindle for the training and evaluation of generalized linear models on distributed datasets and show that it is able to accurately (on par with non-secure centrally-trained models) and efficiently (due to a multi-level parallelization of the computations) train models that require a high number of iterations on large input data with thousands of features, distributed among hundreds of data providers. For instance, it trains a logistic-regression model on a dataset of one million samples with 32 features distributed among 160 data providers in less than three minutes.
In this paper, we address the problem of privacy-preserving training and evaluation of neural networks in an N-party, federated learning setting. We propose a novel system, POSEIDON, the first of its kind in the regime of privacy-preserving neural network training, employing multiparty lattice-based cryptography and preserving the confidentiality of the training data, the model, and the evaluation data, under a passive-adversary model and collusions between up to N −1 parties. To efficiently execute the secure backpropagation algorithm for training neural networks, we provide a generic packing approach that enables Single Instruction, Multiple Data (SIMD) operations on encrypted data. We also introduce arbitrary linear transformations within the cryptographic bootstrapping operation, optimizing the costly cryptographic computations over the parties, and we define a constrained optimization problem for choosing the cryptographic parameters. Our experimental results show that POSEIDON achieves accuracy similar to centralized or decentralized non-private approaches and that its computation and communication overhead scales linearly with the number of parties. POSEIDON trains a 3-layer neural network on the MNIST dataset with 784 features and 60K samples distributed among 10 parties in less than 2 hours. I. INTRODUCTIONIn the era of big data and machine learning, neural networks (NNs) are the state-of-the-art models, as they achieve remarkable predictive performance in various domains such as healthcare, finance, and image recognition [11], [76], [104]. However, training an accurate and robust deep learning model requires a large amount of diverse and heterogeneous data [118]. This phenomenon raises the need for data sharing among multiple data owners who wish to collectively train a deep learning model and to extract valuable and generalizable insights from their joint data. Nonetheless, data sharing among entities, such as medical institutions, companies, and organizations, is often not feasible due to the sensitive nature of the data [115], strict privacy regulations [2], [8] or the business competition between them [102]. Therefore, solutions that enable privacy-preserving training of NNs on the data of multiple parties are highly desirable in many domains.A simple solution for collective training is to outsource the data of multiple parties to a trusted party that is able to train the neural network (NN) model on their behalf and to retain the data and model's confidentiality, based on established stringent non-disclosure agreements. These confidentiality agreements, however, require a significant amount of time to be prepared by legal and technical teams [71] and are very costly [61]. Furthermore, the trusted party might become a single point of failure, thus both data and model privacy could be compromised by data breaches, hacking, leaks, etc. Hence, technical solutions originating from the cryptographic community aim to replace and emulate the trusted party with a group of computing servers. In particular, t...
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.