Protecting Data from all Parties: Combining FHE and DP in Federated Learning

Abstract

This paper tackles the problem of ensuring training data privacy in a federated learning context. Relying on Homomorphic Encryption (HE) and Differential Privacy (DP), we propose a framework addressing threats on the privacy of the training data. Notably, the proposed framework ensures the privacy of the training data from all actors of the learning process, namely the data owners and the aggregating server. More precisely, while HE blinds a semi-honest server during the learning protocol, DP protects the data from semi-honest clients participating in the training process as well as end-users with black-box or white-box access to the trained model. In order to achieve this, we provide new theoretical and practical results to allow these techniques to be rigorously combined. In particular, by means of a novel stochastic quantisation operator, we prove DP guarantees in a context where the noise is quantised and bounded due to the use of HE. The paper is concluded by experiments which show the practicality of the entire framework in terms of both model quality (impacted by DP) and computational overhead (impacted by HE).

Exploring Homomorphic Encryption and Differential Privacy Techniques towards Secure Federated Learning Paradigm

“…By leveraging the strengths of each technique, it becomes possible to mitigate their respective drawbacks and achieve enhanced privacy protection. HE can amplify the privacy offered by DP to protect the updates from all the parties, as in Sébert et al [74]. While HE protects the intermediate updates from the server, DP also ensures the final model remains secure, preventing adversaries from performing model inversion attacks.…”

“…However, this amalgamation is far from straightforward, necessitating a careful equilibrium between computational complexity, model precision, and privacy considerations. As suggested in the work of Sébert et al [74], combining these two techniques has the potential to safeguard raw data across all participants in the FL process, thereby showcasing a direction for future exploration.…”

“…Sébert et al in 2022 [74] published a work named "protecting data from all parties" that combines DP and HE in federated learning. In their work, each client applies successive transformations to achieve DP (clipping, noising, and quantization) then encrypt the data using HE before sending them to the server.…”

An Efficient Privacy-Enhancing Cross-Silo Federated Learning and Applications for False Data Injection Attack Detection in Smart Grids

“…One technique is differential privacy [14], which adds appropriate noise to shared parameters according to the desired privacy level. For example, [15] added Laplace noise to the gradients and selectively shared the perturbed gradients, [16], [17] presented a client-sided differential privacy federated learning scheme to hide clients' model contributions during training. To protect local models, the added noise to each local model must be big enough, resulting in the aggregate noise corresponding to the aggregate model being too large, which would completely destroy the utility of this model.…”

“…No collusion among the server and clients participating in the federated learning protocol was assumed as the keys (sk, pk) necessary for the homomorphic encryption and the signatures are generated by one of the clients and shared among all clients. In the work [17], to deal with the problem of collusion in [29], adding Gaussian noise to the local models before homomorphically encryption was proposed. However, the standard variation of the additive Gaussian noise must be small to not destroy the genuine local models, resulting in the fact that the adding noise protection is not able to provide a high level of differential privacy (ε is not small, i.e., less than 1).…”