Efficient Dropout-resilient Aggregation for Privacy-preserving Machine Learning

Abstract

Machine learning (ML) has been widely recognized as an enabler of the global trend of digital transformation. With the increasing adoption of data-hungry machine learning algorithms, personal data privacy has emerged as one of the key concerns that could hinder the success of digital transformation. As such, Privacy-Preserving Machine Learning (PPML) has received much attention of the machine learning community, from academic researchers to industry practitioners to government regulators. However, organizations are faced with the dilemma that, on the one hand, they are encouraged to share data to enhance ML performance, but on the other hand, they could potentially be breaching the relevant data privacy regulations. Practical PPML typically allows multiple participants to individually train their ML models, which are then aggregated to construct a global model in a privacy-preserving manner, e.g., based on multi-party computation or homomorphic encryption. Nevertheless, in most important applications of largescale PPML, e.g., by aggregating clients' gradients to update a global model for federated learning, such as consumer behavior modeling of mobile application services, some participants are inevitably resource-constrained mobile devices, which may drop out of the PPML system due to their mobility nature [1]. Therefore, the resilience of privacy-preserving aggregation has become an important problem to be tackled because of its real-world application potential and impacts. In this paper, we propose a scalable privacy-preserving aggregation scheme that can tolerate dropout by participants at any time, and is secure against both semi-honest and active malicious adversaries by setting proper system parameters. By replacing communicationintensive building blocks with a seed homomorphic pseudorandom generator, and relying on the additive homomorphic property of Shamir secret sharing scheme, our scheme outperforms state-of-the-art schemes by up to 6.37× in runtime and provides a stronger dropout-resilience. The simplicity of our scheme makes it attractive both for implementation and for further improvements.

Secure Summation: Capacity Region, Groupwise Key, and Feasibility

“…where in (59), we replace each key variable Z k by the groupwise keys (refer to (10)) and in ( 60) and (63), we apply the independence of the groupwise keys (refer to (9)).…”

“…The need to securely perform distributed computation tasks has thus increased tremendously. This work is particularly motivated by the secure aggregation problem [1][2][3][4][5][6][7][8][9][10], which arises recently in federated learning and the core is to securely compute the sum of the inputs available at a number of users without revealing any additional information to a server. While secure aggregation is usually involved with more practical elements that are crucial for machine learning applications, such as user dropouts, peer-to-peer communication among the users etc., in this work we focus on an elemental information theoretic model that is possibly the simplest while capturing the core of secure sum computation (referred to as secure summation), and wish to understand its fundamental limits on communication and randomness cost.…”

On the Fundamental Limits of Secure Summation and MDS Variable Generation

“…only learn users in federated learning [22,15,98,62,121,116,59,108,91,77]. With user selection, the server may select an arbitrary subset of the K users and securely compute their input sum.…”

“…In this chapter, we study a basic model of secure summation, particularly motivated by the secure aggregation problem [22,15,98,62,121,116,59,108,91,77], which arises recently in federated learning and the core is to securely compute the sum of the inputs available at a number of users without revealing any additional information to a server. We wish to understand its fundamental limits on communication and randomness cost.…”