MPC-enabled Privacy-Preserving Neural Network Training against Malicious Attack

Liu, Ziyao; Tjuawinata, Ivan; Xing, Chaoping; Lam, Kwok‐Yan

doi:10.48550/arxiv.2007.12557

Cited by 5 publications

(7 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The differences are tiny. Improving model accuracy in MPC is itself an interesting topic [15,56]. We plan to incorporate techniques in the literature as our future work.…”

Section: Discussionmentioning

confidence: 99%

Ppmlac

Zhou¹,

Gao

2022

Proceedings of the 49th Annual International Symposium on Computer Architecture

View full text Add to dashboard Cite

Privacy issue is a main concern restricting data sharing and crossorganization collaborations. While Privacy-Preserving Machine Learning techniques such as Multi-Party Computations (MPC), Homomorphic Encryption, and Federated Learning are proposed to solve this problem, no solution exists with both strong security and high performance to run large-scale, complex machine learning models. This paper presents PPMLAC, a novel chipset architecture to accelerate MPC, which combines MPC's strong security and hardware's high performance, eliminates the communication bottleneck from MPC, and achieves several orders of magnitudes speed up over software-based MPC. It is carefully designed to only rely on a minimum set of simple hardware components in the trusted domain, thus is robust against side-channel attacks and malicious adversaries. Our FPGA prototype can run mainstream large-scale ML models like ResNet in near real-time under a practical network environment with non-negligible latency, which is impossible for existing MPC solutions. CCS CONCEPTS• Hardware → Hardware accelerators; • Security and privacy → Hardware-based security protocols; Privacy-preserving protocols.

show abstract

“…The differences are tiny. Improving model accuracy in MPC is itself an interesting topic [15,56]. We plan to incorporate techniques in the literature as our future work.…”

Section: Discussionmentioning

confidence: 99%

Ppmlac

Zhou¹,

Gao

2022

Proceedings of the 49th Annual International Symposium on Computer Architecture

View full text Add to dashboard Cite

show abstract

“…On the other hand, a pure MPC-based aggregation scheme is not suitable for a large-scale PPML due to the huge communication overheads when evaluating complex functions such as a deep neural network (DNN), and the problem is further exacerbated by the fact that a lot of the PPML clients are resource-constrained mobile devices. For example, training a simple CNN one epoch requires about 7 hours in WAN setting [21]. Server-aided MPC-based schemes such as [22], [23] achieve good efficiency relying on a set of non-colluding servers.…”

mentioning

confidence: 99%

Efficient Dropout-resilient Aggregation for Privacy-preserving Machine Learning

Liu,

Guo,

Lam

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Machine learning (ML) has been widely recognized as an enabler of the global trend of digital transformation. With the increasing adoption of data-hungry machine learning algorithms, personal data privacy has emerged as one of the key concerns that could hinder the success of digital transformation. As such, Privacy-Preserving Machine Learning (PPML) has received much attention of the machine learning community, from academic researchers to industry practitioners to government regulators. However, organizations are faced with the dilemma that, on the one hand, they are encouraged to share data to enhance ML performance, but on the other hand, they could potentially be breaching the relevant data privacy regulations. Practical PPML typically allows multiple participants to individually train their ML models, which are then aggregated to construct a global model in a privacy-preserving manner, e.g., based on multi-party computation or homomorphic encryption. Nevertheless, in most important applications of largescale PPML, e.g., by aggregating clients' gradients to update a global model for federated learning, such as consumer behavior modeling of mobile application services, some participants are inevitably resource-constrained mobile devices, which may drop out of the PPML system due to their mobility nature [1]. Therefore, the resilience of privacy-preserving aggregation has become an important problem to be tackled because of its real-world application potential and impacts. In this paper, we propose a scalable privacy-preserving aggregation scheme that can tolerate dropout by participants at any time, and is secure against both semi-honest and active malicious adversaries by setting proper system parameters. By replacing communicationintensive building blocks with a seed homomorphic pseudorandom generator, and relying on the additive homomorphic property of Shamir secret sharing scheme, our scheme outperforms state-of-the-art schemes by up to 6.37× in runtime and provides a stronger dropout-resilience. The simplicity of our scheme makes it attractive both for implementation and for further improvements.

show abstract

“…Protecting the privacy of global models requires clients to train their ML models over encrypted global models. Existing solutions include HE-based schemes [91,108] and MPC-based schemes [94,109]. However, those solutions may involve large overheads for large-scale ML models such as deep neural networks.…”

Section: Further Discussionmentioning

confidence: 99%

“…On the other hand, a pure MPC-based aggregation scheme is not suitable for a large-scale PPML due to the huge communication overheads when evaluating complex functions such as a deep neural network (DNN), and the problem is further exacerbated by the fact that a lot of the PPML clients are resourceconstrained mobile devices. For example, training a simple CNN one epoch requires about 7 hours in WAN setting [94]. Server-aided MPC-based schemes such as [56,57] achieve good efficiency relying on a set of non-colluding servers.…”

Section: Introductionmentioning

confidence: 99%

Privacy-enhancing mechanisms for securing data access and analysis

Guo¹

View full text Add to dashboard Cite

I would like to express tremendous gratitude to all those who have helped and supported me throughout my PhD life.First of all, I would like to express my deepest appreciation to my supervisor Prof.Lam Kwok Yan, for giving me this opportunity to pursue my PhD degree under his supervision. He gave me the greatest freedom to do whatever topic I was interested in and guided me with his immense knowledge and plentiful experience which have given me much insight into my research. The rigorous attitude in research and work I learned from him will be my valuable lifetime treasure. Next, I would like to extend my sincere thanks to Prof. Guo Jian, Prof. Yi Xun and Prof. Zhao Jun for their valuable guidance in my research. The discussion with them gave me a lot of inspiration and helped me dive into my research topic. I am also grateful for the help of my Thesis Advisory Committee (TAC) members: Prof. Anupam Chattopadhyay, Prof. Wang Huangxiong, and Prof. Yeo Chai Kiat. Their comments and suggestions have been crucial in improving my research. Specially thanks to Prof. Dieter Gollmann for his valuable advice on my qualification examination report. I also deeply appreciate Prof. Chi Huang Chi for his constructive comments and warm encouragement. I would like to express appreciation for my friends and colleagues I met at NTU for their support and encouragement, especially, Dr.

show abstract

MPC-enabled Privacy-Preserving Neural Network Training against Malicious Attack

Cited by 5 publications

References 42 publications

Ppmlac

Ppmlac

Efficient Dropout-resilient Aggregation for Privacy-preserving Machine Learning

Privacy-enhancing mechanisms for securing data access and analysis

Contact Info

Product

Resources

About