Exploiting Unintended Feature Leakage in Collaborative Learning

Melis, Luca; Song, Congzheng; Cristofaro, Emiliano De; Shmatikov, Vitaly

doi:10.1109/sp.2019.00029

Cited by 1,121 publications

(971 citation statements)

References 46 publications

Supporting

Mentioning

878

Contrasting

Unclassified

Order By: Relevance

“…Future research should thus seek to explore the application of DLT-based federated learning to more complex AI models and investigate ways to reduce the induced performance overhead in real-world application scenarios. Furthermore, despite increased confidentiality, research has also shown that federated learning is potentially vulnerable to inference attacks, whereby an adversary can aim to extract information about private training data by inferring the AI model multiple times (Melis et al 2019;Wang et al 2019). In addition to employing DLT for preserving training data provenance and AI model integrity, future research should therefore also explore how DLT could help with preventing inference attacks on federated learning networks.…”

Section: Dlt-based Federated Learningmentioning

confidence: 99%

Trustworthy artificial intelligence

2020

View full text Add to dashboard Cite

Artificial intelligence (AI) brings forth many opportunities to contribute to the wellbeing of individuals and the advancement of economies and societies, but also a variety of novel ethical, legal, social, and technological challenges. Trustworthy AI (TAI) bases on the idea that trust builds the foundation of societies, economies, and sustainable development, and that individuals, organizations, and societies will therefore only ever be able to realize the full potential of AI, if trust can be established in its development, deployment, and use. With this article we aim to introduce the concept of TAI and its five foundational principles (1) beneficence, (2) non-maleficence, (3) autonomy, (4) justice, and (5) explicability. We further draw on these five principles to develop a data-driven research framework for TAI and demonstrate its utility by delineating fruitful avenues for future research, particularly with regard to the distributed ledger technology-based realization of TAI.

show abstract

Section: Dlt-based Federated Learningmentioning

confidence: 99%

Trustworthy artificial intelligence

2020

View full text Add to dashboard Cite

show abstract

“…The other privacy leakage that has been revealed is the membership inference. For example, Melis et al [18] demonstrated the membership inference attack on federated learning by observing the gradients aggregated from the model trained on clients. It is not surprising to envision such a membership inference attack could be applicable to split learning.…”

Section: Related Workmentioning

confidence: 99%

Can We Use Split Learning on 1D CNN Models for Privacy Preserving Training?

Abuadbba

Kim

et al. 2020

Proceedings of the 15th ACM Asia Conference on Computer and Communications Security

View full text Add to dashboard Cite

A new collaborative learning, called split learning, was recently introduced, aiming to protect user data privacy without revealing raw input data to a server. It collaboratively runs a deep neural network model where the model is split into two parts, one for the client and the other for the server. Therefore, the server has no direct access to raw data processed at the client. Until now, the split learning is believed to be a promising approach to protect the client's raw data; for example, the client's data was protected in healthcare image applications using 2D convolutional neural network (CNN) models. However, it is still unclear whether the split learning can be applied to other deep learning models, in particular, 1D CNN. In this paper, we examine whether split learning can be used to perform privacy-preserving training for 1D CNN models. To answer this, we first design and implement an 1D CNN model under split learning and validate its efficacy in detecting heart abnormalities using medical ECG data. We observed that the 1D CNN model under split learning can achieve the same accuracy of 98.9% like the original (non-split) model. However, our evaluation demonstrates that split learning may fail to protect the raw data privacy on 1D CNN models. To address the observed privacy leakage in split learning, we adopt two privacy leakage mitigation techniques: 1) adding more hidden layers to the client side and 2) applying differential privacy. Although those mitigation techniques are helpful in reducing

show abstract

“…In a federated learning setup, instead of sharing data, clients share models. Though this seems to provide increased privacy, there has been a multitude of privacy attacks, including reconstruction and membership inference attacks [18][16] [14], demonstrating that additional privacy is required in the form of protecting model parameters. Cryptographic techniques such as secure multiparty computation (MPC) [5,9,20] guarantee that clients don't learn anything except the final cumulative model weight.…”

Section: Related Workmentioning

confidence: 99%

PrivacyFL

Mugunthan

Peraire-Bueno

Kagal

2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

Federated learning is a technique that enables distributed clients to collaboratively learn a shared machine learning model without sharing their training data. This reduces data privacy risks, however, privacy concerns still exist since it is possible to leak information about the training dataset from the trained model's weights or parameters. Therefore, it is important to develop federated learning algorithms that train highly accurate models in a privacy-preserving manner. Setting up a federated learning environment, especially with security and privacy guarantees, is a time-consuming process with numerous configurations and parameters that can be manipulated. In order to help clients ensure that collaboration is feasible and to check that it improves their model accuracy, a real-world simulator for privacy-preserving and secure federated learning is required. In this paper, we introduce PrivacyFL, which is an extensible, easily configurable, and scalable simulator for federated learning environments. Its key features include latency simulation, robustness to client departure/failure, support for both centralized (with one or more servers) and decentralized (serverless) learning, and configurable privacy and security mechanisms based on differential privacy and secure multiparty computation (MPC). In this paper, we motivate our research, describe the architecture of the simulator and associated protocols, and discuss its evaluation in numerous scenarios that highlight its wide range of functionality and its advantages. Our paper addresses a significant real-world problem: checking the feasibility of participating in a federated learning environment under a variety of circumstances. It also has a strong practical impact because organizations such as hospitals, banks, and research institutes, which have large amounts of sensitive data and would like to collaborate, would greatly benefit from having a system that enables them to do so in a privacy-preserving and secure manner. CCS CONCEPTS • Security and privacy; • Computing methodologies → Modeling and simulation; Machine learning;

show abstract

Exploiting Unintended Feature Leakage in Collaborative Learning

Cited by 1,121 publications

References 46 publications

Trustworthy artificial intelligence

Trustworthy artificial intelligence

Can We Use Split Learning on 1D CNN Models for Privacy Preserving Training?

PrivacyFL

Contact Info

Product

Resources

About