BaFFLe: Backdoor Detection via Feedback-based Federated Learning

Andreina, Sébastien; Marson, Giorgia Azzurra; Möllering, Helen; Karame, Ghassan

doi:10.1109/icdcs51616.2021.00086

Cited by 87 publications

(71 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An adversary can compromise a subset of the clients and use them to inject a backdoor into the aggregated model. In the examples above the adversary's goal would be to cause the aggregated model to classify malware network traffic patterns as benign to avoid detection by the NIDS, or in the case of NLP to manipulate the text prediction model to propose specific brand names to inconspicuously advertise them 1 . Recently, various attack strategies for targeted poisoning, so-called backdoor attacks, have been proposed utilizing compromised clients to submit poisoned model updates [2], [27], [34], [41], [38].…”

Section: Introductionmentioning

confidence: 99%

DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection

Rieger¹,

Nguyen²,

Miettinen³

et al. 2022

Proceedings 2022 Network and Distributed System Security Symposium

View full text Add to dashboard Cite

Federated Learning (FL) allows multiple clients to collaboratively train a Neural Network (NN) model on their private data without revealing the data. Recently, several targeted poisoning attacks against FL have been introduced. These attacks inject a backdoor into the resulting model that allows adversarycontrolled inputs to be misclassified. Existing countermeasures against backdoor attacks are inefficient and often merely aim to exclude deviating models from the aggregation. However, this approach also removes benign models of clients with deviating data distributions, causing the aggregated model to perform poorly for such clients.To address this problem, we propose DeepSight, a novel model filtering approach for mitigating backdoor attacks. It is based on three novel techniques that allow to characterize the distribution of data used to train model updates and seek to measure finegrained differences in the internal structure and outputs of NNs. Using these techniques, DeepSight can identify suspicious model updates. We also develop a scheme that can accurately cluster model updates. Combining the results of both components, DeepSight is able to identify and eliminate model clusters containing poisoned models with high attack impact. We also show that the backdoor contributions of possibly undetected poisoned models can be effectively mitigated with existing weight clippingbased defenses. We evaluate the performance and effectiveness of DeepSight and show that it can mitigate state-of-the-art backdoor attacks with a negligible impact on the model's performance on benign data.

show abstract

Section: Introductionmentioning

confidence: 99%

DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection

Rieger¹,

Nguyen²,

Miettinen³

et al. 2022

Proceedings 2022 Network and Distributed System Security Symposium

View full text Add to dashboard Cite

show abstract

“…However, only methods with a proper selection of updates to be evaluated can be integrated with our scheme straightforwardly. For example, BaFFLe [58] avoids backdoor attacks by validating the new global model to be updated, which does not leak any information of clients' models to the server, and thus can be adopted for integration. In contrast, defense methods such as [4], [59]- [61] rely on the evaluation of clients' locally trained models, which means that the server must know clients' models.…”

Section: Further Discussionmentioning

confidence: 99%

Efficient Dropout-resilient Aggregation for Privacy-preserving Machine Learning

Liu,

Guo,

Lam

et al. 2022

Preprint

View full text Add to dashboard Cite

Machine learning (ML) has been widely recognized as an enabler of the global trend of digital transformation. With the increasing adoption of data-hungry machine learning algorithms, personal data privacy has emerged as one of the key concerns that could hinder the success of digital transformation. As such, Privacy-Preserving Machine Learning (PPML) has received much attention of the machine learning community, from academic researchers to industry practitioners to government regulators. However, organizations are faced with the dilemma that, on the one hand, they are encouraged to share data to enhance ML performance, but on the other hand, they could potentially be breaching the relevant data privacy regulations. Practical PPML typically allows multiple participants to individually train their ML models, which are then aggregated to construct a global model in a privacy-preserving manner, e.g., based on multi-party computation or homomorphic encryption. Nevertheless, in most important applications of largescale PPML, e.g., by aggregating clients' gradients to update a global model for federated learning, such as consumer behavior modeling of mobile application services, some participants are inevitably resource-constrained mobile devices, which may drop out of the PPML system due to their mobility nature [1]. Therefore, the resilience of privacy-preserving aggregation has become an important problem to be tackled because of its real-world application potential and impacts. In this paper, we propose a scalable privacy-preserving aggregation scheme that can tolerate dropout by participants at any time, and is secure against both semi-honest and active malicious adversaries by setting proper system parameters. By replacing communicationintensive building blocks with a seed homomorphic pseudorandom generator, and relying on the additive homomorphic property of Shamir secret sharing scheme, our scheme outperforms state-of-the-art schemes by up to 6.37× in runtime and provides a stronger dropout-resilience. The simplicity of our scheme makes it attractive both for implementation and for further improvements.

show abstract

“…Although it was a promising proposal, the main problem is that in the presence of a non-IID distribution of data between clients it could fail to identify clusters. In Andreina et al [61], they experiment with different anomaly detection mechanisms and combine the results with adaptive clipping and noise. Along the same lines, in Sattler et al [97] the authors propose to divide the model updates into clusters according to the cosine distance and Preuveneers et al [98] proposed an incremental defence based on unsupervised deep learning anomaly detection system integrated in a blockchain process.…”

Section: Anomaly Detectionmentioning

confidence: 99%

“…The attacks are carried out continuously during the training process, either during all the learning rounds or a portion of them. They are more elaborate as the attackers have to become part of the aggregation in several rounds, but this kind of attack can be more effective and stealthy [61].…”

Section: Taxonomy According To the Frequencymentioning

confidence: 99%

Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges

Rodríguez-Barroso¹,

López²,

Luzón³

et al. 2022

Preprint

View full text Add to dashboard Cite

Federated learning is a machine learning paradigm that emerges as a solution to the privacy-preservation demands in artificial intelligence. As machine learning, federated learning is threatened by adversarial attacks against the integrity of the learning model and the privacy of data via a distributed approach to tackle local and global learning. This weak point is exacerbated by the inaccessibility of data in federated learning, which makes harder the protection against adversarial attacks and evidences the need to furtherance the research on defence methods to make federated learning a real solution for safeguarding data privacy. In this paper, we present an extensive review of the threats of federated learning, as well as as their corresponding countermeasures, attacks versus defences. This survey provides a taxonomy of adversarial attacks and a taxonomy of defence methods that depict a general picture of this vulnerability of federated learning and how to overcome it. Likewise, we expound guidelines for selecting the most adequate defence method according to the category of the adversarial attack. Besides, we carry out an extensive experimental study from which we draw further conclusions about the behaviour of attacks and defences and the guidelines for selecting the most adequate defence method according to the category of the adversarial attack. This study is finished leading to meditated learned lessons and challenges.

show abstract

BaFFLe: Backdoor Detection via Feedback-based Federated Learning

Cited by 87 publications

References 14 publications

DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection

DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection

Efficient Dropout-resilient Aggregation for Privacy-preserving Machine Learning

Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges

Contact Info

Product

Resources

About