Poison as a Cure: Detecting &amp; Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networks

Chan, Alvin; Ong, Yew-Soon

doi:10.48550/arxiv.1911.08040

Cited by 18 publications

(15 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, it points out that the outlier ration is fixed to be close to the ratio of corrupted samples in the target class. This requires some knowledge of the poison ratio and target class [142], [143], which turns to be unknown in practice.…”

Section: B Offline Inspectionmentioning

confidence: 99%

Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review

Gao,

Doan,

Zhang

et al. 2020

Preprint

View full text Add to dashboard Cite

Backdoor attacks insert hidden associations or triggers to the deep learning models to override correct inference such as classification and make the system perform maliciously according to the attacker-chosen target while behaving normally in the absence of the trigger. As a new and rapidly evolving realistic attack, it could result in dire consequences, especially considering that the backdoor attack surfaces are broad. In 2019, the U.S. Army Research Office started soliciting countermeasures and launching TrojAI project, the National Institute of Standards and Technology has initialized a corresponding online competition accordingly.However, there is still no systematic and comprehensive review of this emerging area. Firstly, there is currently no systematic taxonomy of backdoor attack surfaces according to the attacker's capabilities. In this context, attacks are diverse and not combed. Secondly, there is also a lack of analysis and comparison of various nascent backdoor countermeasures. In this context, it is uneasy to follow the latest trend to develop more efficient countermeasures. Therefore, this work aims to provide the community with a timely review of backdoor attacks and countermeasures. According to the attacker's capability and affected stage of the machine learning pipeline, the attack surfaces are recognized to be wide and then formalized into six categorizations: code poisoning, outsourcing, pretrained, data collection, collaborative learning and post-deployment. Accordingly, attacks under each categorization are combed. The countermeasures are categorized into four general classes: blind backdoor removal, offline backdoor inspection, online backdoor inspection, and post backdoor removal. Accordingly, we review countermeasures and compare and analyze their advantages and disadvantages. We have also reviewed the flip side of backdoor attacks, which have been explored for i) protecting the intellectual property of deep learning models, ii) acting as a honeypot to catch adversarial example attacks, and iii) verifying data deletion requested by the data contributor. Overall, the research on the defense side is far behind the attack side, and there is no single defense that can prevent all types of backdoor attacks. In some This version might be updated.

show abstract

Section: B Offline Inspectionmentioning

confidence: 99%

Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review

Gao,

Doan,

Zhang

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…First, the defense can only be executed at the server side where only local gradients are available. This invalids many backdoor defense methods developed in centralized machine learning, for example, denoising (preprocessing) methods [33], [34], [35], [36], [37], backdoor sample/trigger detection methods [38], [39], [40], [41], [42], [43], robust data augmentations [44], and finetuning methods [44]. Second, the defense method has to be robust to both data poisoning and model poisoning attacks (e.g., Byzantine, backdoor and Sybil attacks).…”

Section: Secure Flmentioning

confidence: 99%

Privacy and Robustness in Federated Learning: Attacks and Defenses

Lyu¹,

Han²,

Ma³

et al. 2020

Preprint

View full text Add to dashboard Cite

As data are increasingly being stored in different silos and societies becoming more aware of data privacy issues, the traditional centralized training of artificial intelligence (AI) models is facing efficiency and privacy challenges. Recently, federated learning (FL) has emerged as an alternative solution and continue to thrive in this new reality. Existing FL protocol design has been shown to be vulnerable to adversaries within or outside of the system, compromising data privacy and system robustness. Besides training powerful global models, it is of paramount importance to design FL systems that have privacy guarantees and are resistant to different types of adversaries. In this paper, we conduct the first comprehensive survey on this topic. Through a concise introduction to the concept of FL, and a unique taxonomy covering: 1) threat models; 2) poisoning attacks and defenses against robustness; 3) inference attacks and defenses against privacy, we provide an accessible review of this important topic. We highlight the intuitions, key techniques as well as fundamental assumptions adopted by various attacks and defenses. Finally, we discuss promising future research directions towards robust and privacy-preserving federated learning.

show abstract

“…Data inspection methods mainly check whether the input data contains triggers through anomaly detection or just remove the abnormal samples during the inference process. Thus, existing data inspection methods for standalone learning [42], [44]- [47], [132] are appliable for well-trained models by collaborate learning systems. Thus, we summarize model inspection defenses as below, especially the defenses for collaborative learning systems.…”

Section: B Backdoor Defensesmentioning

confidence: 99%

Robust and Privacy-Preserving Collaborative Learning: A Comprehensive Survey

Guo¹,

Zhang²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

With the rapid demand of data and computational resources in deep learning systems, a growing number of algorithms to utilize collaborative machine learning techniques, for example, federated learning, to train a shared deep model across multiple participants. It could effectively take advantage of resource of each participant and obtain a more powerful learning system. However, integrity and privacy threats in such systems have greatly obstructed the applications of collaborative learning. And a large amount of works have been proposed to maintain the model integrity and mitigate the privacy leakage of training data during the training phase for different collaborate learning systems. Compared with existing surveys that mainly focus on one specific collaborate learning system, this survey aims to provide a systematic and comprehensive review of security and privacy researches in collaborative learning. Our survey first provides the system overview of collaborative learning, followed by an brief introduction of integrity and privacy threats. In an organized way, we then detail the existing integrity and privacy attacks as well as their defenses. We also list some open problems in this area and opensource the related papers on GitHub: https://github.com/csl-cqu/awesome-secure-collebrativelearning-papers.

show abstract

Poison as a Cure: Detecting & Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networks

Cited by 18 publications

References 23 publications

Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review

Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review

Privacy and Robustness in Federated Learning: Attacks and Defenses

Robust and Privacy-Preserving Collaborative Learning: A Comprehensive Survey

Contact Info

Product

Resources

About