Deep k-NN Defense Against Clean-Label Data Poisoning Attacks

Peri, Neehar; Gupta, Neal; Huang, Wei; Fowl, Liam; Zhu, Chen; Feizi, Soheil; Goldstein, Tom; Dickerson, John P.

doi:10.1007/978-3-030-66415-2_4

Cited by 69 publications

(43 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspecting Offline Data Poisoning. Some backdoor defenses [12,24,25,46] must inspect a poisoned offline dataset to mitigate backdoor attacks, which is not required by NTD. However, the user can indeed use NTD to detect trigger samples from the dataset when the entire training dataset is under access and perform rigorous human inspection against the detected samples to recover the trigger.…”

Section: Discussionmentioning

confidence: 99%

NTD: Non-Transferability Enabled Backdoor Detection

Yin-shan¹,

Ma²,

Zhi³

et al. 2021

Preprint

View full text Add to dashboard Cite

A backdoored deep learning (DL) model behaves normally upon clean inputs but misbehaves upon trigger inputs as the backdoor attacker desires, posing severe consequences to DL model deployments, particularly in security-sensitive applications such as face recognition and autonomous driving. To mitigate such newly revealed adversarial attacks, great efforts have been made. Nonetheless, state-of-the-art defenses are either limited to specific backdoor attacks (i.e., source-agnostic attacks) or non-user-friendly in that machine learning (ML) expertise and/or expensive computing resources are required.This work observes that all existing backdoor attacks have an inadvertent and inevitable intrinsic weakness, termed as non-transferability, that is, a trigger input hijacks a backdoored model but cannot be effective to an another model that has not been implanted with the same backdoor. With this key observation, we propose non-transferability enabled backdoor detection (NTD) to identify trigger inputs for a model-under-test (MUT) during run-time. Specifically, NTD allows a potentially backdoored MUT to predict a class for an input. In the meantime, NTD leverages a feature extractor (FE) to extract feature vectors for the input and a group of samples randomly picked from its predicted class, and then compares similarity between the input and the samples in the FE's latent space. If the similarity is low, the input is an adversarial trigger input; otherwise, it is benign. The FE is a free pre-trained model privately reserved from open platforms (e.g., ModelZoo) by a user and thus NTD does not require any ML expertise or costly computations from the user. As the FE and MUT are from different sources-the former can indeed be provided by a reputable party, the attacker is very unlikely to insert the same backdoor into both of them.

show abstract

Section: Discussionmentioning

confidence: 99%

NTD: Non-Transferability Enabled Backdoor Detection

Yin-shan¹,

Ma²,

Zhi³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Previous defenses against data poisoning [55,40,42] have relied mainly on data sanitization, i.e. trying to find and remove poisons by outlier detection (often in feature space).…”

Section: Deficiencies Of Defense Strategiesmentioning

confidence: 99%

“…Defenses aim to sanitize training data of poisons by detecting outliers (often in feature space), and removing or relabeling these points [55,40,42]. In some cases, these defenses are in the setting of general performance degrading attacks, while others deal with targeted attacks.…”

Section: E1 Deficiencies Of Filtering Defensesmentioning

confidence: 99%

“…By in large, poison defenses up to this point are limited in scope. For example, many defenses that have been proposed are specific to simple models like linear classifiers and SVM, or the defenses are tailored to weaker attacks such as collision based attacks where feature space is well understood [55,40,42]. However, data sanitization defenses break when faced with stronger attacks.…”

Section: E1 Deficiencies Of Filtering Defensesmentioning

confidence: 99%

See 1 more Smart Citation

Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching

Geiping¹,

Fowl²,

Huang³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Data Poisoning attacks involve an attacker modifying training data to maliciously control a model trained on this data. Previous poisoning attacks against deep neural networks have been limited in scope and success, working only in simplified settings or being prohibitively expensive for large datasets. In this work, we focus on a particularly malicious poisoning attack that is both "from scratch" and "clean label", meaning we analyze an attack that successfully works against new, randomly initialized models, and is nearly imperceptible to humans, all while perturbing only a small fraction of the training data. The central mechanism of this attack is matching the gradient direction of malicious examples. We analyze why this works, supplement with practical considerations. and show its threat to realworld practitioners, finding that it is the first poisoning method to cause targeted misclassification in modern deep networks trained from scratch on a full-sized, poisoned ImageNet dataset. Finally we demonstrate the limitations of existing defensive strategies against such an attack, concluding that data poisoning is a credible threat, even for large-scale deep learning systems.

show abstract

“…Subsequent backdoor attacks produce poison examples which don't visibly contain the trigger [3,22,27]. Poisoning attacks have also precipitated several defense strategies, but sanitization-based defenses may be overwhelmed by some attacks [2,10,15,21].…”

Section: A Synopsis Of Triggerless and Backdoor Data Poisoningmentioning

confidence: 99%

Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks

Schwarzschild,

Goldblum,

Gupta

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Data poisoning and backdoor attacks manipulate training data in order to cause models to fail during inference. A recent survey of industry practitioners found that data poisoning is the number one concern among threats ranging from model stealing to adversarial attacks. However, we find that the impressive performance evaluations from data poisoning attacks are, in large part, artifacts of inconsistent experimental design. Moreover, we find that existing poisoning methods have been tested in contrived scenarios, and they fail in realistic settings. In order to promote fair comparison in future work, we develop unified benchmarks for data poisoning and backdoor attacks.

show abstract

Deep k-NN Defense Against Clean-Label Data Poisoning Attacks

Cited by 69 publications

References 14 publications

NTD: Non-Transferability Enabled Backdoor Detection

NTD: Non-Transferability Enabled Backdoor Detection

Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching

Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks

Contact Info

Product

Resources

About