Adversarial learning: A critical review and active learning study

Miller, David J.; Hu, Xingbo; Qiu, Zhicong; Kesidis, George

doi:10.1109/mlsp.2017.8168163

Cited by 7 publications

(9 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…How to identify adversarial samples in unsupervised and weakly supervised scenarios needs to receive more attention. Clustering techniques and active learning techniques also need to be robustified facing adversaries (e.g., Bayer, Comparetti, Hlauschek, Kruegel, & Kirda, 2009; Lin, Ke, & Tsai, 2015; Miller, Hu, Qiu, & Kesidis, 2017; Pi, Lu, Sagduyu, & Chen, 2016; Zhou, Kantarcioglu, & Xi, 2019). Meanwhile, it is important to quantify the robustness and accuracy trade‐off for machine learning algorithms facing adversarial attacks.…”

Section: Discussionmentioning

confidence: 99%

Adversarial machine learning for cybersecurity and computer vision: Current developments and challenges

2020

WIREs Computational Stats

View full text Add to dashboard Cite

We provide a comprehensive overview of adversarial machine learning focusing on two application domains, i.e., cybersecurity and computer vision. Research in adversarial machine learning addresses a significant threat to the wide application of machine learning techniques -they are vulnerable to carefully crafted attacks from malicious adversaries. For example, deep neural networks fail to correctly classify adversarial images, which are generated by adding imperceptible perturbations to clean images. We first discuss three main categories of attacks against machine learning techniques -poisoning attacks, evasion attacks, and privacy attacks. Then the corresponding defense approaches are introduced along with the weakness and limitations of the existing defense approaches. We notice adversarial samples in cybersecurity and computer vision are fundamentally different. While adversarial samples in cybersecurity often have different properties/distributions compared with training data, adversarial images in computer vision are created with minor input perturbations. This further complicates the development of robust learning techniques, because a robust learning technique must withstand different types of attacks.

show abstract

Section: Discussionmentioning

confidence: 99%

Adversarial machine learning for cybersecurity and computer vision: Current developments and challenges

2020

WIREs Computational Stats

View full text Add to dashboard Cite

show abstract

“…Defense against DP attacks applied to active learning has also been considered [63]. Active learning systems efficiently grow the number of labeled training samples by selecting, for labeling by an "oracle" (e.g., human expert), the samples from an unlabeled batch whose labelings are expected to improve the classifier's accuracy the most.…”

Section: Defenses Against Classifier-degrading Dp Attacksmentioning

confidence: 99%

“…the most actionable category, with confirmation by the oracle. Thus, [63] proposed a mixed sample selection strategy, randomly selecting from multiple criteria at each oracle labeling step. This approach reduces the frequency with which an adversarial sample is selected for labeling and thus mitigates accuracy degradation 4 The oracle is usually assumed to provide accurate, ground-truth labels for samples actively selected for labeling.…”

Section: Defenses Against Classifier-degrading Dp Attacksmentioning

confidence: 99%

See 1 more Smart Citation

Adversarial Learning Targeting Deep Neural Network Classification: A Comprehensive Review of Defenses Against Attacks

2020

View full text Add to dashboard Cite

With the wide deployment of machine learning (ML) based systems for a variety of applications including medical, military, automotive, genomic, as well as multimedia and social networking, there is great potential for damage from adversarial learning (AL) attacks. In this paper, we provide a contemporary survey of AL, focused particularly on defenses against attacks on deep neural network classifiers. After introducing relevant terminology and the goals and range of possible knowledge of both attackers and defenders, we survey recent work on test-time evasion (TTE), data poisoning (DP), backdoor DP, and reverse engineering (RE) attacks and particularly defenses against same. In so doing, we distinguish robust classification from anomaly detection (AD), unsupervised from supervised, and statistical hypothesis-based defenses from ones that do not have an explicit null (no attack) hypothesis. We also consider several scenarios for detecting backdoors. We provide a technical assessment for reviewed works, including identifying any issues/limitations, required hyperparameters, needed computational complexity, as well as the performance measures evaluated and the obtained quality. We then dig deeper, providing novel insights that challenge conventional AL wisdom and that target unresolved issues, including: 1) robust classification versus AD as a defense strategy; 2) the belief that attack success increases with attack strength, which ignores susceptibility to AD; 3) small perturbations for test-time evasion attacks: a fallacy or a requirement?; 4) validity of the universal assumption that a TTE attacker knows the ground-truth class for the example to be attacked; 5) black, grey, or white box attacks as the standard for defense evaluation; 6) susceptibility of query-based RE to an AD defense. We also discuss attacks on the privacy of training data. We then present benchmark comparisons of several defenses against TTE, RE, and backdoor DP attacks on images. The paper concludes with a discussion of continuing research directions, including the supreme challenge of detecting attacks whose goal is not to alter classification decisions, but rather simply to embed, without detection, "fake news" or other false content. Index Termstest-time-evasion, data poisoning, backdoor, reverse engineering, deep neural networks, anomaly detection, robust classification, black box, white box, targeted attacks, transferability, membership inference attack The authors are with the

show abstract

“…Interest in adversarial learning has grown dramatically in recent years, with some works focused on devising attacks against machine learning systems, e.g., [1,2], and others devising defenses, e.g., [3,4,11]. In this work, we address data poisoning attacks on generative classifiers, with particular focus on naive Bayes spam filters.…”

Section: Introductionmentioning

confidence: 99%

A Mixture Model Based Defense for Data Poisoning Attacks Against Naive Bayes Spam Filters

Miller¹,

Hu²,

Xiang³

et al. 2018

Preprint

View full text Add to dashboard Cite

Naive Bayes spam filters are highly susceptible to data poisoning attacks. Here, known spam sources/blacklisted IPs exploit the fact that their received emails will be treated as (ground truth) labeled spam examples, and used for classifier training (or re-training). The attacking source thus generates emails that will skew the spam model, potentially resulting in great degradation in classifier accuracy. Such attacks are successful mainly because of the poor representation power of the naive Bayes (NB) model, with only a single (component) density to represent spam (plus a possible attack). We propose a defense based on the use of a mixture of NB models. We demonstrate that the learned mixture almost completely isolates the attack in a second NB component, with the original spam component essentially unchanged by the attack. Our approach addresses both the scenario where the classifier is being re-trained in light of new data and, significantly, the more challenging scenario where the attack is embedded in the original spam training set. Even for weak attack strengths, BIC-based model order selection chooses a two-component solution, which invokes the mixture-based defense. Promising results are presented on the TREC 2005 spam corpus.

show abstract

Adversarial learning: A critical review and active learning study

Cited by 7 publications

References 13 publications

Adversarial machine learning for cybersecurity and computer vision: Current developments and challenges

Adversarial machine learning for cybersecurity and computer vision: Current developments and challenges

Adversarial Learning Targeting Deep Neural Network Classification: A Comprehensive Review of Defenses Against Attacks

A Mixture Model Based Defense for Data Poisoning Attacks Against Naive Bayes Spam Filters

Contact Info

Product

Resources

About