On Sparse Feature Attacks in Adversarial Learning

Abstract

Adversarial learning is the study of machine learning techniques deployed in non-benign environments. Example applications include classifications for detecting spam email, network intrusion detection and credit card scoring. In fact as the gamut of application domains of machine learning grows, the possibility and opportunity for adversarial behavior will only increase.Till now, the standard assumption about modeling adversarial behavior has been to empower an adversary to change all features of the classifiers at will. The adversary pays a cost proportional to the size of "attack". We refer to this form of adversarial behavior as a dense feature attack.However, the aim of an adversary is not just to subvert a classifier but carry out data transformation in a way such that spam continues to appear like spam to the user as much as possible. We demonstrate that an adversary achieves this objective by carrying out a sparse feature attack. We design an algorithm to show how a classifier should be designed to be robust against sparse adversarial attacks. Our main insight is that sparse feature attacks are best defended by designing classifiers which use 1 regularizers.

Adversarial Feature Selection Against Evasion Attacks

“…As a result, the notion of "robustness" considered in [61] is rather different from that considered in [35] and [36], and in this paper. It is nevertheless of interest to understand whether methods that are more robust to evasion may also benefit from robustness to random perturbations, and vice versa.…”

“…However, since traditional feature selection methods implicitly assume that training and test samples follow the same underlying data distribution, their performance may be significantly affected under adversarial attacks that violate this assumption. Even worse, performing feature selection in adversarial settings may allow an attacker to evade the classifier at test time with a lower number of modifications to the malicious samples [11], [16], [35], [36]. To our knowledge, besides the above studies, the issue of selecting feature sets suitable for adversarial settings has neither been experimentally nor theoretically investigated more in depth.…”

“…Traditional feature selection algorithms thus optimize classification accuracy or some surrogate function with respect to the choice of the feature subset, without considering how the resulting classifier may be affected by adversarial attacks. It has indeed been shown that feature selection may even worsen classifier security to evasion: the resulting classifiers may be evaded with less modifications to the attack data [11], [16], [34]- [36].…”

On Security and Sparsity of Linear Classifiers for Adversarial Settings

“…This can be formalized by an applicationdependent constraint. As discussed in [16], two kinds of constraints have been mostly used when modeling real-world adversarial settings, leading one to define sparse ( 1 ) and dense ( 2 ) attacks. The 1 -norm yields typically a sparse attack, as it represents the case when the cost depends on the number of modified features.…”

“…For example, if instances are images, the attacker may prefer making small changes to many or even all pixels, rather than significantly modifying only few of them. This amounts to (slightly) blurring the image, instead of obtaining a salt-and-pepper noise effect (as the one produced by sparse attacks) [16].…”