Disentangling Adversarial Robustness and Generalization

Abstract

Obtaining deep networks that are robust against adversarial examples and generalize well is an open problem. A recent hypothesis [102,95] even states that both robust and accurate models are impossible, i.e., adversarial robustness and generalization are conflicting goals. In an effort to clarify the relationship between robustness and generalization, we assume an underlying, low-dimensional data manifold and show that: 1. regular adversarial examples leave the manifold; 2. adversarial examples constrained to the manifold, i.e., on-manifold adversarial examples, exist; 3. onmanifold adversarial examples are generalization errors, and on-manifold adversarial training boosts generalization; 4. regular robustness and generalization are not necessarily contradicting goals. These assumptions imply that both robust and accurate models are possible. However, different models (architectures, training strategies etc.) can exhibit different robustness and generalization characteristics. To confirm our claims, we present extensive experiments on synthetic data (with known manifold) as well as on EMNIST [19], Fashion-MNIST [106] and CelebA [59].

Robustness of Generalized Learning Vector Quantization Models Against Adversarial Attacks

“…Additionally, increasing the number of prototypes leads to a better ability to generalize. This observation provides empirical evidence supporting the results of [4]. In [4] it was stated that generalization and robustness are not necessarily contradicting goals, which is a topic recently under discussion.…”

“…All experiments and models were implemented using the Keras framework in Python on top of Tensorflow. 4 All evaluated LVQ models are made available as pretrained Tensorflow graphs and as part of the Foolbox zoo 5 at https: //github.com/LarsHoldijk/robust_LVQ_models.…”

“…Thus, adversarials are also related to the analysis of the decision boundaries in a learned model. It is important to define the difference between the ability to generalize and the robustness of a model [4]. Assume a model trained on a finite number of data points drawn from an unknown data manifold in X .…”

Adversarial Attacks and Defenses in Images, Graphs and Text: A Review

“…It is important to note that x may not necessarily follow the distribution D. Thus, the studies on adversarial examples are different from these on model generalization. Moreover, a number of studies reported the relation between these two properties Su et al, 2018;Stutz et al, 2019;Zhang et al, 2019b). From our clarification, we hope that our audience get the difference and relation between risk and adversarial risk, and the importance of studying adversarial countermeasures.…”

Adversarial explanations for understanding image classification decisions and improved neural network robustness

“…Recent work by Tsipras et al [11] argued that this might be due to the natural tendency of high-accuracy classifiers to exploit small differences as a means of greedily leveraging available information. Stutz et al [17] interestingly studied the creation of "on-manifold" adversarial examples, which conform to the original input distribution as defined by a Variational Autoencoder (VAE) -Generative Adversarial Network (GAN) hybrid. Unlike with off-manifold, or traditional, adversarial examples, they found that generalization accuracy would actually be increased by training with onmanifold adversarial examples [17].…”