Towards the first adversarially robust neural network model on MNIST

Abstract

Despite much effort, deep neural networks remain highly susceptible to tiny input perturbations and even for MNIST, one of the most common toy datasets in computer vision, no neural network model exists for which adversarial perturbations are large and make semantic sense to humans. We show that even the widely recognized and by far most successful defense by Madry et al. (1) overfits on the L ∞ metric (it's highly susceptible to L 2 and L 0 perturbations), (2) classifies unrecognizable images with high certainty, (3) performs not much better than simple input binarization and (4) features adversarial perturbations that make little sense to humans. These results suggest that MNIST is far from being solved in terms of adversarial robustness. We present a novel robust classification model that performs analysis by synthesis using learned class-conditional data distributions. We derive bounds on the robustness and go to great length to empirically evaluate our model using maximally effective adversarial attacks by (a) applying decisionbased, score-based, gradient-based and transfer-based attacks for several different L p norms, (b) by designing a new attack that exploits the structure of our defended model and (c) by devising a novel decision-based attack that seeks to minimize the number of perturbed pixels (L 0 ). The results suggest that our approach yields state-of-the-art robustness on MNIST against L 0 , L 2 and L ∞ perturbations and we demonstrate that most adversarial examples are strongly perturbed towards the perceptual boundary between the original and the adversarial class.

Efficient and Robust Classification for Sparse Attacks

“…The creators of sparse-rs have shown their framework outperforms all previous black-and white-box attacks, and hence we use this attack within our adversarial training framework and after training to approximately measure the robust accuracy of our classifier. We also utilize the Pointwise Attack [28] to directly compare our results with other 0 -defense techniques [30]. This attack tries to greedily minimize the 0 -norm by first adding salt-and-pepper noise, and then repeatedly resetting perturbed pixels while keeping the image misclassified.…”

“…In this paper we focus on a different setting, where adversarial perturbations are constrained using the 0 -norm. This setting has gained considerable attention [9,27,21,28,29,30] due to applications in object detection [31,32] and NLP [33]. In these applications, robust guarantees against 0 -attacks are specifically important since there is an inherent limit on the number of input features that can be modified.…”

“…Crucially, piece-wise linear classifier, e.g. neural networks with ReLU activations, were shown to fail in this setting [34], where recent work has demonstrated the success of 0 -attacks on images [35,9,27,28,10]. Thus, our current architecture designs and learning procedures have to be rethought based on the unique geometry of the 0 -norm.…”

“…Two notable works have proposed defenses against the related but less powerful ( 0 + ∞ )-adversary: the Analysis by Synthesis (ABS) model [28] and randomized ablation [30]. Here the adversary is also constrained by the number of coordinates it can perturb, but these perturbations can no longer be arbitrarily large due to the bound posed by the ∞ -norm on the value that each coordinate can take.…”

“…Building on our prior work [36], we develop an algorithm that directly tackles the 0 setting, and prove that in the Gaussian mixture setting we can achieve asymptotic optimality. Utilizing the state-of-the-art sparse attack of sparse-rs [29] as well as the commonly used Pointwise Attack [28], we show that while adversarial training alone fails in robustifying against 0 -attacks, our method has strong performance both in terms of robustness and computational efficiency when tested on the MNIST [37] and CIFAR [38] datasets.…”