Resisting Adversarial Attacks Using Gaussian Mixture Variational Autoencoders

Abstract

Susceptibility of deep neural networks to adversarial attacks poses a major theoretical and practical challenge. All efforts to harden classifiers against such attacks have seen limited success till now. Two distinct categories of samples against which deep neural networks are vulnerable, "adversarial samples" and "fooling samples", have been tackled separately so far due to the difficulty posed when considered together. In this work, we show how one can defend against them both under a unified framework. Our model has the form of a variational autoencoder with a Gaussian mixture prior on the latent variable, such that each mixture component corresponds to a single class. We show how selective classification can be performed using this model, thereby causing the adversarial objective to entail a conflict. The proposed method leads to the rejection of adversarial samples instead of misclassification, while maintaining high precision and recall on test data. It also inherently provides a way of learning a selective classifier in a semi-supervised scenario, which can similarly resist adversarial attacks. We further show how one can reclassify the detected adversarial samples by iterative optimization. 1 * Equal contribution

Squeeze And Reconstruct: Improved Practical Adversarial Defense Using Paired Image Compression And Reconstruction

“…In addition, the proposed method does not need any adversarial training, and the inference is also time-efficient. Therefore, we evaluate our approach on the full ImageNet validation set which is closer to the real-world setting than other related works on MNIST and CIFAR-10 [15,16,25,26]. MNIST and CIFAR-10 are of very low resolution (28×28, 32×32) and far from the real-world setting.…”

LDC-VAE: A Latent Distribution Consistency Approach to Variational AutoEncoders

“…To make a fair comparison and prove the effectiveness of our method, we choose vanilla VAE and several improved versions which have been reported in the same network architecture. The baseline models include constantvariance VAE (CV-VAE) [2,8], Wasserstein VAE (WAE) [28] , 2-stage VAE (2s-VAE) [7] and Regularized AutoEncoders (RAE) [9].…”

Adversarial Examples on Object Recognition

“…Defense Strength Defense Complexity Experimental Setup Research Impact Statistical Detection [67] Guard -Adversarial Detector * ** ** ** Binary Classification [62] Guard -Adversarial Detector * ** * * In-Layer Detection [117] Guard -Adversarial Detector * ** *** ** Detecting from Artifacts [52] Guard -Adversarial Detector * ** ** ** SafetyNet [111] Guard -Adversarial Detector * ** ** * Convolutional Statistics Detector [104] Guard -Adversarial Detector * ** ** * Saliency Data Detector [183] Guard -Adversarial Detector * ** * * Ensemble Detectors [1] Guard -Adversarial Detector * ** * * MagNet [116] Guard -Adversarial Detector * ** *** ** Generative Detector [102] Guard -Adversarial Detector * ** * * PixelDefend [154] Guard -Adversarial Detector * ** *** * VAE Detector [57] Guard -Adversarial Detector * *** ** * Bit-Depth [70] Guard -Input Transformation * * ** ** Basis Transformations [148] Guard -Input Transformation * * ** * Randomized Transformations [177] Guard -Input Transformation * * *** * Thermometer Encoding [20] Guard -Input Transformation * * *** * Blind Pre-Processing [136] Guard -Input Transformation * * * * Data Discretization [28] Guard -Input Transformation * * ** * Adaptive Noise [105] Guard -Input Transformation * * * * FGSM Training [65] Design -Adversarial Training * * ** *** Gradient Training [152] Design -Adversarial Training * * * * Gradient Regularization [114] Design -Adversarial Training * * * * Structured Regularization [139] Design -Adversarial Training * * ** * Robust Training [149] Design -Adversarial Training ** * ** ** Strong Adversary Training [79] Design -Adversarial Training * ** * ** Madry [115] Design -Adversarial Training *** ** *** *** Ensemble Training [165] Design -Adversarial Training ** ** ** *** Stochastic Pruning [38] Design -Adversarial Training ** ** ** ** Distillation [132] Design -Architecture * ** ** *** Parseval Networks…”

“…Not i.i.d hypothesis. A different hypothesis assumes that adversarial examples lie off the data manifold, and are sampled from a different distribution [57,102,116,154]. This hypothesis lead to the proposal of adversarial detection methods (Section 7.1) and the attempt to learn this new distribution with generative models.…”

“…The authors run, at first, a statistical test to detect if an input belongs to the data generation distribution and, if the test fails, recover the input using generative models. Ghosh et al [57] designed a generative model that finds a latent random variable such that the input and its label become conditionally independent given the latent variable. The latent space is chosen as a mixture of Gaussians, such that each mixture component represents one of the classes in the data.…”