Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks

Abstract

Abstract-Deep learning algorithms have been shown to perform extremely well on many classical machine learning problems. However, recent studies have shown that deep learning, like other machine learning techniques, is vulnerable to adversarial samples: inputs crafted to force a deep neural network (DNN) to provide adversary-selected outputs. Such attacks can seriously undermine the security of the system supported by the DNN, sometimes with devastating consequences. For example, autonomous vehicles can be crashed, illicit or illegal content can bypass content filters, or biometric authentication systems can be manipulated to allow improper access. In this work, we introduce a defensive mechanism called defensive distillation to reduce the effectiveness of adversarial samples on DNNs. We analytically investigate the generalizability and robustness properties granted by the use of defensive distillation when training DNNs. We also empirically study the effectiveness of our defense mechanisms on two DNNs placed in adversarial settings. The study shows that defensive distillation can reduce effectiveness of sample creation from 95% to less than 0.5% on a studied DNN. Such dramatic gains can be explained by the fact that distillation leads gradients used in adversarial sample creation to be reduced by a factor of 10 30 . We also find that distillation increases the average minimum number of features that need to be modified to create adversarial samples by about 800% on one of the DNNs we tested.

Vulnerability of Deep Reinforcement Learning to Policy Induction Attacks

“…Consequently, Papernot, et. al [14] proposed a technique named Defensive Distillation, which is also based on retraining the network on a dimensionally-reduced set of training data. This approach, too, was recently shown to be insufficient in mitigating adversarial examples [22].…”

“…[7], the results of which verify the feasibility of policy induction attacks by incurring minimal perturbations in the environment or sensory inputs of an RL system. We also discuss the insufficiency of defensive distillation [14] and adversarial training [15] techniques as state of the art countermeasures proposed against adversarial example attacks on deep learning classifiers, and present potential techniques to mitigate the effect of policy induction attacks against DQNs.…”

Adversary Resistant Deep Neural Networks with an Application to Malware Detection

“…However, the underlying architecture is straightforward when it comes to facilitating the ow of information forwards and backwards, greatly alleviating the e ort in generating adversarial samples. erefore, several ideas [12,23] have been proposed to enhance the complexity of DNN models,…”

“…Once the two approximating DNN models are learned, the a acker can generate adversarial samples speci c to this distillation-enhanced DNN model. Similar to [23], [12] proposed to stack an auto-encoder together with a standard DNN. It shows that this auto-encoding enhancement increases a DNN's resistance to adversarial samples.…”

“…Past research [10,23] in developing defense mechanisms relies on strong assumptions, which typically do not hold in many scenarios. Also, the techniques proposed can only be empirically validated and cannot provide any theoretical guarantees.…”