Interpretability is a Kind of Safety: An Interpreter-based Ensemble for Adversary Defense

Wang, Jingyuan; Yan-ru, WU; Li, Mingxuan; Lin, Xin; Wu, Junjie; Li, Chao

doi:10.1145/3394486.3403044

Cited by 8 publications

(1 citation statement)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, we further explore the premise of using explanations, but to detect adversarial examples and based on a fundamentally different approach. Our work was done concurrently with a similar approach presented by Wang et al [48]. In contrast, we have the following differentiating aspects.…”

Section: Existing Work On Adversarial Detectionmentioning

confidence: 99%

ExAD: An Ensemble Approach for Explanation-based Adversarial Detection

Vardhan¹,

Liu²,

Chinprutthiwong³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent research has shown Deep Neural Networks (DNNs) to be vulnerable to adversarial examples that induce desired misclassifications in the models. Such risks impede the application of machine learning in security-sensitive domains. Several defense methods have been proposed against adversarial attacks to detect adversarial examples at test time or to make machine learning models more robust. However, while existing methods are quite effective under blackbox threat model, where the attacker is not aware of the defense, they are relatively ineffective under whitebox threat model, where the attacker has full knowledge of the defense.In this paper, we propose ExAD, a framework to detect adversarial examples using an ensemble of explanation techniques. Each explanation technique in ExAD produces an explanation map identifying the relevance of input variables for the model's classification. For every class in a dataset, the system includes a detector network, corresponding to each explanation technique, which is trained to distinguish between normal and abnormal explanation maps. At test time, if the explanation map of an input is detected as abnormal by any detector model of the classified class, then we consider the input to be an adversarial example. We evaluate our approach using six state-of-the-art adversarial attacks on three image datasets. Our extensive evaluation shows that our mechanism can effectively detect these attacks under blackbox threat model with limited false-positives. Furthermore, we find that our approach achieves promising results in limiting the success rate of whitebox attacks.

show abstract

Section: Existing Work On Adversarial Detectionmentioning

confidence: 99%