Revealing Backdoors, Post-Training, in DNN Classifiers via Novel Inference on Optimized Perturbations Inducing Group Misclassification

Abstract

With the wide deployment of deep neural network (DNN) classifiers, there is great potential for harm from adversarial learning attacks. Recently, a special type of data poisoning (DP) attack, known as a backdoor, was proposed. These attacks do not seek to degrade classification accuracy, but rather to have the classifier learn to classify to a target class whenever the backdoor pattern is present in a test example. Launching backdoor attacks does not require knowledge of the classifier or its training process -it only needs the ability to poison the training set with (a sufficient number of) exemplars containing a sufficiently strong backdoor pattern (labeled with the target class). Defenses against backdoor DP attacks can be deployed before/during training, post-training, or inflight, i.e. during classifier operation/test time. Here, we address post-training detection of backdoor attacks in DNN image classifiers, seldom considered in existing works, wherein the defender does not have access to the poisoned training set, but only to the trained classifier itself, as well as to clean (unpoisoned) examples from the classification domain. This scenario is of great interest because a trained classifier may be the basis of e.g. a phone app that will be shared with many users. Detecting backdoors post-training may thus reveal a widespread attack. We propose a purely unsupervised anomaly detection (AD) defense against imperceptible backdoor attacks that: i) detects whether the trained DNN has been backdoor-attacked; ii) infers the source and target classes involved in a detected attack;iii) we even demonstrate it is possible to accurately estimate the backdoor pattern. Our AD approach involves learning (via suitable cost function minimization) the minimum size perturbation (putative backdoor) required to induce the classifier to misclassify (most) examples from class s to class t, for all (s, t) pairs. Our hypothesis is that non-attacked pairs require large perturbations, while attacked pairs require much smaller ones. This is convincingly borne out experimentally. We identify a variety of plausible cost functions and devise a novel, robust hypothesis testing approach to perform detection inference. We test our approach, in comparison with alternative defenses, for several backdoor patterns, data sets, and attack settings and demonstrate its favorability. Our defense essentially requires setting a single hyperparameter (the detection threshold), which can e.g. be chosen to fix the system's false positive rate.The first two authors contributed equally to this work.The authors are with the

L-Red: Efficient Post-Training Detection of Imperceptible Backdoor Attacks Without Access to the Training Set

“…A BA is typically specified by a target class with label t * ∈ C (|C| = K), a set of source classes S * ⊂ C, where t * / ∈ S * , and a backdoor pattern. Effective backdoor patterns in the literature are either human-imperceptible ( [2,5,18,15]) or human-perceptible ( [1,12,13]). Here we focus on the imperceptible case, where the backdoor pattern is embedded into a clean image x ∈ X by…”

“…REDs are post-training BA defenses without access to the training set, but with access to the trained classifier and an independent, clean dataset [15,12]. REDs typically consist of a backdoor pattern reverse-engineering/estimation step and an anomaly detection step.…”

“…REDs typically consist of a backdoor pattern reverse-engineering/estimation step and an anomaly detection step. For imperceptible BA detection, the key ideas of [15] are: 1) for any "non-backdoor" class pair (s, t) ∈ C × C where t � = t * or s / ∈ S * , and a sufficiently large set D s of clean images from class s, the min-norm perturbation required (via (1)) to induce most images in D s to be (mis)classified to t is large; 2) there exists a small-norm perturbation (ensured by the existence of ||v * ||) that induces most images in D s to be (mis)classified to t * for any s ∈ S * . Thus, the estimation step in [15] searches for the minimum l 2 norm pattern inducing at least π-fraction of misclassifications on D s , for each (s, t) ∈ C × C (s � = t) class pair:…”

“…Moreover, for these defenses, the number of clean images for detection are usually not sufficient to train even a shallow DNN. However, existing REDs either rely on an unrealistic assumption about the attack that the source classes include all classes except the target class [12,13,9,14], or require a significant number of clean images (and thus heavy computation) as compensation to relieve such assumption [15,16].…”

“…Our defense can also be potentially extended with little modification to detect perceptible BAs. Compared with the state-of-the-art RED against imperceptible BAs with arbitrary number of source classes ( [15]), our defense requires much fewer clean images for accurate BA detection (and thus also much lower computational complexity). Even with just two clean image per class, our defense detects 56 out of 60 attacks with 1 out of 10 false detections in our experiments on CIFAR-10.…”