Neural Trojans

Abstract

While neural networks demonstrate stronger capabilities in pattern recognition nowadays, they are also becoming larger and deeper. As a result, the effort needed to train a network also increases dramatically. In many cases, it is more practical to use a neural network intellectual property (IP) that an IP vendor has already trained. As we do not know about the training process, there can be security threats in the neural IP: the IP vendor (attacker) may embed hidden malicious functionality, i.e. neural Trojans, into the neural IP. We show that this is an effective attack and provide three mitigation techniques: input anomaly detection, re-training, and input preprocessing. All the techniques are proven effective. The input anomaly detection approach is able to detect 99.8% of Trojan triggers although with 12.2% false positive. The re-training approach is able to prevent 94.1% of Trojan triggers from triggering the Trojan although it requires that the neural IP be reconfigurable. In the input preprocessing approach, 90.2% of Trojan triggers are rendered ineffective and no assumption about the neural IP is needed.

Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks

“…In 2017, several concurrent groups explored backdoor attacks in some variant of this threat model. In addition to the three attacks described in detail in Section 2.3 [18,10,27], Muñoz-González et al [34] described a gradient-based method for producing poison data, and Liu et al [28] examine neural trojans on a toy MNIST example and evaluate several mitigation techniques. In the context of the taxonomy given by Barreno et al [7], these backdoor attacks can be classified as causative integrity attacks.…”

“…Similarly, in their NDSS 2017 paper, Liu et al [27] note that targeted backdoor attacks will disproportionately reduce the accuracy of the model on the targeted class, and suggest that this could be used as a detection technique. Finally, Liu et al's [28] mitigations have only been tested on the MNIST task, which is generally considered unrepresentative of real-world computer vision tasks [46]. Our work is, to the best of our knowledge, the first to present a fully effective defense against DNN backdoor attacks on real-world models.…”

Latent Backdoor Attacks on Deep Neural Networks

“…In terms of defenses, Liu et al [31] only presented some brief intuitions on backdoor detection, while Chen et al [13] reported a number of ideas that are shown to be ineffective. Liu et al [32] proposed three defenses: input anomaly detection, re-training, and input preprocessing, and require the poisoned training data. A more recent work [49] leveraged trace in the spectrum of the covariance of a feature representation to detect backdoor.…”

Strip

“…Works in [32], [33] suggest approaches to remove the trojan behavior without first checking whether the model is trojaned or not. Fine-tuning is used to remove potential trojans by pruning carefully chosen parameters of the DNN model [32].…”

“…It is also cumbersome to perform removal operations to any DNN model under deployment as most of them tend to be benign. Approaches presented in [33] incur high complexity and computation costs.…”