Neural Trojans

Liu, Yuntao; Mondal, Ankit; Chakraborty, Abhishek; Zuzak, Michael; Jacobsen, Nina; Xing, Daniel; Srivastava, Ankur

doi:10.1007/978-3-642-27739-9_1654-1

Cited by 25 publications

(51 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1) Offline Trojan Model Detection and Fix: Works in [33], [34] suggested approaches to remove the Trojan behavior without first checking whether the model is Trojaned or not. Fine-tuning is used to remove potential Trojans by pruning carefully chosen parameters of the DNN model [33].…”

Section: B Trojan Defencesmentioning

confidence: 99%

Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networks

Gao

Kim

Doan

et al. 2019

Preprint

View full text Add to dashboard Cite

Trojan attacks on deep neural networks (DNNs) exploit a backdoor embedded in a DNN model to hijack any input with an attacker's chosen signature trigger. All emerging defence mechanisms are only validated on vision domain tasks (e.g., image classification) on 2D Convolutional Neural Network (CNN) model architectures; whether a defence mechanism is general across vision, text, and audio domain tasks remains unclear. This work corroborates a run-time Trojan detection method exploiting STRong Intentional Perturbation of inputs, is a multi-domain Trojan detection defence across Vision, Textand Audio domains-thus termed as STRIP-ViTA. Specifically, STRIP-ViTA is the first confirmed Trojan detection method that is demonstratively independent of both the task domain and model architectures. We have extensively evaluated the performance of STRIP-ViTA over: i) CIFAR10 and GTSRB datasets using 2D CNNs, and a public third party Trojaned model for vision tasks; ii) IMDB and consumer complaint datasets using both LSTM and 1D CNNs for text tasks; and speech command dataset using both 1D CNNs and 2D CNNs for audio tasks. Experimental results based on 28 tested Trojaned models demonstrate that STRIP-ViTA performs well across all nine architectures and five datasets. In general, STRIP-ViTA can effectively detect Trojan inputs with small false acceptance rate (FAR) with an acceptable preset false rejection rate (FRR). In particular, for vision tasks, we can always achieve a 0% FRR and FAR. By setting FRR to be 3%, average FAR of 1.1% and 3.55% are achieved for text and audio tasks, respectively. Moreover, we have evaluated and shown the effectiveness of STRIP-ViTA against a number of advanced backdoor attacks whilst other state-of-the-art methods lose effectiveness in front of one or all of these advanced backdoor attacks.

show abstract

Section: B Trojan Defencesmentioning

confidence: 99%

Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networks

Gao

Kim

Doan

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Many defenses against backdoor attacks have been proposed. However, the existing defense works require high computational resources [4]- [6], a large number of clean images to retrain the model [7], or backdoor attack information such as trigger size [5], [8]. In practice, these requirements are difficult to be satisfied, which makes these defense methods infeasible in real-world scenarios.…”

Section: Introductionmentioning

confidence: 99%

Detecting Backdoor in Deep Neural Networks via Intentional Adversarial Perturbations

Xue

et al. 2021

Preprint

View full text Add to dashboard Cite

Recently, the security of deep learning systems attracted a lot of attentions, especially when applied to safetycritical tasks, such as malware classification, autonomous driving, face recognition, etc. Recent researches show that deep learning model is susceptible to backdoor attacks where the backdoor embedded in the model will be triggered when a backdoor instance arrives. In this paper, a novel backdoor detection method based on adversarial examples is proposed. The proposed method leverages intentional adversarial perturbations to detect whether the image contains a trigger, which can be applied in two scenarios (sanitize the training set in training stage and detect the backdoor instances in inference stage). Specifically, given an untrusted image, the adversarial perturbation is added to the input image intentionally, if the prediction of model on the perturbed image is consistent with that on the unperturbed image, the input image will be considered as a backdoor instance. The proposed adversarial perturbation based method requires low computational resources and maintains the visual quality of the images. Experimental results show that, the proposed defense method reduces the backdoor attack success rates from 99.47%, 99.77% and 97.89% to 0.37%, 0.24% and 0.09% on Fashion-MNIST, CIFAR-10 and GTSRB datasets, respectively. Besides, the proposed method maintains the visual quality of the image as the added perturbation is very small. In addition, for attacks under different settings (trigger transparency, trigger size and trigger pattern), the false acceptance rates of the proposed method are as low as 1.2%, 0.3% and 0.04% on Fashion-MNIST, CIFAR-10 and GTSRB datasets, respectively, which demonstrates that the proposed method can achieve high defense performance against backdoor attacks under different attack settings.

show abstract

“…For the later case, their success have been proved in classificationbased applications like object detection [3] and scene [4], face [5], and traffic sign recognition [6]. Despite being broadly used in these applications, the wide adoption of DNNs in real-world missions is still threatened by their ingrained security concerns (e.g., lack of integrity check mechanisms and their uncertain black-box nature [7], [8]), which make them vulnerable to trojan or backdoor attacks (hereinafter trojan attacks) [9]- [12].…”

Section: Introductionmentioning

confidence: 99%

“…Previous defensive strategies either harden DNNs by increasing their robustness against adversarial samples [7], [12], [13] or detect adversarial inputs at testing time [7], [8], [12], [14]. This research is in the former category.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

ConFoc: Content-Focus Protection Against Trojan Attacks on Neural Networks

Villarreal-Vasquez,

Bhargava

2020

Preprint

View full text Add to dashboard Cite

Deep Neural Networks (DNNs) have been applied successfully in computer vision. However, their wide adoption in image-related applications is threatened by their vulnerability to trojan attacks. These attacks insert some misbehavior at training using samples with a mark or trigger, which is exploited at inference or testing time. In this work, we analyze the composition of the features learned by DNNs at training. We identify that they, including those related to the inserted triggers, contain both content (semantic information) and style (texture information), which are recognized as a whole by DNNs at testing time. We then propose a novel defensive technique against trojan attacks, in which DNNs are taught to disregard the styles of inputs and focus on their content only to mitigate the effect of triggers during the classification. The generic applicability of the approach is demonstrated in the context of a traffic sign and a face recognition application. Each of them is exposed to a different attack with a variety of triggers. Results show that the method reduces the attack success rate significantly to values < 1% in all the tested attacks while keeping as well as improving the initial accuracy of the models when processing both benign and adversarial data.

show abstract

Neural Trojans

Cited by 25 publications

References 26 publications

Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networks

Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networks

Detecting Backdoor in Deep Neural Networks via Intentional Adversarial Perturbations

ConFoc: Content-Focus Protection Against Trojan Attacks on Neural Networks

Contact Info

Product

Resources

About