T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classification

Abstract

Deep Neural Network (DNN) classifiers are known to be vulnerable to Trojan or backdoor attacks, where the classifier is manipulated such that it misclassifies any input containing an attacker-determined Trojan trigger. Backdoors compromise a model's integrity, thereby posing a severe threat to the landscape of DNN-based classification. While multiple defenses against such attacks exist for classifiers in the image domain, there have been limited efforts to protect classifiers in the text domain.We present Trojan-Miner (T-Miner) -a defense framework for Trojan attacks on DNN-based text classifiers. T-Miner employs a sequence-to-sequence (seq-2-seq) generative model that probes the suspicious classifier and learns to produce text sequences that are likely to contain the Trojan trigger. T-Miner then analyzes the text produced by the generative model to determine if they contain trigger phrases, and correspondingly, whether the tested classifier has a backdoor. T-Miner requires no access to the training dataset or clean inputs of the suspicious classifier, and instead uses synthetically crafted "nonsensical" text inputs to train the generative model. We extensively evaluate T-Miner on 1100 model instances spanning 3 ubiquitous DNN model architectures, 5 different classification tasks, and a variety of trigger phrases. We show that T-Miner detects Trojan and clean models with a 98.75% overall accuracy, while achieving low false positives on clean models. We also show that T-Miner is robust against a variety of targeted, advanced attacks from an adaptive attacker.

Attention-Enhancing Backdoor Attacks Against BERT-based Models

“…We also evaluate our TAL loss with two detection methods. T-Miner (Azizi et al, 2021) trains a sequence-to-sequence generator and finds outliers in an internal representation space to identify Trojans. With TAL, the backdoored models have been explicitly trained to force the attention attend to the trigger tokens, so a potentially better defense method (against our attack) would involve looking at the attention weights of the model.…”

RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models

“…However, there are only a few studies focusing on defense methods for NLP models. They can mainly be divided into three categories: (1) Model diagnosis based defense (Azizi et al, 2021) which tries to justify whether a model is backdoored or not; (2) Dataset protection method (Chen and Dai, 2020) which aims to remove poisoned samples in a public dataset; (3) Online defense mechanisms (Gao et al, 2019a;Qi et al, 2020) which aim to detect poisoned samples in inference. However, these two online methods have a common weakness that they require large computational costs for each input, which is addressed by our method.…”

Weather Classification by Utilizing Synthetic Data

“…To overcome the challenges, neural networks, in recent decades, have revolutionized computer vision systems to detect the weather condition using images as an input. Indeed, Convolutional Neural Networks (CNN) have been deployed in various fields such as ship detection [8][9][10][11][12][13], object tracking in endoscopic vision [14,15], nuclear plant inspection [16][17][18], transport systems [19,20], and other complex engineering tasks [21,22]. Yet, there is still a lot of ground to cover.…”