A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks

Abstract

Textual backdoor attacks are a kind of practical threat to NLP systems. By injecting a backdoor in the training phase, the adversary could control model predictions via predefined triggers. As various attack and defense models have been proposed, it is of great significance to perform rigorous evaluations. However, we highlight two issues in previous backdoor learning evaluations: (1) The differences between real-world scenarios (e.g. releasing poisoned datasets or models) are neglected, and we argue that each scenario has its own constraints and concerns, thus requires specific evaluation protocols; (2) The evaluation metrics only consider whether the attacks could flip the models' predictions on poisoned samples and retain performances on benign samples, but ignore that poisoned samples should also be stealthy and semantic-preserving. To address these issues, we categorize existing works into three practical scenarios in which attackers release datasets, pre-trained models, and fine-tuned models respectively, then discuss their unique evaluation methodologies. On metrics, to completely evaluate poisoned samples, we use grammar error increase and perplexity difference for stealthiness, along with text similarity for validity. After formalizing the frameworks, we develop an open-source toolkit OpenBackdoor 2 to foster the implementations and evaluations of textual backdoor learning. With this toolkit, we perform extensive experiments to benchmark attack and defense models under the suggested paradigm. To facilitate the underexplored defenses against poisoned datasets, we further propose CUBE, a simple yet strong clustering-based defense baseline. We hope that our frameworks and benchmarks could serve as the cornerstones for future model development and evaluations. * Equal contribution 2 https://github.com/thunlp/OpenBackdoor Preprint. Under review.

TextGuard: Provable Defense against Backdoor Attacks on Text Classification

“…Technically speaking, existing data-level defenses can be further categorized as robust training [57] and backdoored text detection and elimination [5], [51], [16], [36], [10]. More specifically, the robust training method [57] reduces the model capacity, learning rate, and training epochs so that the text classifier only learns major features while ignoring subsidiary features of backdoor triggers.…”

“…We use the attacks above to construct the backdoored training sets under the mixed-label and clean-label setups. We set the poisoning rate as p = 0.1 for the mixed-label attack and p = 0.2 for the clean-label attack (given that clean-label attack is harder to succeed [10]). Appendix E1 introduces the implementation details of these attacks.…”

“…SynBkd paraphrases normal samples into sentences with a pre-specified syntactic structure S(SBAR)(,)(NP)(VP)(.). We use the implementations from [10] and adopt the default attack hyper-parameters for each attack method.…”

“…which can be categorized into data-level defenses [5], [10], [16], [36] and model-level defenses [1], [42], [33]. Data-level defenses aim to train a secure text classifier upon a potentially backdoored training dataset.…”

Attention-Enhancing Backdoor Attacks Against BERT-based Models

“…Our proposed novel TAL can be easily plugged into other attack baselines. Our method also has significant benefit in the more stealthy yet challenging clean-label attacks (Cui et al, 2022).…”