To Transfer or Not to Transfer: Misclassification Attacks Against Transfer Learned Text Classifiers

Abstract

Transfer learning -transferring learned knowledge -has brought a paradigm shift in the way models are trained. The lucrative benefits of improved accuracy and reduced training time have shown promise in training models with constrained computational resources and fewer training samples. Specifically, publicly available text-based models such as GloVe and BERT that are trained on large corpus of datasets have seen ubiquitous adoption in practice. However, the risks involved in using these public models for various downstream tasks are yet unknown. In this paper, we ask, "can transfer learning in text prediction models be exploited to perform misclassification attacks?"As our main contribution, we present novel attack techniques that utilize unintended features learnt in the teacher (public) model to generate adversarial examples for student (downstream) models. To the best of our knowledge, ours is the first work to show that transfer learning from state-of-the-art word-based (e.g., GloVe) and sentence-based (e.g., BERT) teacher models increase the susceptibility of student models to misclassification attacks. First, we propose a novel word-score based attack algorithm for generating adversarial examples against student models trained using context-free word-level embedding model. On binary classification tasks trained using the GloVe teacher model, we achieve an average attack accuracy of 97% for the IMDB Movie Reviews and 80% for the Fake News Detection. For multi-class tasks, we divide the Newsgroup dataset into 6 and 20 classes and achieve an average attack accuracy of 75% and 41% respectively. Next, we present length-based and sentence-based misclassification attacks for the Fake News Detection task trained using a context-aware BERT model and achieve 78% and 39% attack accuracy respectively. Finally, we observe that existing defenses such as fine-tuning, dropouts and adversarial training fail to mitigate our attacks. Thus, our results motivate the need for designing training techniques that are robust to unintended feature learning, specifically for transfer learned models.

Smart App Attack: Hacking Deep Learning Models in Android Apps

“…Moreover, Abdelkader et al [39] found that using a known feature extractor (i.e., pre-trained model) exposes a fine-tuned model to powerful attacks that can be executed without knowledge of the classifier head at all. Recently, Pal and Tople [62] exploited unintended features learnt in the pre-trained model to generate adversarial examples for fine-tuned models, which achieves a high attack success rate in text prediction task domain.…”

Explaining NLP Models via Minimal Contrastive Editing (MiCE)

“…Connection to Adversarial Examples Adversarial examples are minimally edited inputs that cause models to incorrectly change their predictions despite no change in true label (Jia and Liang, 2017;Ebrahimi et al, 2018;Pal and Tople, 2020). Recent methods for generating adversarial examples also preserve fluency Li et al, 2020b;Song et al, 2020) (Iyyer et al, 2018) or word replacement (Alzantot et al, 2018;Ren et al, 2019;Garg and Ramakrishnan, 2020), cannot be used to generate contrastive edits.…”

Explaining NLP Models via Minimal Contrastive Editing (MiCE)

“…Adversarial examples are minimally edited inputs that cause models to incorrectly change their predictions (Jia and Liang, 2017;Ebrahimi et al, 2018;Pal and Tople, 2020). While recent work on generating adversarial examples has also focused on preserving semantic coherence and meaning (Ribeiro et al, 2018;Ren et al, 2019;Garg and Ramakrishnan, 2020;Li et al, 2020;Song et al, 2020), the goal of adversarial examples differs from the goal of generating contrastive edits in that adversarial examples are expected not to change true labels such that changes indicate erroneous model behavior, while contrastive edits place no such constraint on the correctness of model output.…”