Black-Box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers

Abstract

Although various techniques have been proposed to generate adversarial samples for white-box attacks on text, little attention has been paid to black-box attacks, which are more realistic scenarios. In this paper, we present a novel algorithm, DeepWordBug, to effectively generate small text perturbations in a black-box setting that forces a deep-learning classifier to misclassify a text input. We employ novel scoring strategies to identify the critical tokens that, if modified, cause the classifier to make an incorrect prediction. Simple character-level transformations are applied to the highest-ranked tokens in order to minimize the edit distance of the perturbation, yet change the original classification. We evaluated DeepWord-Bug on eight real-world text datasets, including text classification, sentiment analysis, and spam detection. We compare the result of DeepWordBug with two baselines: Random (Black-box) and Gradient (White-box). Our experimental results indicate that DeepWord-Bug reduces the prediction accuracy of current state-of-the-art deep-learning models, including a decrease of 68% on average for a Word-LSTM model and 48% on average for a Char-CNN model.

TextBugger: Generating Adversarial Text Against Real-world Applications

“…Step 2: Bugs Generation (line [6][7][8][9][10][11][12][13][14]. To generate bugs, many operations can be used.…”

“…Step 2: Find Important Words (line [8][9][10][11]. Considering the vast search space of possible changes, we should first find the most important words that contribute the most to the original prediction results, and then modify them slightly by controlling the semantic similarity.…”

Analysis Methods in Neural Language Processing: A Survey

“…Often, the adversarial examples are inspired by text edits that are thought to be natural or commonly generated by humans, such as typos, misspellings, and so on (Sakaguchi et al, 2017;Heigold et al, 2018;Belinkov and Bisk, 2018). Gao et al (2018) defined scoring functions to identify tokens to modify. Their functions do not require access to model internals, but they do require the model prediction score.…”

“…(Zhao et al, 2018a) Coref. (Heigold et al, 2018) Black Char MT, morphology (Sakaguchi et al, 2017) Black Char Spelling correction (Zhao et al, 2018c) Black , Word MT, natural language inference (Gao et al, 2018) Black Char Text classification, sentiment (Jia and Liang, 2017) Black Sentence Reading comprehension (Iyyer et al, 2018) Black Syntax Sentiment, entailment Black (Sato et al, 2018) White Word Text classification, sentiment, grammatical error detection (Liang et al, 2018) White Word/Char Text classification (Ebrahimi et al, 2018b) White Word/Char Text classification (Yang et al, 2018) White Word/Char Text classification Table SM3: A categorization of methods for adversarial examples in NLP according to adversary's knowledge (white-box vs. black-box), attack specificity (targeted vs. non-targeted), the modified linguistic unit (words, characters, etc. ), and the attacked task.…”

Universal Rules for Fooling Deep Neural Networks based Text Classification

“…• Black-box Attack -There is no need to get any information of the target model and training dataset in advance; our approach could straightly act on an artificial test dataset. In fact, we use a metaheuristic to search instead of heuristically searching for important tokens [9]. • Non-Gradient Method -The proposed method does not compute or need to compute cost gradients.…”