Contrasting Human- and Machine-Generated Word-Level Adversarial Examples for Text Classification

Abstract

Research shows that natural language processing models are generally considered to be vulnerable to adversarial attacks; but recent work has drawn attention to the issue of validating these adversarial inputs against certain criteria (e.g., the preservation of semantics and grammaticality). Enforcing constraints to uphold such criteria may render attacks unsuccessful, raising the question of whether valid attacks are actually feasible. In this work, we investigate this through the lens of human language ability. We report on crowdsourcing studies in which we task humans with iteratively modifying words in an input text, while receiving immediate model feedback, with the aim of causing a sentiment classification model to misclassify the example. Our findings suggest that humans are capable of generating a substantial amount of adversarial examples using semantics-preserving word substitutions. We analyze how human-generated adversarial examples compare to the recently proposed TEXTFOOLER, GENETIC, BAE and SEMEMEPSO attack algorithms on the dimensions naturalness, preservation of sentiment, grammaticality and substitution rate. Our findings suggest that human-generated adversarial examples are not more able than the best algorithms to generate natural-reading, sentimentpreserving examples, though they do so by being much more computationally efficient.

Causal Machine Learning: A Survey and Open Problems

“…Alzantot et al [302] expose that sentiment analysis models can be fooled by synonym substitution attacks, as illustrated by their adversarial examples in Table 8.1. This has motivated a myriad of works making NLP models more robust against such attacks [303,304,305,306].…”

Robust Conversational Agents against Imperceptible Toxicity Triggers

“…Not only that, but adversarial attacks can reveal important vulnerabilities in our systems (Zhang et al, 2020a). Although previous work has studied adversarial examples in NLP (Li et al, 2017;Zang et al, 2020;Morris et al, 2020;Mozes et al, 2021) most of them focused on accuracy as a metric of interest. Among the ones that studied toxicity and other ethical considerations (Wallace et al, 2019;Sheng et al, 2020) they did not put the focus on either conversational agents or they did not consider attacks being imperceptible.…”

Towards Stronger Adversarial Baselines Through Human-AI Collaboration

“…The Adversarial NLI project asks humans to annotate mislabeled data and uses humans as adversaries to create a benchmark natural language inference (NLI) dataset for a more robust NLP model (Nie et al, 2020). The most related work compares the performance of human-and machinegenerated word-level adversarial examples for NLP classification tasks (Mozes et al, 2021).…”

“…A saliency map shows what words the target model identifies as most important that are most likely to affect the prediction, and then marks those words with colors with different intensities. Unlike (Mozes et al, 2021), where the interface displays word saliencies calculated by replacing the word with an out-of-vocabulary token, we implement the built-in method in each automated attack to calculate the saliency score. For example, BAE and TextFooler simply delete the word and calculate the word saliencies, while PWWS replaces each word with an unknown token and calculates the weighted saliency.…”

“…Thus far, human expertise in adversarial NLP tasks has been limited. There is a growing body of work in which humans are asked to craft inputs where a given model will perform poorly, but they receive little support in doing so -sometimes word saliences (Mozes et al, 2021), sometimes model predictions (Kiela et al, 2021), and sometimes even less. Overall, the effort between humans and machines is still largely separate; that is, humans generate adversarial examples alone based on model interpretations, without directly interacting with any attack algorithms.…”