Searching for a Search Method: Benchmarking Search Algorithms for Generating NLP Adversarial Examples

Abstract

We study the behavior of several black-box search algorithms used for generating adversarial examples for natural language processing (NLP) tasks. We perform a fine-grained analysis of three elements relevant to search: search algorithm, search space, and search budget. When new search methods are proposed in past work, the attack search space is often modified alongside the search method. Without ablation studies benchmarking the search algorithm change with the search space held constant, an increase in attack success rate could from an improved search method or a less restrictive search space. Additionally, many previous studies fail to properly consider the search algorithms' run-time cost, which is essential for downstream tasks like adversarial training. Our experiments provide a reproducible benchmark of search algorithms across a variety of search spaces and query budgets to guide future research in adversarial NLP. Based on our experiments, we recommend greedy attacks with word importance ranking when under a time constraint or attacking long inputs, and either beam search or particle swarm optimization otherwise. * * Equal contribution. Code implementation shared via https://github.com/QData/TextAttack 1 In this work, we use "adversarial example generation methods" and "adversarial attacks" interchangeably.

Robust Natural Language Processing: Recent Advances, Challenges, and Future Directions

“…Yoo et al [15] presented review of search algorithms for generating adversarial examples as a means to achieve robustness. However, their work is limited in scope in that it only focuses on adversarial examples as a means to seek robustness.…”

“…3) Sparse Projected Gradient Descent: The projected gradient descent method is a type of greedy algorithm which has been applied broadly to machine learning models [36]. In this method, each element in the input text is considered for substitution and the best perturbations are selected from all possible perturbations and rerun until no more perturbations are possible [15]. This attack method has been utilized in several research works [15], [43], [90] with various promising results.…”

“…In this method, each element in the input text is considered for substitution and the best perturbations are selected from all possible perturbations and rerun until no more perturbations are possible [15]. This attack method has been utilized in several research works [15], [43], [90] with various promising results. For example, Barham et al [43] introduced a sparse projected gradient descent (SPGD) method for crafting interpretable AEs for text applications.…”

“…In this method, each member of the population is perturbed by creating all potential candidate obtained by replacing each input and then sampling one input example, at each iteration. Using this algorithm, we are able to find the best perturbed input among all members of the population [15]. This attack technique has been used in multiple studies addressing machine learning robustness to adversarial attacks in general, including the studies in [12], [15], [53], [96], [97].…”

“…Broadly speaking, such efforts in the literature are either focused on developing new attacks or better training models to make models resistant to such attacks (i.e., defenses) [13]. To sum up the research efforts dedicated understanding robustness in the literature, there are several research surveys that have addressed specific aspects of NLP robustness, e.g., data augmentation [14], search methods [15], pretrained models [16], and adversarial attacks [17]. However, the literature lacks research studies that provide a systematic overview of the state-of-the-art in this space across a range of variables; applications, technique, metrics, benchmark datasets, threat models, tasks, embedding techniques, learning techniques, goals, defense mechanisms, and performance.…”