A Survey of Data Augmentation Approaches for NLP

Robust Natural Language Processing: Recent Advances, Challenges, and Future Directions

“…For the completeness of treatment of the subject, we address those related surveys in the following. Feng et al [14] conducted an extensive survey on data augmentation for NLP robustness. They studied various data augmentation techniques, including rule-based and modelbased techniques as strategies to robustify NLP models to adversarial attacks.…”

“…The authors claim that they can certify the classifications of over 50% texts to any perturbation of 5 words on AGNEWS dataset, and 2 words in the SST-2 dataset (dataset-dependent). The interested reader could find more details in the comprehensive survey on data augmentation techniques, their advantages, and disadvantages in [14].…”

“…For instance, one approach to show this practice is by exposing the model to a training set, and not to samples from within the distribution of the testing set. One possible benefit of examining models with such settings is that surface cues can be identified to show limitations of models, as pointed out in a previous section, where gaps could be addressed by using techniques that can help with robustness and generalization, e.g., data augmentation techniques [14].…”

“…Broadly speaking, such efforts in the literature are either focused on developing new attacks or better training models to make models resistant to such attacks (i.e., defenses) [13]. To sum up the research efforts dedicated understanding robustness in the literature, there are several research surveys that have addressed specific aspects of NLP robustness, e.g., data augmentation [14], search methods [15], pretrained models [16], and adversarial attacks [17]. However, the literature lacks research studies that provide a systematic overview of the state-of-the-art in this space across a range of variables; applications, technique, metrics, benchmark datasets, threat models, tasks, embedding techniques, learning techniques, goals, defense mechanisms, and performance.…”

The King is Naked: on the Notion of Robustness for Natural Language Processing

“…Neural network robustness naturally complements the perspective offered by brittleness as it involves the certification of a model against a wide range of attacks (Huang et al 2017). In NLP, similarly to computer vision (Akhtar and Mian 2018), the majority of works have adopted the narrow notion of robustness, in terms of invariance to minor perturbations of an input text (Gowal et al 2018;Jia et al 2019;Dong et al 2021;La Malfa et al 2020), while only a minority have contested these limitations, either implicitly (Ribeiro et al 2020) or explicitly (Morris 2020;Morris et al 2020a;Xu et al 2020), mainly due to the difficulty of automatically generating semantically involved test beds (Feng et al 2021). Although adversarial data augmentation in NLP is well established (Morris et al 2020b), robustness to semantically coherent, yet possibly diverging, examples is still in its 'adolescence' (Ribeiro, Singh, and Guestrin 2018), as many highly accurate NLP models cannot recognize cogent linguistic phenomena even on low-order tasks such as binary classification (Barnes, Øvrelid, and Velldal 2019).…”