Crafting Adversarial Examples for Neural Machine Translation

Abstract

Effective adversary generation for neural machine translation (NMT) is a crucial prerequisite for building robust machine translation systems. In this work, we investigate veritable evaluations of NMT adversarial attacks, and propose a novel method to craft NMT adversarial examples. We first show the current NMT adversarial attacks may be improperly estimated by the commonly used monodirectional translation, and we propose to leverage the round-trip translation technique to build valid metrics for evaluating NMT adversarial attacks. Our intuition is that an effective NMT adversarial example, which imposes minor shifting on the source and degrades the translation dramatically, would naturally lead to a semantic-destroyed round-trip translation result. We then propose a promising black-box attack method called Word Saliency speedup Local Search (WSLS) that could effectively attack the mainstream NMT architectures. Comprehensive experiments demonstrate that the proposed metrics could accurately evaluate the attack effectiveness, and the proposed WSLS could significantly break the state-of-art NMT models with small perturbation. Besides, WSLS exhibits strong transferability on attacking Baidu and Bing online translators.

A Word is Worth A Thousand Dollars: Adversarial Attack on Tweets Fools Stock Prediction

“…Even though the success rate of concatenation attack lags behind the state-of-the-art textual attack, the manipulation attack achieves performance of the same ballpark, which demonstrates the efficacy of optimization-based attack and our solvers. More importantly, it implies that the attack is not transferable between the two tasks, documenting more evidence on language attack transferability (Yuan et al, 2021;He et al, 2021). The bottom line is that they are two different tasks under different assumptions.…”

“…Our code is available at https://github.com/yonxie/ AdvFinTweet It is now known that text-based deep learning models can be vulnerable to adversarial attacks (Szegedy et al, 2014;Goodfellow et al, 2015). The perturbation can be at the sentence level (e.g., Xu et al, 2021;Iyyer et al, 2018;Ribeiro et al, 2018), the word level (e.g., Zhang et al, 2019;Alzantot et al, 2018;Zang et al, 2020;Jin et al, 2020;Lei et al, 2019;Lin et al, 2021), or both (Chen et al, 2021). We are interested in whether such adversarial attack vulnerability also exists in stock prediction models, as these models embrace more and more human-generated media data (e.g., Twitter, Reddit, Stocktwit, Yahoo News (Xu and Cohen, 2018;Sawhney et al, 2021)).…”

Generating Authentic Adversarial Examples beyond Meaning-preserving with Doubly Round-trip Translation

“…The previous approaches for constructing NMT adversarial examples can be divided into two branches: white-box and black-box. The whitebox approaches are based on the assumption that the architecture and parameters of the NMT model are accessible (Ebrahimi et al, 2018;Cheng et al, 2019;Chen et al, 2021). These methods usually achieve superior performance since they can construct and defend the adversaries tailored for the model.…”

A Word is Worth A Thousand Dollars: Adversarial Attack on Tweets Fools Stock Prediction

“…Our code is available at https://github.com/yonxie/ AdvFinTweet It is now known that text-based deep learning models can be vulnerable to adversarial attacks (Szegedy et al, 2014;Goodfellow et al, 2015). The perturbation can be at the sentence level (e.g., Xu et al, 2021;Iyyer et al, 2018;Ribeiro et al, 2018), the word level (e.g., Zhang et al, 2019;Alzantot et al, 2018;Zang et al, 2020;Jin et al, 2020;Lei et al, 2019;Zhang et al, 2021;Lin et al, 2021), or both (Chen et al, 2021). We are interested in whether such adversarial attack vulnerability also exists in stock prediction models, as these models embrace more and more human-generated media data (e.g., Twitter, Reddit, Stocktwit, Yahoo News (Xu and Cohen, 2018;Sawhney et al, 2021)).…”