Towards Robust Neural Machine Translation

Abstract

Small perturbations in the input can severely distort intermediate representations and thus impact translation quality of neural machine translation (NMT) models. In this paper, we propose to improve the robustness of NMT models with adversarial stability training. The basic idea is to make both the encoder and decoder in NMT models robust against input perturbations by enabling them to behave similarly for the original input and its perturbed counterpart. Experimental results on Chinese-English, English-German and English-French translation tasks show that our approaches can not only achieve significant improvements over strong NMT systems but also improve the robustness of NMT models.

DTMT: A Novel Deep Transition Architecture for Neural Machine Translation

“…Experimental results on NIST Chinese⇒English translation show that DTMT can outperform the Transformer model by +2.09 BLEU points and achieve the best results ever reported in the same dataset. On WMT14 English⇒German and English⇒French translation, it consistently leads to substantial improvements and shows superior quality to the state-of-the-art NMT systems (Vaswani et al 2017;Cheng et al 2018). The main contributions of this paper can be summarized as follows:…”

“…Zhang et al (2018) propose to exploit both left-to-right and right-to-left decoding strategies to capture bidirectional dependencies. Meng et al (2018) propose key-value memory augmented attention to improve the adequacy of translation. Compared with them, our baseline system SHALLOWRNMT outperforms their best models by more than 3 BLEU points.…”

An Analysis of Source-Side Grammatical Errors in NMT

“…For instance, one could optimize for f-BLEU or any of the other reference-less measures that we proposed, in the same way that an MT system is optimized for BLEU (either by explicitly using their scores through reinforcement learning or by simply using the metric as an early stopping criterion over a development set). Cheng et al (2018) recently proposed an approach for training more robust MT systems, albeit in a supervised setting where noise is injected on parallel data, and the proposed solutions of Belinkov and Bisk (2018) and Anastasopoulos et al (2019) fall within the same category. However, no approach has, to our knowledge, used GEC corpora for training MT systems robust to grammatical errors.…”

Cross-Lingual Machine Reading Comprehension

“…• Translation: We use Google Neural Machine Translation (GNMT) system for translation. 7 We evaluated GNMT system on NIST MT02/03/04/05/06/08 Chinese-English set and achieved an average BLEU score of 43.24, compared to previous best work (43.20) (Cheng et al, 2018), yielding state-of-the-art performance.…”