AdMix: A Mixed Sample Data Augmentation Method for Neural Machine Translation

Abstract

In Neural Machine Translation (NMT), data augmentation methods such as back-translation have proven their effectiveness in improving translation performance. In this paper, we propose a novel data augmentation approach for NMT, which is independent of any additional training data. Our approach, AdMix, consists of two parts: 1) introduce faint discrete noise (word replacement, word dropping, word swapping) into the original sentence pairs to form augmented samples; 2) generate new synthetic training data by softly mixing the augmented samples with their original samples in training corpus. Experiments on three translation datasets of different scales show that AdMix achieves significant improvements (1.0 to 2.7 BLEU points) over strong Transformer baseline. When combined with other data augmentation techniques (e.g., back-translation), our approach can obtain further improvements.

Measuring the Impact of Data Augmentation Methods for Extremely Low-Resource NMT

“…Sugiyama and Yoshinaga (2019) show that back translation has a significant positive impact on context-aware large-scale NMT tasks. Several iterations of previous work (Jin et al 2022, Aji & Heafield 2020, Li & Specia 2019 show that back-translation can supplement other data augmentation techniques to improve performance in neural translation tasks. Such work emphasizes the need to better understand the impact of backtranslation in low-resource environments so that such work can keep pace with work in highresource settings.…”