AugVic: Exploiting BiText Vicinity for Low-Resource NMT

Abstract

The success of Neural Machine Translation (NMT) largely depends on the availability of large bitext training corpora. Due to the lack of such large corpora in low-resource language pairs, NMT systems often exhibit poor performance. Extra relevant monolingual data often helps, but acquiring it could be quite expensive, especially for low-resource languages. Moreover, domain mismatch between bitext (train/test) and monolingual data might degrade the performance. To alleviate such issues, we propose AUGVIC, a novel data augmentation framework for low-resource NMT which exploits the vicinal samples of the given bitext without using any extra monolingual data explicitly. It can diversify the in-domain bitext data with finer level control. Through extensive experiments on four low-resource language pairs comprising data from different domains, we have shown that our method is comparable to the traditional back-translation that uses extra in-domain monolingual data. When we combine the synthetic parallel data generated from AUGVIC with the ones from the extra monolingual data, we achieve further improvements. We show that AUGVIC helps to attenuate the discrepancies between relevant and distant-domain monolingual data in traditional back-translation. To understand the contributions of different components of AUGVIC, we perform an in-depth framework analysis.

CipherDAug: Ciphertext based Data Augmentation for Neural Machine Translation

“…Kiyono 2021, Karpukhin et al 2019). An advantage of soft replacements over hard ones is that they take into account the context of the tokens being replaced (Liu et al, 2021;Mohiuddin et al, 2021). These methods require architectural changes to a model whereas CipherDAug does not.…”

CipherDAug: Ciphertext based Data Augmentation for Neural Machine Translation

“…Kiyono 2021, Karpukhin et al 2019). An advantage of soft replacements over hard ones is that they take into account the context of the tokens being replaced Mohiuddin et al, 2021). These methods require architectural changes to a model whereas CipherDAug does not.…”

Towards robust natural language and image processing In low-resource scenarios

“…In Chapter 3 we leverage language models to generate synthetic labeled sequences for sequence tagging task data augmentation. Data augmentation has also been proven to be useful in the crosslingual settings [68][69][70][71][72][73]. Moreover most of the exiting methods overlook the better utilization of multilingual training data when such resources are available, so in Chapter 3 we explore methods to exploit translation models and multilingual resources for data augmentation.…”