Active Learning Approaches to Enhancing Neural Machine Translation

Abstract

Active learning is an efficient approach for mitigating data dependency when training neural machine translation (NMT) models. In this paper we explore new training frameworks by incorporating active learning into various techniques such as transfer learning and iterative back-translation (IBT) under a limited human translation budget. We design a word frequency based acquisition function and combine it with a strong uncertainty based method. The combined method steadily outperforms all other acquisition functions in various scenarios. As far as we know, we are the first to do a large-scale study on actively training Transformer (Vaswani et al., 2017) for NMT. Specifically, with a human translation budget of only 20% of the original parallel corpus, we manage to surpass Transformer trained on the entire parallel corpus in three language pairs.

Active Learning for Neural Machine Translation

“…We develop a word frequency-based technique that considers linguistic factors for the data driven function. It has been found that active NMT training is advantageous for both varieties of acquisition functions [Zhao et al, 2020]. In this way, we can build a non-redundant translated corpus on which NMT can be trained to achieve a better performance than models trained on randomly built corpora.…”

DaLC: Domain Adaptation Learning Curve Prediction for Neural Machine Translation

“…Active Learning (AL) algorithms are built to select the most useful samples for improving the performance of a given model. Therefore, the criteria used by AL algorithms in NLP or MT tasks (Zhang et al, 2017;Zhao et al, 2020;Peris and Casacuberta, 2018;Dou et al, 2020;Zhan et al, 2021) could also serve as discriminative features when it comes to predicting the future NMT performance. We reuse some of the scoring functions introduced by previous work as specified in Section 4.2.…”

DaLC: Domain Adaptation Learning Curve Prediction for Neural Machine Translation

“…Active Learning (AL) algorithms are built to select the most useful samples for improving the performance of a given model. Therefore, the criteria used by AL algorithms in NLP or MT tasks (Zhang et al, 2017;Zhao et al, 2020;Peris and Casacuberta, 2018;Dou et al, 2020;Zhan et al, 2021) could also serve as discriminative features when it comes to predicting the future NMT performance. We reuse some of the scoring functions introduced by previous work as specified in Section 4.2.…”