Neural machine translation of low-resource languages using SMT phrase pair injection

Abstract

Neural machine translation (NMT) has recently shown promising results on publicly available benchmark datasets and is being rapidly adopted in various production systems. However, it requires high-quality large-scale parallel corpus, and it is not always possible to have sufficiently large corpus as it requires time, money, and professionals. Hence, many existing large-scale parallel corpus are limited to the specific languages and domains. In this paper, we propose an effective approach to improve an NMT system in low-resource scenario without using any additional data. Our approach aims at augmenting the original training data by means of parallel phrases extracted from the original training data itself using a statistical machine translation (SMT) system. Our proposed approach is based on the gated recurrent unit (GRU) and transformer networks. We choose the Hindi–English, Hindi–Bengali datasets for Health, Tourism, and Judicial (only for Hindi–English) domains. We train our NMT models for 10 translation directions, each using only 5–23k parallel sentences. Experiments show the improvements in the range of 1.38–15.36 BiLingual Evaluation Understudy points over the baseline systems. Experiments show that transformer models perform better than GRU models in low-resource scenarios. In addition to that, we also find that our proposed method outperforms SMT—which is known to work better than the neural models in low-resource scenarios—for some translation directions. In order to further show the effectiveness of our proposed model, we also employ our approach to another interesting NMT task, for example, old-to-modern English translation, using a tiny parallel corpus of only 2.7K sentences. For this task, we use publicly available old-modern English text which is approximately 1000 years old. Evaluation for this task shows significant improvement over the baseline NMT.

Improved English to Hindi Multimodal Neural Machine Translation

“…This score was improved later in the task of WAT2020 (Laskar et al, 2020c) and utilizes pre-train word embeddings of the monolingual corpus and additional parallel data of IITB. This work attempts to utilize phrase pairs (Sen et al, 2020) to enhance the translational performance of the WAT2021: English to Hindi multimodal translation task.…”

“…The attention-based NMT yields substantial performance for Indian language translation Laskar et al, 2019aLaskar et al, ,b, 2020aLaskar et al, , 2021b. Moreover, NMT performance can be enhanced by utilizing monolingual data (Sennrich et al, 2016;Zhang and Zong, 2016;Laskar et al, 2020b) and phrase pair injection (Sen et al, 2020), effective in low resource language pair translation. This paper aims English to Hindi translation using the multimodal concept by taking advantage of monolingual data and phrase pair injections to improve the translation quality at the WAT2021 translation task.…”

Proceedings of the 8th Workshop on Asian Translation (WAT2021)

“…In (Sen et al, 2020), authors used SMT-based phrase pairs to augment with the original parallel data to improve low-resource language pairs translation. In SMT 3 , Giza++ word alignment tool is used to extract phrase pair.…”

“…In SMT 3 , Giza++ word alignment tool is used to extract phrase pair. Inspired by the work (Sen et al, 2020), we have extracted phrase pairs using Giza++ 4 . Then after removing duplicates and blank lines, the obtained phrase pairs are augmented to the original parallel data.…”

“…The attention-based NMT yields substantial performance for Indian language translation Laskar et al, 2019aLaskar et al, ,b, 2020aLaskar et al, , 2021b. Moreover, NMT performance can be enhanced by utilizing monolingual data Laskar et al, 2020b) and phrase pair injection (Sen et al, 2020), effective in low resource language pair translation. This paper aims English to Hindi translation using the multimodal concept by taking advantage of monolingual data and phrase pair injections to improve the translation quality at the WAT2021 translation task.…”