Addressing word-order Divergence in Multilingual Neural Machine Translation for extremely Low Resource Languages

Murthy, Rudra; Kunchukuttan, Anoop; Bhattacharyya, Pushpak

doi:10.18653/v1/n19-1387

Cited by 30 publications

(38 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The choice of a single encoder for all languages is also promoted by Hokamp et al [64], who opt for language-specific decoders. Murthy et al [101] pointed out that the sentence representations generated by the encoder are dependent on the word order of the language and are, hence, language-specific. They focused on reordering input sentences to reduce the divergence caused due to different word orders to improve the quality of transfer learning.…”

Section: Addressing Language Divergencementioning

confidence: 99%

“…Although it is important to address this issue there are surprisingly few works that address it. Murthy et al [101] showed that reducing the word order divergence between source languages by reordering the parent sentences to match child word order is beneficial in extremely low-resource scenarios. Since reordering is part of the pre-processing pipeline, it is referred to as pre-ordering.…”

Section: Syntactic Transfermentioning

confidence: 99%

See 1 more Smart Citation

A Survey of Multilingual Neural Machine Translation

2020

Self Cite

View full text Add to dashboard Cite

We present a survey on multilingual neural machine translation (MNMT), which has gained a lot of traction in recent years. MNMT has been useful in improving translation quality as a result of translation knowledge transfer (transfer learning). MNMT is more promising and interesting than its statistical machine translation counterpart, because end-to-end modeling and distributed representations open new avenues for research on machine translation. Many approaches have been proposed to exploit multilingual parallel corpora for improving translation quality. However, the lack of a comprehensive survey makes it difficult to determine which approaches are promising and, hence, deserve further exploration. In this article, we present an indepth survey of existing literature on MNMT. We first categorize various approaches based on their central use-case and then further categorize them based on resource scenarios, underlying modeling principles, coreissues, and challenges. Wherever possible, we address the strengths and weaknesses of several techniques by comparing them with each other. We also discuss the future directions for MNMT. This article is aimed towards both beginners and experts in NMT. We hope this article will serve as a starting point as well as a source of new ideas for researchers and engineers interested in MNMT.

show abstract

Section: Addressing Language Divergencementioning

confidence: 99%

Section: Syntactic Transfermentioning

confidence: 99%

A Survey of Multilingual Neural Machine Translation

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Fine-tuning based transfer learning has been studied for transferring proper parameters Gu et al, 2018b), lexical Nguyen and Chiang, 2017;Gu et al, 2018a;Lakew et al, 2018), and syntactic (Gu et al, 2018a;Murthy et al, 2018) knowledge from a resource-rich language pair to a resource-poor language pair. On the other hand, Chu et al (2017) proposed a more robust training approach for domain adaptation, called mixed fine-tuning, which uses a mixture of data from different domains.…”

Section: Related Workmentioning

confidence: 99%

Exploiting Multilingualism through Multistage Fine-Tuning for Low-Resource Neural Machine Translation

Dabre¹,

Fujita²,

Chu³

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

This paper highlights the impressive utility of multi-parallel corpora for transfer learning in a one-to-many low-resource neural machine translation (NMT) setting. We report on a systematic comparison of multistage finetuning configurations, consisting of (1) pretraining on an external large (209k-440k) parallel corpus for English and a helping target language, (2) mixed pre-training or fine-tuning on a mixture of the external and low-resource (18k) target parallel corpora, and (3) pure finetuning on the target parallel corpora. Our experiments confirm that multi-parallel corpora are extremely useful despite their scarcity and content-wise redundancy thus exhibiting the true power of multilingualism. Even when the helping target language is not one of the target languages of our concern, our multistage finetuning can give 3-9 BLEU score gains over a simple one-to-one model.

show abstract

“…Murthy et al [27] exploited preordering for low-resource NMT with transfer learning. They first trained the translation model on languages with an abundant parallel corpus.…”

Section: B Usage Of Reordering Information In Nmtmentioning

confidence: 99%

Preordering Encoding on Transformer for Translation

Kawara

Chu

Arase

2021

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

The difference in word orders between source and target languages is a serious hurdle for machine translation. Preordering methods, which reorder the words in a source sentence before translation to obtain a similar word ordering with a target language, significantly improve the quality in statistical machine translation. While the information on the preordering position improved the translation quality in recurrent neural network-based models, questions such as how to use preordering information and whether it is helpful for the Transformer model remain unaddressed. In this paper, we successfully employed preordering techniques in the Transformer-based neural machine translation. Specifically, we proposed a novel preordering encoding that exploits the reordering information of the source and target sentences as positional encoding in the Transformer model. Experimental results on ASPEC Japanese-English and WMT 2015 English-German, English-Czech, and English-Russian translation tasks confirmed that the proposed method significantly improved the translation quality evaluated by the BLEU scores of the Transformer model by 1.34 points in the Japanese-to-English task, 2.19 points in the English-to-German task, 0.15 points in the Czech-to-English task, and 1.48 points in the English-to-Russian task.

show abstract

Addressing word-order Divergence in Multilingual Neural Machine Translation for extremely Low Resource Languages

Cited by 30 publications

References 14 publications

A Survey of Multilingual Neural Machine Translation

A Survey of Multilingual Neural Machine Translation

Exploiting Multilingualism through Multistage Fine-Tuning for Low-Resource Neural Machine Translation

Preordering Encoding on Transformer for Translation

Contact Info

Product

Resources

About