Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages

Kim, Yunsu; Petrov, Petre; Petrushkov, Pavel; Khadivi, Shahram; Ney, Hermann

doi:10.18653/v1/d19-1080

Cited by 44 publications

(37 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Combining Pre-trained Encoders and Decoders. Kim et al [72] combined S-P encoder with P-T decoder to create the S-T model. They improved the simple initialization using some finetuning objectives and/or source-pivot adaptation to ensure that source and pivot representations are aligned.…”

Section: Zero-resource Translationmentioning

confidence: 99%

A Survey of Multilingual Neural Machine Translation

2020

View full text Add to dashboard Cite

We present a survey on multilingual neural machine translation (MNMT), which has gained a lot of traction in recent years. MNMT has been useful in improving translation quality as a result of translation knowledge transfer (transfer learning). MNMT is more promising and interesting than its statistical machine translation counterpart, because end-to-end modeling and distributed representations open new avenues for research on machine translation. Many approaches have been proposed to exploit multilingual parallel corpora for improving translation quality. However, the lack of a comprehensive survey makes it difficult to determine which approaches are promising and, hence, deserve further exploration. In this article, we present an indepth survey of existing literature on MNMT. We first categorize various approaches based on their central use-case and then further categorize them based on resource scenarios, underlying modeling principles, coreissues, and challenges. Wherever possible, we address the strengths and weaknesses of several techniques by comparing them with each other. We also discuss the future directions for MNMT. This article is aimed towards both beginners and experts in NMT. We hope this article will serve as a starting point as well as a source of new ideas for researchers and engineers interested in MNMT.

show abstract

Section: Zero-resource Translationmentioning

confidence: 99%

A Survey of Multilingual Neural Machine Translation

2020

View full text Add to dashboard Cite

show abstract

“…Back-translation is simple and easy to achieve without modifying the architecture of the machine translation models. Back-translation has been studied in both SMT [111][112][113] and NMT [23,80,110,[114][115][116].…”

Section: Pivot Translationmentioning

confidence: 99%

“…However, CBMT systems suffer from the lack of parallel corpora for under-resourced languages to train machine translation systems. A number of the methods have been proposed to address the non-availability of parallel corpora for under-resourced languages, such as pivot-based approaches [23][24][25], zero-shot translation [26][27][28][29][30] and unsupervised methods [31][32][33], which are described in detail in following sections. A large array of techniques have been applied to overcome the data sparsity problem in MT, and virtually all of them seem to be based on the field of transfer learning from high-resource languages in recent years.…”

Section: Introductionmentioning

confidence: 99%

A Survey of Orthographic Information in Machine Translation

et al. 2021

View full text Add to dashboard Cite

Machine translation is one of the applications of natural language processing which has been explored in different languages. Recently researchers started paying attention towards machine translation for resource-poor languages and closely related languages. A widespread and underlying problem for these machine translation systems is the linguistic difference and variation in orthographic conventions which causes many issues to traditional approaches. Two languages written in two different orthographies are not easily comparable but orthographic information can also be used to improve the machine translation system. This article offers a survey of research regarding orthography’s influence on machine translation of under-resourced languages. It introduces under-resourced languages in terms of machine translation and how orthographic information can be utilised to improve machine translation. We describe previous work in this area, discussing what underlying assumptions were made, and showing how orthographic knowledge improves the performance of machine translation of under-resourced languages. We discuss different types of machine translation and demonstrate a recent trend that seeks to link orthographic information with well-established machine translation methods. Considerable attention is given to current efforts using cognate information at different levels of machine translation and the lessons that can be drawn from this. Additionally, multilingual neural machine translation of closely related languages is given a particular focus in this survey. This article ends with a discussion of the way forward in machine translation with orthographic information, focusing on multilingual settings and bilingual lexicon induction.

show abstract

“…However, achieving satisfactory performance in low-resource settings turns out to be challenging for NMT systems (Koehn and Knowles, 2017). Recent research has mainly focused on creating and cleaning parallel (Ramasamy et al, 2014;Islam, 2018) and comparable data (Tiedemann, 2012), utilizing bilingual lexicon induction (Conneau et al, 2017;Artetxe et al, 2018;Joty, 2019, 2020;, fine-grained hyperparameter tuning (Sennrich and Zhang, 2019), and using other language pairs as pivot (Cheng et al, 2017;Kim et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

AugVic: Exploiting BiText Vicinity for Low-Resource NMT

Mohiuddin¹,

Bari²,

Joty³

2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

The success of Neural Machine Translation (NMT) largely depends on the availability of large bitext training corpora. Due to the lack of such large corpora in low-resource language pairs, NMT systems often exhibit poor performance. Extra relevant monolingual data often helps, but acquiring it could be quite expensive, especially for low-resource languages. Moreover, domain mismatch between bitext (train/test) and monolingual data might degrade the performance. To alleviate such issues, we propose AUGVIC, a novel data augmentation framework for low-resource NMT which exploits the vicinal samples of the given bitext without using any extra monolingual data explicitly. It can diversify the in-domain bitext data with finer level control. Through extensive experiments on four low-resource language pairs comprising data from different domains, we have shown that our method is comparable to the traditional back-translation that uses extra in-domain monolingual data. When we combine the synthetic parallel data generated from AUGVIC with the ones from the extra monolingual data, we achieve further improvements. We show that AUGVIC helps to attenuate the discrepancies between relevant and distant-domain monolingual data in traditional back-translation. To understand the contributions of different components of AUGVIC, we perform an in-depth framework analysis.

show abstract

Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages

Cited by 44 publications

References 26 publications

A Survey of Multilingual Neural Machine Translation

A Survey of Multilingual Neural Machine Translation

A Survey of Orthographic Information in Machine Translation

AugVic: Exploiting BiText Vicinity for Low-Resource NMT

Contact Info

Product

Resources

About