Generalizing Back-Translation in Neural Machine Translation

Graça, Miguel; Kim, Yunsu; Schamper, Julian; Khadivi, Shahram; Ney, Hermann

doi:10.48550/arxiv.1906.07286

Cited by 2 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Various studies have investigated back-translation with the aim to improve the backward model, to select the most suitable generation/decoding methods and to reduce the impact of the ratio of the synthetic to real bitext. Various studies have found that the quality of the models trained using back-translation depends on the quality of the backward model [5], [9], [11], [15], [25], [27], [28]. To improve the quality of the synthetic parallel data, [9] used iterative back-translationiteratively using the back-translated data to improve the backward model, Kocmi and Bojar [28] and Dabre et al [29] used high resource languages through transfer learning and Zhang et al [30] explored the use of both target and source monolingual data to improve both the backward and forward models.…”

Section: B Leveraging Monolingual Data For Nmtmentioning

confidence: 99%

See 1 more Smart Citation

Tag-less Back-Translation

Abdulmumin

Galadanci

Aliyu

2021

Preprint

View full text Add to dashboard Cite

An effective method to generate a large number of parallel sentences for training improved neural machine translation (NMT) systems is the use of the back-translations of the target-side monolingual data. The standard back-translation method has been shown to be unable to efficiently utilize the available huge amount of existing monolingual data because of the inability of translation models to differentiate between the authentic and synthetic parallel data during training. Tagging, or using gates, has been used to enable translation models to distinguish between synthetic and authentic data, improving standard back-translation and also enabling the use of iterative back-translation on language pairs that underperformed using standard back-translation. In this work, we approach back-translation as a domain adaptation problem, eliminating the need for explicit tagging. In the approach - tag-less back-translation - the synthetic and authentic parallel data are treated as out-of-domain and in-domain data respectively and, through pre-training and fine-tuning, the translation model is shown to be able to learn more efficiently from them during training. Experimental results have shown that the approach outperforms the standard and tagged back-translation approaches on low resource English-Vietnamese and English-German neural machine translation.

show abstract

Section: B Leveraging Monolingual Data For Nmtmentioning

confidence: 99%

“…Recently, researchers have proposed methods to exploit the easier-to-get monolingual data of one or both of the languages to augment the parallel data and improve the performance of the translation models. Such methods include integrating a language model [8], back-translation [9]- [11], forward translation [12] and dual learning [13].…”

Section: Introductionmentioning

confidence: 99%

Tag-less Back-Translation

Abdulmumin

Galadanci

Aliyu

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…To generate a better synthetic sentence for the BT method, various generation methods have been proposed such as greedy, beam search, stochastic sampling, filtered sampling (Cheng, 2019;He et al, 2016;Imamura et al, 2018;Edunov et al, 2018;Fadaee and Monz, 2018;Graça et al, 2019;Caswell et al, 2019;Wu et al, 2019). Among them, stochastic sampling or filtered sampling demonstrated promising performances.…”

Section: Introductionmentioning

confidence: 99%

End-to-End Training of Both Translation Models in the Back-Translation Framework

Heo¹,

Choi²

2022

Preprint

View full text Add to dashboard Cite

Semi-supervised learning algorithms in neural machine translation (NMT) have significantly improved translation quality compared to the supervised learning algorithms by using additional monolingual corpora. Among them, back-translation is a theoretically wellstructured and cutting-edge method. Given two pre-trained NMT models between source and target languages, one translates a monolingual sentence as a latent sentence, and the other reconstructs the monolingual input sentence given the latent sentence. Therefore, previous works tried to apply the variational auto-encoder's (VAE) training framework to the back-translation framework. However, the discrete property of the latent sentence made it impossible to use backpropagation in the framework. This paper proposes a categorical reparameterization trick that generates a differentiable sentence, with which we practically implement the VAE's training framework for the back-translation and train it by end-to-end backpropagation. In addition, we propose several regularization techniques that are especially advantageous to this framework. In our experiments, we demonstrate that our method makes backpropagation available through the latent sentences and improves the BLEU scores on the datasets of the WMT18 translation task.

show abstract

Generalizing Back-Translation in Neural Machine Translation

Cited by 2 publications

References 0 publications

Tag-less Back-Translation

Tag-less Back-Translation

End-to-End Training of Both Translation Models in the Back-Translation Framework

Contact Info

Product

Resources

About