Instance Weighting for Neural Machine Translation Domain Adaptation

Wang, Rui; Utiyama, Masao; Liu, Lemao; Chen, Kehai; Sumita, Eiichiro

doi:10.18653/v1/d17-1155

Cited by 128 publications

(90 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Meanwhile, applying data weighting into NMT domain adaptation has attracted much attention. Wang et al (2017a) and Wang et al (2017b) proposed several sentence and domain weighting methods with a dynamic weight learning strategy. Zhang et al (2019a) ranked unlabeled domain training samples based on their similarity to in-domain data, and then adopts a probabilistic curriculum learning strategy during training.…”

Section: Related Workmentioning

confidence: 99%

Iterative Dual Domain Adaptation for Neural Machine Translation

Zeng¹,

Liu²,

Su³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Previous studies on the domain adaptation for neural machine translation (NMT) mainly focus on the one-pass transferring out-ofdomain translation knowledge to in-domain NMT model. In this paper, we argue that such a strategy fails to fully extract the domainshared translation knowledge, and repeatedly utilizing corpora of different domains can lead to better distillation of domain-shared translation knowledge. To this end, we propose an iterative dual domain adaptation framework for NMT. Specifically, we first pretrain in-domain and out-of-domain NMT models using their own training corpora respectively, and then iteratively perform bidirectional translation knowledge transfer (from indomain to out-of-domain and then vice versa) based on knowledge distillation until the indomain NMT model convergences. Furthermore, we extend the proposed framework to the scenario of multiple out-of-domain training corpora, where the above-mentioned transfer is performed sequentially between the indomain and each out-of-domain NMT models in the ascending order of their domain similarities. Empirical results on Chinese-English and English-German translation tasks demonstrate the effectiveness of our framework.

show abstract

Section: Related Workmentioning

confidence: 99%

Iterative Dual Domain Adaptation for Neural Machine Translation

Zeng¹,

Liu²,

Su³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

show abstract

“…They modify NMT model to also accept the marked target sentence as input and train it to produce similar sentences that do not contain marked words. Wang et al, 2017) have proposed sentence level weighting method for domain adaptation in NMT.…”

Section: Related Workmentioning

confidence: 99%

Learning from Chunk-based Feedback in Neural Machine Translation

Petrushkov

Khadivi

Matusov

2018

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

View full text Add to dashboard Cite

We empirically investigate learning from partial feedback in neural machine translation (NMT), when partial feedback is collected by asking users to highlight a correct chunk of a translation. We propose a simple and effective way of utilizing such feedback in NMT training. We demonstrate how the common machine translation problem of domain mismatch between training and deployment can be reduced solely based on chunk-level user feedback. We conduct a series of simulation experiments to test the effectiveness of the proposed method. Our results show that chunk-level feedback outperforms sentence based feedback by up to 2.61% BLEU absolute.

show abstract

“…Our training objective (2) can be seen as a generalization of the original training objective (1) and instance weighting methods [19,20]. The original loss (1) sets w t = 1 for every word in all sentences.…”

Section: Objectivementioning

confidence: 99%

“…Mixed fine-tuning [18] combines fine-tuning and multi-domain NMT. Similar to instance weighting in SMT, sentence/domain weighting methods [19,20] can also be used for NMT domain adaptation based on different objectives. DA with meta information [21] is proposed to train topic-aware models using domain-specific tags for the decoder.…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Domain Adaptation for Neural Machine Translation

Yang

Chen

Wang

et al. 2018

2018 24th International Conference on Pattern Recognition (ICPR)

View full text Add to dashboard Cite

In this paper, we empirically investigate applying word-level weights to adapt neural machine translation to e-commerce domains, where small e-commerce datasets and large outof-domain datasets are available. In order to mine indomain like words in the out-of-domain datasets, we compute word weights by using a domain-specific and a nondomain-specific language model followed by smoothing and binary quantization. The baseline model is trained on mixed in-domain and out-of-domain datasets. Experimental results on En → Zh e-commerce domain translation show that compared to continuing training without word weights, it improves MT quality by up to 2.11% BLEU absolute and 1.59% TER. We have also trained models using fine-tuning on the in-domain data. Pre-training a model with word weights improves fine-tuning up to 1.24% BLEU absolute and 1.64% TER, respectively.

show abstract

Instance Weighting for Neural Machine Translation Domain Adaptation

Cited by 128 publications

References 13 publications

Iterative Dual Domain Adaptation for Neural Machine Translation

Iterative Dual Domain Adaptation for Neural Machine Translation

Learning from Chunk-based Feedback in Neural Machine Translation

Unsupervised Domain Adaptation for Neural Machine Translation

Contact Info

Product

Resources

About