A Survey of Domain Adaptation for Machine Translation

Chu, Chenhui; Wang, Rui

doi:10.2197/ipsjjip.28.413

Cited by 115 publications

(124 citation statements)

References 50 publications

(36 reference statements)

Supporting

Mentioning

120

Contrasting

Unclassified

Order By: Relevance

“…Other surveys Comprehensive reviews on DA exist, each with a different focus: visual applications (Csurka, 2017;Patel et al, 2015;Wilson and Cook, 2020), machine translation (MT) (Chu and Wang, 2018), pre-neural DA methods in NLP (Jiang, 2008;Margolis, 2011). Seminal surveys in machine learning on transfer learning include Pan and Yang (2009), Weiss et al (2016), and.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Neural Unsupervised Domain Adaptation in NLP—A Survey

Ramponi¹,

Plank²

2020

Proceedings of the 28th International Conference on Computational Linguistics

135

108

View full text Add to dashboard Cite

Deep neural networks excel at learning from labeled data and achieve state-of-the-art results on a wide array of Natural Language Processing tasks. In contrast, learning from unlabeled data, especially under domain shift, remains a challenge. Motivated by the latest advances, in this survey we review neural unsupervised domain adaptation techniques which do not require labeled target domain data. This is a more challenging yet a more widely applicable setup. We outline methods, from early traditional non-neural methods to pre-trained model transfer. We also revisit the notion of domain, and we uncover a bias in the type of Natural Language Processing tasks which received most attention. Lastly, we outline future directions, particularly the broader need for out-of-distribution generalization of future NLP. 1

show abstract

Section: Introductionmentioning

confidence: 99%

“…We take inspiration of the data-centric and model-centric terms fromChu and Wang (2018) in MT, and add hybrid.3 We disregard methods which are task-specific (like leveraging a sentiment thesaurus).…”

mentioning

confidence: 99%

Neural Unsupervised Domain Adaptation in NLP—A Survey

Ramponi¹,

Plank²

2020

Proceedings of the 28th International Conference on Computational Linguistics

135

108

View full text Add to dashboard Cite

show abstract

“…domain transfer (Shimodaira, 2000;Subbaswamy and Saria, 2020). Accordingly, substantial effort has been devoted to developing computational methods for domain adaptation (Imran et al, 2016;Chu and Wang, 2018). Outcomes from this work often provide a solid foundation for use across multiple natural language processing tasks (Daume III and Marcu, 2006).…”

Section: Introductionmentioning

confidence: 99%

Do Models of Mental Health Based on Social Media Data Generalize?

Harrigian¹,

Aguirre²,

Dredze³

2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Proxy-based methods for annotating mental health status in social media have grown popular in computational research due to their ability to gather large training samples. However, an emerging body of literature has raised new concerns regarding the validity of these types of methods for use in clinical applications. To further understand the robustness of distantly supervised mental health models, we explore the generalization ability of machine learning classifiers trained to detect depression in individuals across multiple social media platforms. Our experiments not only reveal that substantial loss occurs when transferring between platforms, but also that there exist several unreliable confounding factors that may enable researchers to overestimate classification performance. Based on these results, we enumerate recommendations for future mental health dataset construction.

show abstract

“…During training, we evaluate the performance of the model after every epoch using a development set from the Biomedical domain. Our system is prone to over-fitting as the Biomedical (2014 and2018) training data sets that we use are significantly smaller (see Table 1) as compared to News. Generally over-fitting means that the model performs excellent on the training data, but worse on (Koehn, 2017) any other unseen data.…”

Section: Transfer Learningmentioning

confidence: 99%

“…Data-driven machine translation models assume the training data and test data have the same distribution and feature space (Koehn, 2009), which is rare in real-world applications (Olive et al, 2011). In statistical machine translation, a standard solution is to apply domain adaptation (Xu et al, 2007;Foster and Kuhn, 2007;Chu and Wang, 2018). For example, interpolating phrase or word probabilities in a sentence learned on in-domain and outof-domain data and then computing their product.…”

Section: Introductionmentioning

confidence: 99%

Hunter NMT System for WMT18 Biomedical Translation Task: Transfer Learning in Neural Machine Translation

Khan¹,

Panda²,

Xu³

et al. 2018

Proceedings of the Third Conference on Machine Translation: Shared Task Papers

View full text Add to dashboard Cite

This paper describes the submission of Hunter Neural Machine Translation (NMT) to the WMT'18 Biomedical translation task from English to French. The discrepancy between training and test data distribution brings a challenge to translate text in new domains. Beyond the previous work of combining in-domain with out-of-domain models, we found accuracy and efficiency gain in combining different in-domain models. We conduct extensive experiments on NMT with transfer learning. We train on different in-domain Biomedical datasets one after another. That means parameters of the previous training serve as the initialization of the next one. Together with a pre-trained out-of-domain News model, we enhanced translation quality with 3.73 BLEU points over the baseline. Furthermore, we applied ensemble learning on training models of intermediate epochs and achieved an improvement of 4.02 BLEU points over the baseline. Overall, our system is 11.29 BLEU points above the best system of last year on the EDP 2017 test set.

show abstract

A Survey of Domain Adaptation for Machine Translation

Cited by 115 publications

References 50 publications

Neural Unsupervised Domain Adaptation in NLP—A Survey

Neural Unsupervised Domain Adaptation in NLP—A Survey

Do Models of Mental Health Based on Social Media Data Generalize?

Hunter NMT System for WMT18 Biomedical Translation Task: Transfer Learning in Neural Machine Translation

Contact Info

Product

Resources

About