Dynamic Tuning and Weighting of Meta-learning for NMT Domain Adaptation

Song, Ziyue; Ma, Zhiyuan; Qi, Kaiyue; Liu, Gongshen

doi:10.1007/978-3-030-86383-8_46

Cited by 1 publication

(1 citation statement)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They also note that meta-training all Transformer parameters instead of an adapter leads to catastrophic forgetting of the original domain. Song et al (2021) also meta-learn adapter layers, but adjust meta-learning learning rate α dynamically to make the process sensitive to domain differences. They focus on domain differences in terms of model confidence in modelling a particular domain, and how representative each sentence pair is of a particular domain.…”

Section: Meta-learningmentioning

confidence: 99%

Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey

Saunders¹

2022

jair

View full text Add to dashboard Cite

The development of deep learning techniques has allowed Neural Machine Translation (NMT) models to become extremely powerful, given sufficient training data and training time. However, systems struggle when translating text from a new domain with a distinct style or vocabulary. Fine-tuning on in-domain data allows good domain adaptation, but requires sufficient relevant bilingual data. Even if this is available, simple fine-tuning can cause overfitting to new data and catastrophic forgetting of previously learned behaviour. We survey approaches to domain adaptation for NMT, particularly where a system may need to translate across multiple domains. We divide techniques into those revolving around data selection or generation, model architecture, parameter adaptation procedure, and inference procedure. We finally highlight the benefits of domain adaptation and multidomain adaptation techniques to other lines of NMT research.

show abstract

Section: Meta-learningmentioning

confidence: 99%

Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey

Saunders¹

2022

jair

View full text Add to dashboard Cite

show abstract

Dynamic Tuning and Weighting of Meta-learning for NMT Domain Adaptation

Cited by 1 publication

References 12 publications

Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey

Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey

Contact Info

Product

Resources

About