Selecting Artificially-Generated Sentences for Fine-Tuning Neural Machine Translation

Poncelas, Alberto; Way, Andy

doi:10.18653/v1/w19-8629

Cited by 10 publications

(7 citation statements)

References 17 publications

(14 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, we are interested in further exploring the algorithms explained in this work using NMT, using different configurations or artificial datasets (Poncelas, de Buy Wenniger, and Way 2019a;Poncelas and Way 2019;Soto et al 2020). Even if NMT systems work better with big amounts of data, data selection algorithms are useful to perform the so called "fine-tuning" (Luong and Manning 2015;Freitag and Al-Onaizan 2016), where pre-built system are improved with a small portion of in-domain data.…”

Section: Discussionmentioning

confidence: 99%

Improved feature decay algorithms for statistical machine translation

2020

Self Cite

View full text Add to dashboard Cite

In machine-learning applications, data selection is of crucial importance if good runtime performance is to be achieved. In a scenario where the test set is accessible when the model is being built, training instances can be selected so they are the most relevant for the test set. Feature Decay Algorithms (FDA) are a technique for data selection that has demonstrated excellent performance in a number of tasks. This method maximizes the diversity of the n-grams in the training set by devaluing those ones that have already been included. We focus on this method to undertake deeper research on how to select better training data instances. We give an overview of FDA and propose improvements in terms of speed and quality. Using German-to-English parallel data, first we create a novel approach that decreases the execution time of FDA when multiple computation units are available. In addition, we obtain improvements on translation quality by extending FDA using information from the parallel corpus that is generally ignored.

show abstract

Section: Discussionmentioning

confidence: 99%

Improved feature decay algorithms for statistical machine translation

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…These target sentences are then back-translated. The syntheticsource sentence pairs are typically used directly for fine-tuning the model, but can also be used as candidates for a domain-specific data selection scheme (Poncelas & Way, 2019).…”

Section: Back Translationmentioning

confidence: 99%

Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey

Saunders¹

2022

jair

View full text Add to dashboard Cite

The development of deep learning techniques has allowed Neural Machine Translation (NMT) models to become extremely powerful, given sufficient training data and training time. However, systems struggle when translating text from a new domain with a distinct style or vocabulary. Fine-tuning on in-domain data allows good domain adaptation, but requires sufficient relevant bilingual data. Even if this is available, simple fine-tuning can cause overfitting to new data and catastrophic forgetting of previously learned behaviour. We survey approaches to domain adaptation for NMT, particularly where a system may need to translate across multiple domains. We divide techniques into those revolving around data selection or generation, model architecture, parameter adaptation procedure, and inference procedure. We finally highlight the benefits of domain adaptation and multidomain adaptation techniques to other lines of NMT research.

show abstract

“…We trained the system with GSW_NORM-DE and specialised it with GSW_NORM-DE_PE (as suggested in Sennrich and Zhang, 2019). The purpose of this approach is to use a larger corpus with low quality segments for training to increase vocabulary coverage (Poncelas and Way, 2019) and then to specialise with high quality segments to eliminate noise.…”

Section: Systemsmentioning

confidence: 99%

Producing Standard German Subtitles for Swiss German TV Content

Gerlach,

Mutal,

Pierrette

2022

Ninth Workshop on Speech and Language Processing for Assistive Technologies (SLPAT-2022)

View full text Add to dashboard Cite

In this study we compare two approaches (neural machine translation and edit-based) and the use of synthetic data for the task of translating normalised Swiss German ASR output into correct written Standard German for subtitles, with a special focus on syntactic divergences. Results suggest that NMT is better suited to this task and that relatively simple rule-based generation of synthetic data could be a valuable approach for cases where little training data is available and transformations are simple.

show abstract

Selecting Artificially-Generated Sentences for Fine-Tuning Neural Machine Translation

Cited by 10 publications

References 17 publications

Improved feature decay algorithms for statistical machine translation

Improved feature decay algorithms for statistical machine translation

Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey

Producing Standard German Subtitles for Swiss German TV Content

Contact Info

Product

Resources

About