Empirical Evaluation of Active Learning Techniques for Neural MT

Zeng, Xiangkai; Garg, Sarthak; Chatterjee, Rajen; Nallasamy, Udhyakumar; Paulik, Matthias

doi:10.18653/v1/d19-6110

Cited by 9 publications

(21 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The key idea is to determine sentences in U S for which the model is relatively weaker. Round-trip-translationlikelihood (RTTL) (Zeng et al, 2019;Haffari et al, 2009) is the current SOTA approach for modeldriven strategies. It gives higher score to sentences for which, the model is unsure during backtranslation.…”

Section: Background: Active Learning In Nmtmentioning

confidence: 99%

“…These methods operated and evaluated using phrase-based machine translation systems. Recent works combined AL strategies with the transformer architecture (Vaswani et al, 2017) for NMT systems (Zeng et al, 2019;. Interactive NMT includes user feedback to update the NMT model in real-time (Peris and Casacuberta, 2018;Domingo et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

“…One way to save costs is by employing Active Learning (AL) strategies with NMT (Zeng et al, 2019;Ambati, 2012;Haffari et al, 2009). The goal of AL is to maximise translation quality for an annotation budget of labelling B sentences.…”

Section: Introductionmentioning

confidence: 99%

“…We label only the most informative B sentences in the whole unlabelled dataset using selection strategies. Previous works on AL (Zeng et al, 2019;Peris and Casacuberta, 2018) consider annotation budgets between hundred thousand to million sentences. But, it is not always possible to afford the annotation such amounts of data for low-resource languages.…”

Section: Introductionmentioning

confidence: 99%

“…We consider budgets that can annotate between 0 to 50k sentence pairs as low-annotation budgets2 We follow the terminology inZeng et al (2019) …”

mentioning

confidence: 99%

See 4 more Smart Citations

Cost-Effective Training in Low-Resource Neural Machine Translation

Koneru¹,

Liu²,

Niehues³

2022

Preprint

View full text Add to dashboard Cite

While Active Learning (AL) techniques are explored in Neural Machine Translation (NMT), only a few works focus on tackling low annotation budgets where a limited number of sentences can get translated. Such situations are especially challenging and can occur for endangered languages with few human annotators or having cost constraints to label large amounts of data. Although AL is shown to be helpful with large budgets, it is not enough to build high-quality translation systems in these low-resource conditions. In this work, we propose a cost-effective training procedure to increase the performance of NMT models utilizing a small number of annotated sentences and dictionary entries. Our method leverages monolingual data with self-supervised objectives and a small-scale, inexpensive dictionary for additional supervision to initialize the NMT model before applying AL. We show that improving the model using a combination of these knowledge sources is essential to exploit AL strategies and increase gains in lowresource conditions. We also present a novel AL strategy inspired by domain adaptation for NMT and show that it is effective for low budgets. We propose a new hybrid data-driven approach, which samples sentences that are diverse from the labelled data and also most similar to unlabelled data. Finally, we show that initializing the NMT model and further using our AL strategy can achieve gains of up to 13 BLEU compared to conventional AL methods.

show abstract

Section: Background: Active Learning In Nmtmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…We consider budgets that can annotate between 0 to 50k sentence pairs as low-annotation budgets2 We follow the terminology inZeng et al (2019) …”

mentioning

confidence: 99%

See 3 more Smart Citations

Cost-Effective Training in Low-Resource Neural Machine Translation

Koneru¹,

Liu²,

Niehues³

2022

Preprint

View full text Add to dashboard Cite

show abstract

A Survey on Deep Active Learning: Recent Advances and New Frontiers

Li,

Wang,

Chen

et al. 2024

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Active learning seeks to achieve strong performance with fewer training samples. It does this by iteratively asking an oracle to label newly selected samples in a human-in-the-loop manner. This technique has gained increasing popularity due to its broad applicability, yet its survey papers, especially for deep active learning (DAL), remain scarce. Therefore, we conduct an advanced and comprehensive survey on DAL. We first introduce reviewed paper collection and filtering. Second, we formally define the DAL task and summarize the most influential baselines and widely used datasets. Third, we systematically provide a taxonomy of DAL methods from five perspectives, including annotation types, query strategies, deep model architectures, learning paradigms, and training processes, and objectively analyze their strengths and weaknesses. Then, we comprehensively summarize the main applications of DAL in natural language processing (NLP), computer vision (CV), data mining (DM), and so on. Finally, we discuss challenges and perspectives after a detailed analysis of current studies. This work aims to serve as a useful and quick guide for researchers in overcoming difficulties in DAL. We hope that this survey will spur further progress in this burgeoning field.

show abstract

Onception: Active Learning with Expert Advice for Real World Machine Translation

Mendonça

Rei

Coheur

et al. 2023

Computational Linguistics

View full text Add to dashboard Cite

Active learning can play an important role in low-resource settings (i.e., where annotated data is scarce), by selecting which instances may be more worthy to annotate. Most active learning approaches for Machine Translation assume the existence of a pool of sentences in a source language, and rely on human annotators to provide translations or post-edits, which can still be costly. In this article, we apply active learning to a real world human-in-the-loop scenario in which we assume that: (1) the source sentences may not be readily available, but instead arrive in a stream; (2) the automatic translations receive feedback in the form of a rating, instead of a correct/edited translation, since the human-in-the-loop might be a user looking for a translation, but not be able to provide one. To tackle the challenge of deciding whether each incoming pair source–translations is worthy to query for human feedback, we resort to a number of stream-based active learning query strategies. Moreover, since we do not know in advance which query strategy will be the most adequate for a certain language pair and set of Machine Translation models, we propose to dynamically combine multiple strategies using prediction with expert advice. Our experiments on different language pairs and feedback settings show that using active learning allows to converge to the best Machine Translation systems with fewer human interactions. Furthermore, combining multiple strategies using prediction with expert advice outperforms several individual active learning strategies with even fewer interactions, particularly in partial feedback settings.

show abstract

Empirical Evaluation of Active Learning Techniques for Neural MT

Cited by 9 publications

References 22 publications

Cost-Effective Training in Low-Resource Neural Machine Translation

Cost-Effective Training in Low-Resource Neural Machine Translation

A Survey on Deep Active Learning: Recent Advances and New Frontiers

Onception: Active Learning with Expert Advice for Real World Machine Translation

Contact Info

Product

Resources

About