Adaptive Quality Estimation for Machine Translation

Turchi, Marco; Anastasopoulos, Antonios; Souza, José G. C. de; Negri, Matteo

doi:10.3115/v1/p14-1067

Cited by 15 publications

(7 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…in seconds) or effort (e.g. number of editing operations) required to correct machine-translated sentences into publishable translations (Specia et al, 2009;Mehdad et al, 2012;Turchi et al, 2014a;C. de Souza et al, 2015).…”

Section: Introductionmentioning

confidence: 99%

MT Quality Estimation for Computer-assisted Translation: Does it Really Help?

Turchi

Negri

Federico

2015

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Confere

Self Cite

View full text Add to dashboard Cite

The usefulness of translation quality estimation (QE) to increase productivity in a computer-assisted translation (CAT) framework is a widely held assumption (Specia, 2011;Huang et al., 2014). So far, however, the validity of this assumption has not been yet demonstrated through sound evaluations in realistic settings. To this aim, we report on an evaluation involving professional translators operating with a CAT tool in controlled but natural conditions. Contrastive experiments are carried out by measuring post-editing time differences when: i) translation suggestions are presented together with binary quality estimates, and ii) the same suggestions are presented without quality indicators. Translators' productivity in the two conditions is analysed in a principled way, accounting for the main factors (e.g. differences in translators' behaviour, quality of the suggestions) that directly impact on time measurements. While the general assumption about the usefulness of QE is verified, significance testing results reveal that real productivity gains can be observed only under specific conditions.

show abstract

Section: Introductionmentioning

confidence: 99%

MT Quality Estimation for Computer-assisted Translation: Does it Really Help?

Turchi

Negri

Federico

2015

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Confere

Self Cite

View full text Add to dashboard Cite

show abstract

“…In our experiments, we study two metrics for measuring the simplicity of sentences. Note that in the field of quality estimation for MT (Specia et al, 2010;Fonseca et al, 2019), researchers have proposed several existing techniques to estimate the simplicity of sentences (Turchi et al, 2014;Shah et al, 2015;Kim and Lee, 2016;Kepler et al, 2019;Zhou et al, 2019;Hou et al, 2019), and here we select a few representative approaches.…”

Section: Simplicity Metricsmentioning

confidence: 99%

Dynamic Data Selection and Weighting for Iterative Back-Translation

Dou

Anastasopoulos

Neubig

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

Self Cite

View full text Add to dashboard Cite

Back-translation has proven to be an effective method to utilize monolingual data in neural machine translation (NMT), and iteratively conducting back-translation can further improve the model performance. Selecting which monolingual data to back-translate is crucial, as we require that the resulting synthetic data are of high quality and reflect the target domain. To achieve these two goals, data selection and weighting strategies have been proposed, with a common practice being to select samples close to the target domain but also dissimilar to the average general-domain text. In this paper, we provide insights into this commonly used approach and generalize it to a dynamic curriculum learning strategy, which is applied to iterative back-translation models.In addition, we propose weighting strategies based on both the current quality of the sentence and its improvement over the previous iteration. We evaluate our models on domain adaptation, low-resource, and high-resource MT settings and on two language pairs. Experimental results demonstrate that our methods achieve improvements of up to 1.8 BLEU points over competitive baselines. 1

show abstract

“…The few previous approaches to develop QE models that are able to learn from a continuous stream of data suffer from the following limitations: they do not have an explicit objective that encourages the model to exploit common structures shared among different users to continually adapt efficiently for new users (Turchi et al, 2014), or assume a fixed number of users, and that the identity of each user is known in advance (de Souza et al, 2015). In addition, these previous approaches do not explicitly account for the underlying uncertainties in the data in order to improve performance.…”

Section: Introductionmentioning

confidence: 99%

Continual Quality Estimation with Online Bayesian Meta-Learning

Obamuyide¹,

Fomicheva²,

Specia³

2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Most current quality estimation (QE) models for machine translation are trained and evaluated in a static setting where training and test data are assumed to be from a fixed distribution. However, in real-life settings, the test data that a deployed QE model would be exposed to may differ from its training data. In particular, training samples are often labelled by one or a small set of annotators, whose perceptions of translation quality and needs may differ substantially from those of endusers, who will employ predictions in practice. To address this challenge, we propose an online Bayesian meta-learning framework for the continuous training of QE models that is able to adapt them to the needs of different users, while being robust to distributional shifts in training and test data. Experiments on data with varying number of users and language characteristics validate the effectiveness of the proposed approach.

show abstract

Adaptive Quality Estimation for Machine Translation

Cited by 15 publications

References 15 publications

MT Quality Estimation for Computer-assisted Translation: Does it Really Help?

MT Quality Estimation for Computer-assisted Translation: Does it Really Help?

Dynamic Data Selection and Weighting for Iterative Back-Translation

Continual Quality Estimation with Online Bayesian Meta-Learning

Contact Info

Product

Resources

About