Findings of the WMT 2019 Shared Tasks on Quality Estimation

Fonseca, Erick Rocha; Yankovskaya, Lisa; Martins, André F. T.; Fishel, Mark; Federmann, Christian

doi:10.18653/v1/w19-5401

Cited by 61 publications

(60 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• biomedical translation (Bawden et al, 2019b) • automatic post-editing (Chatterjee et al, 2019) • metrics • quality estimation (Fonseca et al, 2019) • parallel corpus filtering • robustness (Li et al, 2019b) In the news translation task (Section 2), participants were asked to translate a shared test set, optionally restricting themselves to the provided training data ("constrained" condition). We 1 http://www.statmt.org/wmt19/ held 18 translation tasks this year, between English and each of Chinese, Czech (into Czech only), German, Finnish, Lithuanian, and Russian.…”

Section: Introductionmentioning

confidence: 99%

Findings of the 2019 Conference on Machine Translation (WMT19)

Barrault¹,

Bojar²,

Costa-jussà³

et al. 2019

Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1)

Self Cite

288

246

View full text Add to dashboard Cite

This paper presents the results of the premier shared task organized alongside the Conference on Machine Translation (WMT) 2019. Participants were asked to build machine translation systems for any of 18 language pairs, to be evaluated on a test set of news stories. The main metric for this task is human judgment of translation quality. The task was also opened up to additional test suites to probe specific aspects of translation.

show abstract

Section: Introductionmentioning

confidence: 99%

Findings of the 2019 Conference on Machine Translation (WMT19)

Barrault¹,

Bojar²,

Costa-jussà³

et al. 2019

Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1)

Self Cite

288

246

View full text Add to dashboard Cite

show abstract

“…Results for DAseg-newstest2019 We prepared scores for all language pairs described in 3.3 by using non-tuned models trained on seven language pairs and for De-En, En-Ru, Ru-En, Fi-En by using fine-tuned models. Results of this submission will be available (Fonseca et al, 2019).…”

Section: Resultsmentioning

confidence: 99%

Quality Estimation and Translation Metrics via Pre-trained Word and Sentence Embeddings

Yankovskaya¹,

Tättar²,

Fishel³

2019

Proceedings of the Fourth Conference on Machine Translation (Volume 3: Shared Task Papers, Day 2)

Self Cite

View full text Add to dashboard Cite

We propose the use of pre-trained embeddings as features of a regression model for sentencelevel quality estimation of machine translation. In our work we combine freely available BERT and LASER multilingual embeddings to train a neural-based regression model. In the second proposed method we use as an input features not only pre-trained embeddings, but also log probability of any machine translation (MT) system. Both methods are applied to several language pairs and are evaluated both as a classical quality estimation system (predicting the HTER score) as well as an MT metric (predicting human judgements of translation quality).

show abstract

“…For MT QE, we use English-German training and test data provided for the WMT19 QE Shared Task (Fonseca et al, 2019, Task 1), consisting of source sentences, automatic translations, and manually corrected reference translations. For the supervised estimation, we use a multilayer perceptron with a hidden layer of size 256, trained to estimate the HTER value using the mean-squared-error loss.…”

Section: Methodsmentioning

confidence: 99%

On the Language Neutrality of Pre-trained Multilingual Representations

Libovický¹,

Rosa²,

Fraser³

2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Multilingual contextual embeddings, such as multilingual BERT and XLM-RoBERTa, have proved useful for many multi-lingual tasks. Previous work probed the cross-linguality of the representations indirectly using zero-shot transfer learning on morphological and syntactic tasks. We instead investigate the languageneutrality of multilingual contextual embeddings directly and with respect to lexical semantics. Our results show that contextual embeddings are more language-neutral and, in general, more informative than aligned static word-type embeddings, which are explicitly trained for language neutrality. Contextual embeddings are still only moderately languageneutral by default, so we propose two simple methods for achieving stronger language neutrality: first, by unsupervised centering of the representation for each language and second, by fitting an explicit projection on small parallel data. Besides, we show how to reach stateof-the-art accuracy on language identification and match the performance of statistical methods for word alignment of parallel sentences without using parallel data.

show abstract

Findings of the WMT 2019 Shared Tasks on Quality Estimation

Cited by 61 publications

References 11 publications

Findings of the 2019 Conference on Machine Translation (WMT19)

Findings of the 2019 Conference on Machine Translation (WMT19)

Quality Estimation and Translation Metrics via Pre-trained Word and Sentence Embeddings

On the Language Neutrality of Pre-trained Multilingual Representations

Contact Info

Product

Resources

About