Visualizing Neural Machine Translation Attention and Confidence

Rikters, Matīss; Fishel, Mark; Bojar, Ondřej

doi:10.1515/pralin-2017-0037

Cited by 20 publications

(14 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Estimating the confidence or quality of the output of MT systems (Ueffing and Ney, 2007;Specia et al, 2009;Bach et al, 2011;Salehi et al, 2014;Rikters and Fishel, 2017;Kepler et al, 2019) is important for enabling downstream applications such as post-editing and interactive MT to better cope with translation mistakes. While existing methods rely on external models to estimate confidence, our approach leverages model uncertainty to derive confidence measures.…”

Section: Confidence Estimationmentioning

confidence: 99%

Improving Back-Translation with Uncertainty-based Confidence Estimation

Wang¹,

Liu²,

Wang³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

While back-translation is simple and effective in exploiting abundant monolingual corpora to improve low-resource neural machine translation (NMT), the synthetic bilingual corpora generated by NMT models trained on limited authentic bilingual data are inevitably noisy. In this work, we propose to quantify the confidence of NMT model predictions based on model uncertainty. With word-and sentence-level confidence measures based on uncertainty, it is possible for back-translation to better cope with noise in synthetic bilingual corpora. Experiments on Chinese-English and English-German translation tasks show that uncertainty-based confidence estimation significantly improves the performance of backtranslation. 1

show abstract

Section: Confidence Estimationmentioning

confidence: 99%

Improving Back-Translation with Uncertainty-based Confidence Estimation

Wang¹,

Liu²,

Wang³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

show abstract

“…Other NN-based algorithms explore internal information from neural models as an indicator of translation quality. They rely on the entropy of attention weights in RNN-based NMT systems [23,35]. However, attention-based indicators perform competitively only when combined with other QE features in a supervised framework.…”

Section: Related Workmentioning

confidence: 99%

An Oblivious Approach to Machine Translation Quality Estimation

Elmakias

Vilenchik

2021

Mathematics

View full text Add to dashboard Cite

Machine translation (MT) is being used by millions of people daily, and therefore evaluating the quality of such systems is an important task. While human expert evaluation of MT output remains the most accurate method, it is not scalable by any means. Automatic procedures that perform the task of Machine Translation Quality Estimation (MT-QE) are typically trained on a large corpus of source–target sentence pairs, which are labeled with human judgment scores. Furthermore, the test set is typically drawn from the same distribution as the train. However, recently, interest in low-resource and unsupervised MT-QE has gained momentum. In this paper, we define and study a further restriction of the unsupervised MT-QE setting that we call oblivious MT-QE. Besides having no access no human judgment scores, the algorithm has no access to the test text’s distribution. We propose an oblivious MT-QE system based on a new notion of sentence cohesiveness that we introduce. We tested our system on standard competition datasets for various language pairs. In all cases, the performance of our system was comparable to the performance of the non-oblivious baseline system provided by the competition organizers. Our results suggest that reasonable MT-QE can be carried out even in the restrictive oblivious setting.

show abstract

“…Stejně jako v městském autobuse či tramvaji. For inspecting the NMT attention alignments, we developed a tool (Rikters et al, 2017a) that takes data produced by Neural Monkey as input and produces a soft alignment visualization by connecting words and subword units (Sennrich et al, 2016b) as shown in Figure 5, which shows an example translation with two systems for En → Cs. Here it is clear that in the baseline alignment no attention goes to the word "městě" or the subword units "autobu@@" and "se" when translating "city".…”

Section: Referencementioning

confidence: 99%

“…To outperform the baselines, we explored 4 areas for improvements -1) filtering backtranslated data; 2) named entity forcing; 3) hybrid system combination; and 4) NMTspecific post-processing. The section is based on the paper of Rikters et al (2017a).…”

Section: Simple System Combination Using Neural Network Attentionmentioning

confidence: 99%

Hybrid Machine Translation by Combining Output from Multiple Machine Translation Systems

Rikters¹

2019

BJMC

Self Cite

View full text Add to dashboard Cite

This paper aims to combine output from various machine translation (MT) systems so that the overall translation quality of the source text would increase. Applicability of the developed methods for small, morphologically rich and under-resourced languages is evaluated, especially Latvian and Estonian. Existing methods have been analysed, and several combinations of methods have been proposed. The proposed methods have been implemented and evaluated using automatic and human evaluation. During this research novel methods have been created that structure source language sentences into linguistically motivated fragments and combine them using a character level neural language model; combine neural machine translation output by employing sourcetranslation attention alignments; use a multi-pass approach to produce additional incrementally improving training data. The key results of this research are new state-of-the-art machine translation systems for English ↔ Estonian; approaches for utilising neural MT generated attention alignments for MT combination and comprehension of resulting translations; MT combination systems for combining output from English → Latvian statistical MT. A practical application of the methods is implemented and described.

show abstract

Visualizing Neural Machine Translation Attention and Confidence

Cited by 20 publications

References 10 publications

Improving Back-Translation with Uncertainty-based Confidence Estimation

Improving Back-Translation with Uncertainty-based Confidence Estimation

An Oblivious Approach to Machine Translation Quality Estimation

Hybrid Machine Translation by Combining Output from Multiple Machine Translation Systems

Contact Info

Product

Resources

About