Improving Word Sense Disambiguation in Neural Machine Translation
            with Sense Embeddings

Gonzales, Annette Rios; Mascarell, Laura; Sennrich, Rico

doi:10.18653/v1/w17-4702

Cited by 99 publications

(65 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While this entirely automatic setup could give rise to inconsistencies which would require manual correction as in Rios Gonzales et al (2017), we argue that BabelNet constraints already provide some filtering (for example mostly keeping number constant). Given our aim to scale up to a large number of languages, the need for human intervention would make the creation of a large scale multilingual benchmark difficult and costly.…”

Section: Methodsmentioning

confidence: 95%

“…However, all these test suites require significant amounts of expert knowledge and manual work for identifying the divergences and compiling the examples, which typically limits their coverage to a small number of language pairs and directions. For example, the test sets built by Rios Gonzales et al (2017) cover only 65 ambiguous words for two language pair directions.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The MuCoW Test Suite at WMT 2019: Automatically Harvested Multilingual Contrastive Word Sense Disambiguation Test Sets for Machine Translation

Raganato¹,

Scherrer²,

Tiedemann³

2019

Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1)

View full text Add to dashboard Cite

Supervised Neural Machine Translation (NMT) systems currently achieve impressive translation quality for many language pairs. One of the key features of a correct translation is the ability to perform word sense disambiguation (WSD), i.e., to translate an ambiguous word with its correct sense. Existing evaluation benchmarks on WSD capabilities of translation systems rely heavily on manual work and cover only few language pairs and few word types. We present MU-COW, a multilingual contrastive test suite that covers 16 language pairs with more than 200 000 contrastive sentence pairs, automatically built from word-aligned parallel corpora and the wide-coverage multilingual sense inventory of BabelNet. We evaluate the quality of the ambiguity lexicons and of the resulting test suite on all submissions from 9 language pairs presented in the WMT19 news shared translation task, plus on other 5 language pairs using pretrained NMT models. The MUCOW test suite is available at http://github. com/Helsinki-NLP/MuCoW.

show abstract

Section: Methodsmentioning

confidence: 95%

Section: Introductionmentioning

confidence: 99%

The MuCoW Test Suite at WMT 2019: Automatically Harvested Multilingual Contrastive Word Sense Disambiguation Test Sets for Machine Translation

Raganato¹,

Scherrer²,

Tiedemann³

2019

Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1)

View full text Add to dashboard Cite

show abstract

“…In this respect, a model with predefined fixed patterns may struggle to encode global semantic features. To this end, we evaluate our models on two German-English WSD test suites, ContraWSD (Rios Gonzales et al, 2017) and MuCoW (Raganato et al, 2019). 9 Table 6 shows the performance of our models on the WSD benchmarks.…”

Section: Word Sense Disambiguationmentioning

confidence: 99%

Fixed Encoder Self-Attention Patterns in Transformer-Based Machine Translation

Raganato

Scherrer

Tiedemann

2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Transformer-based models have brought a radical change to neural machine translation. A key feature of the Transformer architecture is the so-called multi-head attention mechanism, which allows the model to focus simultaneously on different parts of the input. However, recent works have shown that most attention heads learn simple, and often redundant, positional patterns. In this paper, we propose to replace all but one attention head of each encoder layer with simple fixed -non-learnable -attentive patterns that are solely based on position and do not require any external knowledge. Our experiments with different data sizes and multiple language pairs show that fixing the attention heads on the encoder side of the Transformer at training time does not impact the translation quality and even increases BLEU scores by up to 3 points in low-resource scenarios.

show abstract

“…mined by the structural features of the sentence. In particular, to commercialize the results of WSD, it will be necessary to address most words and their senses in a wide range of domains; these will range from information retrieval [41], [45], [49] or machine translation [4], [14], [30], [37] to even second language education [7], [50], using various sources such as movies and books.…”

Section: Introductionmentioning

confidence: 99%

Hybrid Sense Classification Method for Large-Scale Word Sense Disambiguation

2020

View full text Add to dashboard Cite

Word sense disambiguation (WSD) is a task of determining a reasonable sense of a word in a particular context. Although recent studies have demonstrated some progress in the advancement of neural language models, the scope of research is still such that the senses of several words can only be determined in a few domains. Therefore, it is necessary to move toward developing a highly scalable process that can address a lot of senses occurring in various domains. This paper introduces a new large WSD dataset that is automatically constructed from the Oxford Dictionary, which is widely used as a standard source for the meaning of words. We propose a new WSD model that individually determines the sense of the word in accordance with its part of speech in the context. In addition, we introduce a hybrid sense prediction method that separately classifies the less frequently used senses for achieving a reasonable performance. We have conducted comparative experiments to demonstrate that the proposed method is more reliable compared with the baseline approaches. Also, we investigated the adaptation of the method to a realistic environment with the use of news articles. INDEX TERMS Computational and artificial intelligence, English vocabulary learning, natural language processing, neural networks, word sense disambiguation. YOONSEOK HEO received the B.S. and M.S. degrees in computer science (major in in natural language generation) from Sogang Unversity. He is currently pursuing the Ph.D. degree with the Department of Computer Science, Sogang University. He worked as a Researcher with Gachon University, in 2018. He is interested in spoken dialogue system, machine translation, question answering, machine reading comprehension, and named entity recognition. His current research focuses on the way of exploiting multimodal resources for machine translation and addressing large-scale open domain texts for machine reading comprehension.

show abstract

Improving Word Sense Disambiguation in Neural Machine Translation with Sense Embeddings

Cited by 99 publications

References 18 publications

The MuCoW Test Suite at WMT 2019: Automatically Harvested Multilingual Contrastive Word Sense Disambiguation Test Sets for Machine Translation

The MuCoW Test Suite at WMT 2019: Automatically Harvested Multilingual Contrastive Word Sense Disambiguation Test Sets for Machine Translation

Fixed Encoder Self-Attention Patterns in Transformer-Based Machine Translation

Hybrid Sense Classification Method for Large-Scale Word Sense Disambiguation

Contact Info

Product

Resources

About