Making Sense of Word Embeddings

Abstract

We present a simple yet effective approach for learning word sense embeddings. In contrast to existing techniques, which either directly learn sense representations from corpora or rely on sense inventories from lexical resources, our approach can induce a sense inventory from existing word embeddings via clustering of ego-networks of related words. An integrated WSD mechanism enables labeling of words in context with learned sense vectors, which gives rise to downstream applications. Experiments show that the performance of our method is comparable to state-of-the-art unsupervised WSD systems.

Improving Word Sense Disambiguation in Neural Machine Translation with Sense Embeddings

“…contrastive: Il a la rétraction facile mais c'est parce qu'il débute. (Pelevina et al, 2016), which has been shown to perform as good as stat-of-theart unsupervised WSD systems. The method to learn the sense embeddings using SenseGram consists of four steps that we briefly summarise here.…”

Unsupervised Does Not Mean Uninterpretable: The Case for Word Sense Induction and Disambiguation

“…These vectors of each instance are then clustered. Multi-prototype extensions of the skipgram model (Mikolov et al, 2013) that use no predefined sense inventory learn one embedding word vector per one word sense and are commonly fitted with a disambiguation mechanism (Huang et al, 2012;Tian et al, 2014;Neelakantan et al, 2014;Bartunov et al, 2016;Li and Jurafsky, 2015;Pelevina et al, 2016). Comparisons of the AdaGram (Bartunov et al, 2016) to (Neelakantan et al, 2014) on three SemEval word sense induction and disambiguation datasets show the advantage of the former.…”

“…For this reason, we use AdaGram as a representative of the state-of-the-art word sense embeddings in our experiments. In addition, we compare to SenseGram, an alternative sense embedding based approach by Pelevina et al (2016). What makes the comparison to the later method interesting is that this approach is similar to ours, but instead of sparse representations the authors rely on word embeddings, making their approach less interpretable.…”

“…Table 3 presents results of evaluation of the best configuration of our approach trained on the ukWaC corpus. We compare our approach to four SemEval participants and two state-of-the-art systems based on word sense embeddings: AdaGram (Bartunov et al, 2016) based on Bayesian stick-breaking process 5 and SenseGram (Pelevina et al, 2016) based on clustering of ego-network 5 https://github.com/sbos/AdaGram.jl generated using word embeddings 6 . The AI-KU system (Baskaya et al, 2013) directly clusters test contexts using the k-means algorithm based on lexical substitution features.…”

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations

“…AdaGram (Bartunov et al, 2016) is a system that learns sense embeddings using a Bayesian extension of the Skip-gram model and provides WSD functionality based on the induced sense inventory. SenseGram (Pelevina et al, 2016) is a system that transforms word embeddings to sense embeddings via graph clustering and uses them for WSD. Other methods to learn sense embeddings were proposed, but these do not feature open implementations for WSD.…”