AutoSense Model for Word Sense Induction

Amplayo, Reinald Kim; Hwang, Seung Won; Song, Min

doi:10.1609/aaai.v33i01.33016212

Cited by 8 publications

(3 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The relevance of graphical approaches is proved by the fact that increasingly sophisticated graphical models dominated the state-of-the-art results of WSI up until recently (e.g. Lau, Cook & Baldwin 2013;Wang et al 2015;Komninos & Manandhar 2016;Amplayo, Hwang & Song 2019;cf. Amrami & Goldberg 2019).…”

Section: International and Hungarian Backgroundmentioning

confidence: 99%

A proof-of-concept meaning discrimination experiment to compile a word-in-context dataset for adjectives – A graph-based distributional approach

Héja

Ligeti-Nagy

2022

ALing

View full text Add to dashboard Cite

The Word-in-Context corpus, which forms part of the SuperGLUE benchmark dataset, focuses on a specific sense disambiguation task: it has to be decided whether two occurrences of a given target word in two different contexts convey the same meaning or not. Unfortunately, the WiC database exhibits a relatively low consistency in terms of inter-annotator agreement, which implies that the meaning discrimination task is not well defined even for humans. The present paper aims at tackling this problem through anchoring semantic information to observable surface data. For doing so, we have experimented with a graph-based distributional approach, where both sparse and dense adjectival vector representations served as input. According to our expectations the algorithm is able to anchor the semantic information to contextual data, and therefore it is able to provide clear and explicit criteria as to when the same meaning should be assigned to the occurrences. Moreover, since this method does not rely on any external knowledge base, it should be suitable for any low- or medium-resourced language.

show abstract

Section: International and Hungarian Backgroundmentioning

confidence: 99%

A proof-of-concept meaning discrimination experiment to compile a word-in-context dataset for adjectives – A graph-based distributional approach

Héja

Ligeti-Nagy

2022

ALing

View full text Add to dashboard Cite

show abstract

“…Here, the definition of context may vary from window-based context to latent topic-alike context. Afterwards, the resulting clusters are either used as senses directly (Kutuzov, 2018), or employed further to learn sense embeddings via Chinese Restaurant Process algorithm (Li and Jurafsky, 2015), AdaGram, a Bayesian extension of the Skip-Gram model (Bartunov et al, 2016), AutoSense, an extension of the LDA topic model (Amplayo et al, 2019), and other techniques.…”

Section: Related Workmentioning

confidence: 99%

Word Sense Disambiguation for 158 Languages using Word Embeddings Only

Logacheva,

Teslenko,

Shelmanov

et al. 2020

Preprint

View full text Add to dashboard Cite

Disambiguation of word senses in context is easy for humans, but is a major challenge for automatic approaches. Sophisticated supervised and knowledge-based models were developed to solve this task. However, (i) the inherent Zipfian distribution of supervised training instances for a given word and/or (ii) the quality of linguistic knowledge representations motivate the development of completely unsupervised and knowledge-free approaches to word sense disambiguation (WSD). They are particularly useful for under-resourced languages which do not have any resources for building either supervised and/or knowledge-based models. In this paper, we present a method that takes as input a standard pre-trained word embedding model and induces a fully-fledged word sense inventory, which can be used for disambiguation in context. We use this method to induce a collection of sense inventories for 158 languages on the basis of the original pre-trained fastText word embeddings by Grave et al. (2018), enabling WSD in these languages. Models and system are available online.

show abstract

“…For instance, [13] relies on the Hierarchical Dirichlet Process and [6] employs the Stick-breaking Process. In [2] a rather complicated custom graphical model is proposed which aims at solving the sense granularity problem. Graph Clustering methods like [21,9] first build a graph with nodes corresponding to words, and weighted edges representing semantic similarity strength or co-occurrence frequency.…”

Section: Related Workmentioning

confidence: 99%

Combining Neural Language Models for Word Sense Induction

Arefyev¹,

Sheludko²,

Aleksashina³

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Word sense induction (WSI) is the problem of grouping occurrences of an ambiguous word according to the expressed sense of this word. Recently a new approach to this task was proposed, which generates possible substitutes for the ambiguous word in a particular context using neural language models, and then clusters sparse bag-of-words vectors built from these substitutes. In this work, we apply this approach to the Russian language and improve it in two ways. First, we propose methods of combining left and right contexts, resulting in better substitutes generated. Second, instead of fixed number of clusters for all ambiguous words we propose a technique for selecting individual number of clusters for each word. Our approach established new state-of-the-art level, improving current best results of WSI for the Russian language on two RUSSE 2018 datasets by a large margin.

show abstract

AutoSense Model for Word Sense Induction

Cited by 8 publications

References 8 publications

A proof-of-concept meaning discrimination experiment to compile a word-in-context dataset for adjectives – A graph-based distributional approach

A proof-of-concept meaning discrimination experiment to compile a word-in-context dataset for adjectives – A graph-based distributional approach

Word Sense Disambiguation for 158 Languages using Word Embeddings Only

Combining Neural Language Models for Word Sense Induction

Contact Info

Product

Resources

About