Word2vec Not Dead: Predicting Hypernyms of Co-Hyponyms Is Better Than Reading Definitions

Abstract

Expert-built lexical resources are known to provide information of good quality for the cost of low coverage. This property limits their applicability in modern NLP applications. Building descriptions of lexical-semantic relations manually in sufficient volume requires a huge amount of qualified human labour. However, given some initial version of a taxonomy is already built, automatic or semi-automatic taxonomy enrichment systems can greatly reduce the required efforts. We propose and experiment with two approaches to taxonomy enrichment, one utilizing information from word definitions and another from word usages, and also a combination of them. The first method retrieves co-hyponyms for the target word from distributional semantic models (word2vec) or language models (XLM-R), then looks for hypernyms of co-hyponyms in the taxonomy. The second method tries to extract hypernyms directly from Wiktionary definitions. The proposed methods were evaluated on the Dialogue-2020 shared task on taxonomy enrichment. We found that predicting hypernyms of cohyponyms achieves better results in this task. The combination of both methods improves results further and is among 3 best-performing systems for verbs. An important part of the work is detailed qualitative and error analysis of the proposed methods, which provide interesting observations of their behaviour and ideas for the future work.

Taxonomy enrichment with text and graph vector representations

“…assigned one or more hypernyms [37] or ranked all hypernyms by suitability for a particular word [20]. They also used a range of additional resources, such as Wiktionary, dictionaries, additional corpora [2]. Interestingly, only one of the well-performing models [78] used context-informed embeddings (BERT [24]) or external tools such as online Machine Translation (MT) and search engines (the best-performing model denoted as Yuriy in the workshop description paper).…”

“…Another option is to train word2vec embeddings from scratch and cast the task as a classification problem [37]. Some participants compare the approach based on XLM-R model [19] with the word2vec "hypernyms of co-hyponyms" method [2]. It considers nearest neighbours as co-hyponyms and takes their hypernyms as candidate synsets.…”

“…We release all our code and datasets to enable further research in this direction. 2 The rest of the paper is organised as follows. In Section 2 we formulate the task and provide the relevant definitions.…”

Studying Taxonomy Enrichment on Diachronic WordNet Versions

“…They cast the task as a classification problem where words need to be assigned one or more hypernyms (Kunilovskaya et al, 2020) or ranked all hypernyms by suitability for a particular word (Dale, 2020). They also used a range of additional resources, such as Wiktionary (Arefyev et al, 2020), dictionaries, additional corpora. Interestingly, only one of the well-performing models (Tikhomirov et al, 2020) used context-informed embeddings (BERT).…”

RUSSE'2020: Findings of the First Taxonomy Enrichment Task for the Russian language

“…MorphoBabushka (alvadia, maxfed, joystick) [2] This team used the following pipeline. First, they retrieved nearest neighbors for the target word from word2vec "SkipGram with Negative Sampling" model trained on Librusec book collection [1] and search for their direct and indirect hypernyms in RuWordNet.…”