Discriminating between Lexico-Semantic Relations with the Specialization Tensor Model

Abstract

We present a simple and effective feedforward neural architecture for discriminating between lexico-semantic relations (synonymy, antonymy, hypernymy, and meronymy). Our Specialization Tensor Model (STM) simultaneously produces multiple different specializations of input distributional word vectors, tailored for predicting lexico-semantic relations for word pairs. STM outperforms more complex state-of-the-art architectures on two benchmark datasets and exhibits stable performance across languages. We also show that, if coupled with a lingual distributional space, the proposed model can transfer the prediction of lexico-semantic relations to a resource-lean target language without any training data.

Combining Discourse Markers and Cross-lingual Embeddings for Synonym–Antonym Classification

“…The 42% remaining cases involve a number of challenging antonyms and synonyms, but also a few remarkably easy cases. The latter include mor-phologically marked antonyms (e.g., Richtigkeit 'correctness' vs. Unrichtigkeit 'incorrectness') and instances of synonyms that involve two identical words (e.g., Verwandtschaft/Verwandtschaft 'kinship'), which presumably is an artifact of the automatic translation step used in creating the German dataset (see Glavaš and Vulić (2018)). Both types of errors could be identified by simple heuristics but are incorrectly classified by the model.…”

“…Supervision for lexical classification tasks is not available in all languages. To overcome this difficulty, some approaches (Mrkšić et al, 2017;Glavaš and Vulić, 2018) combine resources for English and cross-lingual word embeddings to distinguish lexical relations in other languages. That is, they train and test one model across different languages.…”

“…The resulting cross-lingual embeddings can be used to share supervision for lexical classification tasks across languages, when annotated data is not available in one language. For instance, a model for distinguishing lexical relations such as hypernymy and meronymy can be transferred to other languages (Glavaš and Vulić, 2018). However, this kind of transfer, using only cross-lingual embeddings, misses useful monolingual information available in the target language.…”

Weakly Supervised Cross-lingual Semantic Relation Classification via Knowledge Distillation

“…There are broadly two strategies to solve this problem. One line of work (Glavaš and Vulić, 2018;Mrkšić et al, 2017) uses model transfer, where a single model is trained on data from a high-resource language, and is then ported to the target language using cross-lingual embeddings. In contrast, Roth and Upadhyay (2019) translate training data from English into a target language using a combination of unsupervised cross-lingual embeddings (Artetxe et al, 2018) and monolingual information from the target language.…”

SphereRE: Distinguishing Lexical Relations with Hyperspherical Relation Embeddings

“…Apart from Hearst patterns (Hearst, 1992), there are few high-quality textual patterns to recognize lexical relations other than hypernymy. Distributional approaches consider the global contexts of terms to predict lexical relations using word embeddings (Baroni et al, 2012;Glavas and Vulic, 2018). They are reported to outperform several path-based approaches, but can suffer from the "lexical memorization" problem (Levy et al, 2015;.…”