Counter-fitting Word Vectors to Linguistic Constraints

Abstract

In this work, we present a novel counter-fitting method which injects antonymy and synonymy constraints into vector space representations in order to improve the vectors' capability for judging semantic similarity. Applying this method to publicly available pre-trained word vectors leads to a new state of the art performance on the SimLex-999 dataset. We also show how the method can be used to tailor the word vector space for the downstream task of dialogue state tracking, resulting in robust improvements across different dialogue domains.

Dual Tensor Model for Detecting Asymmetric Lexico-Semantic Relations

“…However, we often need embeddings to be similar only if an exact lexico-semantic relation holds between the words. Numerous methods for specializing word embeddings for particular relations have been proposed (Yu and Dredze, 2014;Faruqui et al, 2015;Kiela et al, 2015;Mrkšić et al, 2016, inter alia), primarily aiming to differentiate synonymic similarity from other types of semantic relatedness.…”

“…Having an explicit embedding specialization function alleviates the need to specialize the entire unspecialized embedding space at once, like existing models do (Faruqui et al, 2015;Mrkšić et al, 2016).…”

“…By introducing dedicated tensors we -unlike existing models, which directly propagate updates to unspecialized embeddings (Faruqui et al, 2015;Mrkšić et al, 2016) -explicitly learn the specialization function. With an explicit specialization function, we do not have to specialize the whole embedding space at once.…”

“…Consequently, a number of approaches have been proposed for specializing distributional spaces for specific lexico-semantic relations, either by (1) modifying the learning objective or regularization of the original embedding model by incorporating linguistic constraints (Yu and Dredze, 2014;Kiela et al, 2015) or (2) retroactively fitting the pre-trained unspecialized embeddings to linguistic constraints (Faruqui et al, 2015;Mrkšić et al, 2016). However, these methods specialize distributional vector spaces primarily for detecting the symmetric relation of semantic similarity (i.e., graded synonymy) and not for asymmetric lexico-semantic relations such as hypernymy and meronymy.…”

BIBI System Description: Building with CNNs and Breaking with Deep Reinforcement Learning

“…Although OOVs are a significant challenge, we believe they can be overcome by training better sentiment-sensitized word embeddings (Mrkšić et al, 2016), or combining the system with character-level normalization methods (Han and Baldwin, 2011).…”