Deriving Boolean structures from distributional vectors

Abstract

Corpus-based distributional semantic models capture degrees of semantic relatedness among the words of very large vocabularies, but have problems with logical phenomena such as entailment, that are instead elegantly handled by model-theoretic approaches, which, in turn, do not scale up. We combine the advantages of the two views by inducing a mapping from distributional vectors of words (or sentences) into a Boolean structure of the kind in which natural language terms are assumed to denote. We evaluate this Boolean Distributional Semantic Model (BDSM) on recognizing entailment between words and sentences. The method achieves results comparable to a state-of-the-art SVM, degrades more gracefully when less training data are available and displays interesting qualitative properties.

A Vector Space for Distributional Semantics for Entailment

“…The semi-supervised model of Kruszewski et al (2015) also models entailment in a vector space, but they use a discrete vector space. They train a mapping from distributional semantic vectors to Boolean vectors such that feature inclusion respects a training set of entailment relations.…”

Relations such as Hypernymy: Identifying and Exploiting Hearst Patterns in Distributional Vectors for Lexical Entailment

“…Indeed, showed that two existing lexical entailment models fail to account for similarity between the antecedent and consequent, and conclude that such models are only learning to predict prototypicality: that is, they predict that cat entails animal because animal is usually entailed, and therefore will also predict that sofa entails animal. Yet it remains unclear why such models make for such strong baselines (Weeds et al, 2014;Kruszewski et al, 2015;.…”

“…Research on lexical entailment using distributional semantics has now spanned more than a decade, and has been approached using both unsupervised (Weeds et al, 2004;Kotlerman et al, 2010;Lenci and Benotto, 2012;Santus, 2013) and supervised techniques (Baroni et al, 2012;Fu et al, 2014;Roller et al, 2014;Weeds et al, 2014;Kruszewski et al, 2015;Turney and Mohammad, 2015;Santus et al, 2016). Most of the work in unsupervised methods is based on the Distributional Inclusion Hypothesis (Weeds et al, 2004;Zhitomirsky-Geffet and Dagan, 2005), which states that the contexts in which a hypernym appear should be a superset over its hyponyms' contexts.…”

“…That is, when the training and test sets were carefully constructed to ensure they were completely disjoint, it performed extremely poorly. Nonetheless, Concat is continually used as a strong baseline in more recent work (Kruszewski et al, 2015).…”

“…Recently, other works have begun to analyze Concat and Diff for their ability to go beyond just hypernymy detection. Vylomova et al (2016) take an extensive look at Diff's ability to model a wide variety of lexical relations and conclude it is generally robust, and Kruszewski et al (2015) have success with a neural network model based on the Distributional Inclusion Hypothesis.…”