Experiments with three approaches to recognizing lexical entailment

Abstract

Inference in natural language often involves recognizing lexical entailment (RLE); that is, identifying whether one word entails another. For example, buy entails own. Two general strategies for RLE have been proposed: One strategy is to manually construct an asymmetric similarity measure for context vectors (directional similarity) and another is to treat RLE as a problem of learning to recognize semantic relations using supervised machine learning techniques (relation classification). In this paper, we experiment with two recent state-of-the-art representatives of the two general strategies. The first approach is an asymmetric similarity measure (an instance of the directional similarity strategy), designed to capture the degree to which the contexts of a word, a, form a subset of the contexts of another word, b. The second approach (an instance of the relation classification strategy) represents a word pair, a : b, with a feature vector that is the concatenation of the context vectors of a and b, and then applies supervised learning to a training set of labeled feature vectors. Additionally, we introduce a third approach that is a new instance of the relation classification strategy. The third approach represents a word pair, a : b, with a feature vector in which the features are the differences in the similarities of a and b to a set of reference words. All three approaches use vector space models (VSMs) of semantics, based on word-context matrices. We perform an extensive evaluation of the three approaches using three different datasets. The proposed new approach (similarity differences) performs significantly better than the other two approaches on some datasets and there is no dataset for which it is significantly worse. Along the way, we address some of the concerns raised in past research, regarding the treatment of RLE as a problem of semantic relation classification, and we suggest it is beneficial to make connections between the research in lexical entailment and the research in semantic relation classification.

HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment

“…This is different from the definition used in (Zhitomirsky- Geffet and Dagan 2009;Kotlerman et al 2010;Turney and Mohammad 2015) as substitutable lexical entailment: this relation holds for a pair of words (X, Y ) if a possible meaning of one word (i.e., X) entails a meaning of the other, and the entailing word can substitute the entailed one in some typical contexts. This definition is looser and more general than the TYPE-OF definition, as it also encompasses other lexical relations such as synonymy, metonymy, meronymy, etc.…”

“…Supervised models, on the other hand, attempt to learn the asymmetric operator from a training set, differing mostly in the feature selection to represent each candidate pair of words (Baroni et al 2012;Fu et al 2014;Rimell 2014;Weeds et al 2014;Roller, Erk, and Boleda 2014;Fu et al 2015;Shwartz, Goldberg, and Dagan 2016;Roller and Erk 2016). 8 An overview of the supervised techniques also discussing their main shortcomings is provided by , while a thorough discussion of differences between unsupervised and supervised entailment models is provided by Turney and Mohammad (2015).…”

“…For instance,Turney and Mohammad (2015) argue that in the sentences Jane dropped the glass and Jane dropped something fragile, the concept glass should entail fragile. 4 Other variants of the same definition replace TYPE-OF with KIND-OF or INSTANCE-OF.…”

Detecting Asymmetric Semantic Relations in Context: A Case-Study on Hypernymy Detection

“…For instance, the substitutional definition for entailment by Zhitomirsky-Geffet and Dagan (2009) asks the reader to think of a natural sentence that provides the missing context to the two words being considered, thus constraining the possible senses of the two words. On the other hand, Turney and Mohammad (2013) propose a relational definition, inviting the reader to imagine a semantic relation that connects the two words and constrains their possible senses. In contrast, we propose to detect hypernymy between word meanings described by specific contexts.…”

Exploring Vector Spaces for Semantic Relations

“…The second one tries to infer taxonomical relations, i.e. hypernymy, or lexical entailment, in supervised experiments (Weeds et al, 2014;Turney and Mohammad, 2014). For the hypernymy detection task, relation directionality can be captured by the inclusion of the hyponym's context in the broader term (Kotlerman et al, 2010), as well as by measuring the informativeness of their contexts (Santus et al, 2014).…”