Relations such as Hypernymy: Identifying and Exploiting Hearst Patterns in Distributional Vectors for Lexical Entailment

Abstract

We consider the task of predicting lexical entailment using distributional vectors. We perform a novel qualitative analysis of one existing model which was previously shown to only measure the prototypicality of word pairs. We find that the model strongly learns to identify hypernyms using Hearst patterns, which are well known to be predictive of lexical relations. We present a novel model which exploits this behavior as a method of feature extraction in an iterative procedure similar to Principal Component Analysis. Our model combines the extracted features with the strengths of other proposed models in the literature, and matches or outperforms prior work on multiple data sets.

How Well Can We Predict Hypernyms from Word Embeddings? A Dataset-Centric Analysis

“…Vylomova et al (2016) learned a range of semantic relations, including hypernymy, using the diff operator and achieved positive results. Roller and Erk (2016) showed that concat with a logistic regression classifier learns to extract Hearst patterns (such as, including, etc.) from distributional vectors.…”

“…This observation is relatively general and robust across several choices of datasets, models and embeddings. For example, Levy et al (2015) achieve up to 0.85 F1, while Roller and Erk (2016) achieve up to 0.90 F1. Both of these results are achieved on the Baroni dataset (Baroni et al, 2012).…”

“…For most other datasets, models achieve promising scores above 0.60 F1 points; e.g. Roller and Erk (2016) report 0.66 F1 points for a linear model on the balanced Turney dataset (Turney and Mohammad, 2015).…”

Learning Turkish Hypernymy Using Word Embeddings

“…Ref. 33 utilized the vector concatenation and difference on the task of detecting lexical entailment and showed that the study can be used to train a detector to identify several different kinds of Hearst's patterns. Ref.…”

SemEval-2018 Task 9: Hypernym Discovery

“…the capability to relate generic terms or classes to their specific instances, lies at the core of human cognition. It is not surprising, therefore, that identifying hypernymic (is-a) relations has been pursued in NLP for more than two decades (Shwartz et al, 2016): indeed, successfully identifying this lexical relation substantially improves Question Answering applications (Prager et al, 2008;Yahya et al, 2013), Textual Entailment and Semantic Search systems (Hoffart et al, 2014;Roller et al, 2014;Roller and Erk, 2016). In addition, hypernymic relations are the backbone of almost every ontology, semantic network and taxonomy (Yu et al, 2015), which are in turn useful resources for downstream tasks such as web retrieval, website navigation or records management (Bordea et al, 2015).…”