Co-occurrence Retrieval: A Flexible Framework for Lexical Distributional Similarity

Abstract

Techniques that exploit knowledge of distributional similarity between words have been proposed in many areas of Natural Language Processing. For example, in language modeling, the sparse data problem can be alleviated by estimating the probabilities of unseen co-occurrences of events from the probabilities of seen co-occurrences of similar events. In other applications, distributional similarity is taken to be an approximation to semantic similarity. However, due to the wide range of potential applications and the lack of a strict definition of the concept of distributional similarity, many methods of calculating distributional similarity have been proposed or adopted.In this work, a flexible, parameterized framework for calculating distributional similarity is proposed. Within this framework, the problem of finding distributionally similar words is cast as one of co-occurrence retrieval (CR) for which precision and recall can be measured by analogy with the way they are measured in document retrieval. As will be shown, a number of popular existing measures of distributional similarity are simulated with parameter settings within the CR framework. In this article, the CR framework is then used to systematically investigate three fundamental questions concerning distributional similarity. First, is the relationship of lexical similarity necessarily symmetric, or are there advantages to be gained from considering it as an asymmetric relationship? Second, are some co-occurrences inherently more salient than others in the calculation of distributional similarity? Third, is it necessary to consider the difference in the extent to which each word occurs in each co-occurrence type?Two application-based tasks are used for evaluation: automatic thesaurus generation and pseudo-disambiguation. It is possible to achieve significantly better results on both these tasks by varying the parameters within the CR framework rather than using other existing distributional similarity measures; it will also be shown that any single unparameterized measure is unlikely to be able to do better on both tasks. This is due to an inherent asymmetry in lexical substitutability and therefore also in lexical distributional similarity.

Corpus domain effects on distributional semantic modeling of medical terms

“…Automated approaches for representing the semantic content of terms and similarity and relatedness between them have been widely used in a number of Natural Language Processing (NLP) applications in both general English (Budanitsky and Hirst, 2006;Landauer, 2006;Resnik, 1999;Weeds and Weir, 2005) and specialized terminological domains such as bioinformatics (Ferreira et al, 2013;Lord et al, 2003;Mazandu et al, 2016;Wang et al, 2007;Yang et al, 2012) and medicine (Garla and Brandt, 2012;Lee et al, 2008;Liu et al, 2012;Pakhomov et al, 2010;Pedersen et al, 2007;Sajadi, 2014). A subset of these methods, distributional semantics, relies on the co-occurrence information between words obtained from large corpora of text and makes the assumption that words with similar or related meanings tend to occur in similar contexts.…”

Looking for Hyponyms in Vector Space

“…Entailment and inference systems can improve sentence-level entailment resolution by detecting the presence and direction of wordlevel hyponymy relations. Distributionally similar words have been used for smoothing language models and word co-occurrence probabilities (Dagan et al, 1999;Weeds and Weir, 2005), and hyponyms can be more suitable for this application.…”

Comparing and combining semantic verb classifications

“…Relying on such lexical acquisition methods, automatically induced features and relations can complement directly accessible ones. In a second step, it will often be necessary to bridge non-identical but related properties; this task can be approached using standard approaches to semantic similarity, such as distributional measures (Dagan et al, 1999;Curran, 2004;Weeds and Weir, 2005;Budanitsky and Hirst, 2006;Padó and Lapata, 2007). Once a set of (comparable) rich features is available, we assume that an automation of the verb class linking is relatively straightforward.…”