Similarity-based estimation of word cooccurrence probabilities

Abstract

In many applications of natural language processing (NLP) it is necessary to determine the likelihood of a given word combination. For example, a speech recognizer may need to determine which of the two word combinations "eat a peach" and "eat a beach" is more likely. Statistical NLP methods determine the likelihood of a word combination from its frequency in a training corpus. However, the nature of language is such that many word combinations are infrequent and do not occur in any given corpus. In this work we propose a method for estimating the probability of such previously unseen word combinations using available information on "most similar" words.We describe probabilistic word association models based on distributional word similarity, and apply them to two tasks, language modeling and pseudo-word disambiguation. In the language modeling task, a similarity-based model is used to improve probability estimates for unseen bigrams in a back-off language model. The similarity-based method yields a 20% perplexity improvement in the prediction of unseen bigrams and statistically significant reductions in speech-recognition error.We also compare four similarity-based estimation methods against back-off and maximumlikelihood estimation methods on a pseudo-word sense disambiguation task in which we controlled for both unigram and bigram frequency to avoid giving too much weight to easy-todisambiguate high-frequency configurations. The similarity-based methods perform up to 40% better on this particular task.

Comparing and combining semantic verb classifications

“…Relying on such lexical acquisition methods, automatically induced features and relations can complement directly accessible ones. In a second step, it will often be necessary to bridge non-identical but related properties; this task can be approached using standard approaches to semantic similarity, such as distributional measures (Dagan et al, 1999;Curran, 2004;Weeds and Weir, 2005;Budanitsky and Hirst, 2006;Padó and Lapata, 2007). Once a set of (comparable) rich features is available, we assume that an automation of the verb class linking is relatively straightforward.…”

El análisis reticular de coincidencias

“…Por su lado, el análisis cualitativo comparado (Ragin 1987(Ragin , 2000Medina et al 2017) comparte los datos binarios como entrada, aunque use un procedimiento distinto de tratamiento de la información basado en la lógica de (Bool 2003). Finalmente, cabe referirse al llamado análisis de coocurrencias que se aplica básicamente en dos ámbitos: en el de las estructuras comunitarias de especies (Sanderson 2000, Griffith et al 2016) y en el del análisis de contenido basado en redes semánti-cas (Dagan et al 1999, Matsuo e Ishizuka 2002, que se centra en el número de veces que aparecen determinados vocablos en un conjunto determinado de unidades de texto.…”

Untitled

“…We also thank the reviewers of this paper for their constructive criticisms, and the editors of the present issue, Claire Cardie and Ray Mooney, for their help and suggestions. Portions of this work have appeared previously (Dagan, Pereira, & Lee, 1994;Dagan, Lee, & Pereira, 1997); we thank the reviewers of those papers for their comments. Part of this work was done while the first author was a member of technical staff and then a visitor at AT&T Labs, and the second author was a graduate student at Harvard University and a summer visitor at AT&T Labs.…”