A model of lexical attraction and repulsion

Beeferman, Doug; Berger, Adam; Lafferty, John

doi:10.3115/979617.979665

Cited by 26 publications

(28 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…At the simplest level, if the algorithm is allowed to use a longer text segment around a seed word, a larger set of terms is likely to be measured. More interesting, in language usage, at least for English, the tendency is to avoid repeating a word in an adjacent sentence and to use a replacement term, such as a synonym (see, e.g., Beeferman et al, 1997). The relevancy scores of the words that are also seen in the one-sentence lists are, for the most part, higher in the three-sentence lists.…”

Section: Stabilitymentioning

confidence: 99%

“…In essence, a word can be defined by its context in usage. Beeferman and colleagues observed that words tend to correlate with other words over a certain range within the text stream (Beeferman, Berger, & Lafferty, 1997). Computational linguists have also exploited this aspect of language-for word sense disambiguation, as a particular example (Yarowsky, 1995).…”

Section: Foundations Of the Techniquementioning

confidence: 99%

See 1 more Smart Citation

Evaluation of unsupervised semantic mapping of natural language with Leximancer concept mapping

Smith

Humphreys

2006

Behavior Research Methods

940

763

View full text Add to dashboard Cite

The Leximancer system is a relatively new method for transforming lexical co-occurrence information from natural language into semantic patterns in an unsupervised manner. It employs two stages of co-occurrence information extraction-semantic and relational-using a different algorithm for each stage. The algorithms used are statistical, but they employ nonlinear dynamics and machine learning. This article is an attempt to validate the output of Leximancer, using a set of evaluation criteria taken from content analysis that are appropriate for knowledge discovery tasks.

show abstract

Section: Stabilitymentioning

confidence: 99%

Section: Foundations Of the Techniquementioning

confidence: 99%

Evaluation of unsupervised semantic mapping of natural language with Leximancer concept mapping

Smith

Humphreys

2006

Behavior Research Methods

940

763

View full text Add to dashboard Cite

show abstract

“…Not only can the Gaussian prior be applied to maximum entropy modeling, but it can also be applied in the more general minimum divergence paradigm [37,38]. Maximizing entropy is equivalent to finding the model with the smallest Kullback-Leibler divergence from the uniform distribution.…”

Section: Discussionmentioning

confidence: 99%

A Gaussian Prior for Smoothing Maximum Entropy Models

Chen¹,

Rosenfeld²

1999

229

140

View full text Add to dashboard Cite

In certain contexts, maximum entropy (ME) modeling can be viewed as maximum likelihood training for exponential models, and like other maximum likelihood methods is prone to overfitting of training data. Several smoothing methods for maximum entropy models have been proposed to address this problem, but previous results do not make it clear how these smoothing methods compare with smoothing methods for other types of related models. In this work, we survey previous work in maximum entropy smoothing and compare the performance of several of these algorithms with conventional techniques for smoothing n-gram language models. Because of the mature body of research in n-gram model smoothing and the close connection between maximum entropy and conventional n-gram models, this domain is well-suited to gauge the performance of maximum entropy smoothing methods. Over a large number of data sets, we find that an ME smoothing method proposed to us by Lafferty [1] performs as well as or better than all other algorithms under consideration. This general and efficient method involves using a Gaussian prior on the parameters of the model and selecting maximum a posteriori instead of maximum likelihood parameter values. We contrast this method with previous n-gram smoothing methods to explain its superior performance.

show abstract

“…The method we use here, described in Beeferman, Berger, and Lafferty (1997), starts with a trigram model as a prior, or default distribution, and tacks onto the model a set of features to account for the long-range lexical properties of language. The features are trigger pairs, automatically discovered by analyzing a corpus of text using a mutual information heuristic Figure 2.…”

Section: Some Doctors Are More Skilled At Doing the Procedures Than Otmentioning

confidence: 99%

Untitled

1999

Self Cite

View full text Add to dashboard Cite

Abstract. This paper introduces a new statistical approach to automatically partitioning text into coherent segments. The approach is based on a technique that incrementally builds an exponential model to extract features that are correlated with the presence of boundaries in labeled training text. The models use two classes of features: topicality features that use adaptive language models in a novel way to detect broad changes of topic, and cue-word features that detect occurrences of specific words, which may be domain-specific, that tend to be used near segment boundaries. Assessment of our approach on quantitative and qualitative grounds demonstrates its effectiveness in two very different domains, Wall Street Journal news articles and television broadcast news story transcripts. Quantitative results on these domains are presented using a new probabilistically motivated error metric, which combines precision and recall in a natural and flexible way. This metric is used to make a quantitative assessment of the relative contributions of the different feature types, as well as a comparison with decision trees and previously proposed text segmentation algorithms.

show abstract

A model of lexical attraction and repulsion

Cited by 26 publications

References 3 publications

Evaluation of unsupervised semantic mapping of natural language with Leximancer concept mapping

Evaluation of unsupervised semantic mapping of natural language with Leximancer concept mapping

A Gaussian Prior for Smoothing Maximum Entropy Models

Untitled

Contact Info

Product

Resources

About