Statistical generation of a technical vocabulary

Stone, Don C.; Rubinoff, Morris

doi:10.1002/asi.5090190409

Cited by 20 publications

(14 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In computing the combined scores given in equations (16) and (17), we have employed an optimization procedure to choose the coefficients. In such a computation there is always the possibility of overtraining, so that the results are applicable only to the particular data set on which we have done the training.…”

Section: Resultsmentioning

confidence: 99%

“…It has been proposed that non-content-bearing terms are well modeled by a single Poisson distribution, whereas content-bearing terms require a two-Poisson or some more complicated model. [15][16][17] In fact, one of our scoring methods is based on the degree that a term's distribution deviates from a Poisson distribution. The greater this deviation, the more likely that the term is a useful one.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Corpus-based Statistical Screening for Phrase Identification

Kim

Wilbur

2000

Journal of the American Medical Informatics Association

View full text Add to dashboard Cite

A b s t r a c t Purpose:The authors study the extraction of useful phrases from a natural language database by statistical methods. The aim is to leverage human effort by providing preprocessed phrase lists with a high percentage of useful material. Method:The approach is to develop six different scoring methods that are based on different aspects of phrase occurrence. The emphasis here is not on lexical information or syntactic structure but rather on the statistical properties of word pairs and triples that can be obtained from a large database. Measurements:The Unified Medical Language System (UMLS) incorporates a large list of humanly acceptable phrases in the medical field as a part of its structure. The authors use this list of phrases as a gold standard for validating their methods. A good method is one that ranks the UMLS phrases high among all phrases studied. Measurements are 11-point average precision values and precision-recall curves based on the rankings. Result:The authors find of six different scoring methods that each proves effective in identifying UMLS quality phrases in a large subset of MEDLINE. These methods are applicable both to word pairs and word triples. All six methods are optimally combined to produce composite scoring methods that are more effective than any single method. The quality of the composite methods appears sufficient to support the automatic placement of hyperlinks in text at the site of highly ranked phrases. Conclusion:Statistical scoring methods provide a promising approach to the extraction of useful phrases from a natural language database for the purpose of indexing or providing hyperlinks in text.

show abstract

Section: Resultsmentioning

confidence: 99%

mentioning

confidence: 99%

Corpus-based Statistical Screening for Phrase Identification

Kim

Wilbur

2000

Journal of the American Medical Informatics Association

View full text Add to dashboard Cite

show abstract

“…(Bookstein & Swanson, 1974;1975), (Harter, 1975), (Stone & Rubinoff, 1968), (Dennis, 1967), (Damerau, 1965). Det kan ikke desto mindre vaere ganske illustrativt at underbygge det intuitivt plausible i det anførte fagsprogssyn ved en simpel logisk test, hvor det iagttages, hvorvidt et bestemt ord enten optraeder eller ikke optraeder i en tekstsamlings forskellige delkorpora.…”

Section: Fagsprogsopfattelseunclassified

Kvantitativ terminologi. En sprogstatistisk tilgang til automatisk identifikation og udtrækning af teksttypiske og -begrænsede lemmata i fagsprogstekster

Blom¹

2002

NyS

View full text Add to dashboard Cite

show abstract

“…It is possible to think of such features as having been produced by a conceptual feature generator that produces features at a fixed average rate with the interval between feature occurrences a random exponential variable. The Poisson model has been suggested as providing a satisfactory description of the occurrence frequencies of natural language terms [12,23,36]. Use of this distribution may be helpful if advantage is taken of content analytic methods for counting the number of times particular positive or negative terms or expressions occur in a message [6,18].…”

Section: Poisson Independently Distributed Featuresmentioning

confidence: 99%

Minimizing information overload: the ranking of electronic messages

Losee¹

1989

Journal of Studies in International Education

View full text Add to dashboard Cite

The decision to examine a message at a particular point in time should be made rationally and economically if the message recipient is to operate efficiently. Electronic message distribution systems, electronic bulletin board systems, and telephone systems capable of leaving digitized voice messages can contribute to "information overload," defined as the economic loss associated with the examination of a number of non-or less-relevant messages. Our model provides a formal method for minimizing expected information overload.The proposed adaptive model predicts the usefulness of a message based on the available message features and may be used to rank messages by expected importance or economic worth. The assumptions of binary and two Poisson independent probabilistic distributions of message feature frequencies are examined, and methods of incorporating these distributions into the ranking model are examined. Ways to incorporate user supplied relevance feedback are suggested. Analytic performance measures are proposed to predict system quality. Other message handling models, including rule based expert systems, are seen as special cases of the model. The performance is given for a set of UNIX shell programs which rank messages. Problems with the use of this formal model are examined, and areas for future research are suggested.

show abstract

Statistical generation of a technical vocabulary

Abstract: The results of an experiment in the use of statistical techniques for extracting a technical vocabulary from document texts are presented and discussed.

Cited by 20 publications

References 1 publication

Corpus-based Statistical Screening for Phrase Identification

Corpus-based Statistical Screening for Phrase Identification

Kvantitativ terminologi. En sprogstatistisk tilgang til automatisk identifikation og udtrækning af teksttypiske og -begrænsede lemmata i fagsprogstekster

Minimizing information overload: the ranking of electronic messages

Contact Info

Product

Resources

About