Keyword detection in natural languages and DNA

Ortuño, M.; Carpena, Pedro; Bernaola‐Galván, Pedro; Muñoz, Enrique; Somoza, A. M.

doi:10.1209/epl/i2002-00528-3

Cited by 107 publications

(114 citation statements)

References 19 publications

(11 reference statements)

Supporting

Mentioning

109

Contrasting

Unclassified

Order By: Relevance

“…The representative of content words is "WHALE" which is the 23rd ranked word but the most common noun. The main conclusion in [35,36] is confirmed that function words such as THE tend to be evenly scattered whereas content words such as WHALE tends to be clustered, leaving huge gaps between clusters (and the low-rank-number spacings are much larger). When a power-law function is applied to fit the inter-word spacings, the fitting performance is not good.…”

Section: Ranked Inter-word Spacing Distributionmentioning

confidence: 89%

“…Motivated by level statistics in quantum disorder systems, it has been proposed that distance between successive occurrences of the same word might be related to whether or not that word plays an important role in the text (the so-called "keyword") [35,36]. Similar studies of gap distributions are also common in bioinformatics, such as the in-frame start-to-stop codon distances (which defines "open reading frames") [37,38] or in-frame stop-stop codon distances [39].…”

Section: Ranked Inter-word Spacing Distributionmentioning

confidence: 99%

“…Two sets of result are presented: one for the weighted regression (weight w r = 1/r), another without weights. SSE: sum of squared error (Equation (10)); ∆AIC: Akaike information criterion relative to the best model (whose AIC is set at 0) (Equation (14) It is proposed in [35] that if a word is randomly distributed along the word sequence, it is a Poisson process and the nearest-neighbor spacing will follow an exponential distribution. That equation is exactly the Gusein-Zade function in Equation (8).…”

Section: Ranked Inter-word Spacing Distributionmentioning

confidence: 99%

See 2 more Smart Citations

Fitting Ranked Linguistic Data with Two-Parameter Functions

Miramóntes

Cocho

2010

Entropy

View full text Add to dashboard Cite

It is well known that many ranked linguistic data can fit well with one-parameter models such as Zipf's law for ranked word frequencies. However, in cases where discrepancies from the one-parameter model occur (these will come at the two extremes of the rank), it is natural to use one more parameter in the fitting model. In this paper, we compare several two-parameter models, including Beta function, Yule function, Weibull function-all can be framed as a multiple regression in the logarithmic scale-in their fitting performance of several ranked linguistic data, such as letter frequencies, word-spacings, and word frequencies. We observed that Beta function fits the ranked letter frequency the best, Yule function fits the ranked word-spacing distribution the best, and Altmann, Beta, Yule functions all slightly outperform the Zipf's power-law function in word rankedfrequency distribution.

show abstract

Section: Ranked Inter-word Spacing Distributionmentioning

confidence: 89%

Section: Ranked Inter-word Spacing Distributionmentioning

confidence: 99%

Section: Ranked Inter-word Spacing Distributionmentioning

confidence: 99%

See 1 more Smart Citation

Fitting Ranked Linguistic Data with Two-Parameter Functions

Miramóntes

Cocho

2010

Entropy

View full text Add to dashboard Cite

show abstract

“…It has been widely employed to detect keywords in texts as an alternative to the tf-idf technique [53]. In addition, the intermittency has proven relevant to detect keywords in genetic sequences [32]. A qualitative comparison of words taking distinct values of intermittency is provided in figure 2, which shows the distribution of the words 'Carmylle' (f i = 54) and 'feel' (f i = 54) in the book 'Adventures of Sally', by Pelham Grenville Wodehouse.…”

Section: Intermittencymentioning

confidence: 99%

Authorship recognition via fluctuation analysis of network topology and word intermittency

Amancio¹

2015

J. Stat. Mech.

View full text Add to dashboard Cite

Abstract. Statistical methods have been widely employed in many practicalnatural language processing applications. More specifically, complex network concepts and methods from dynamical systems theory have been successfully applied to recognize stylistic patterns in written texts. Despite the large number of studies devoted to representing texts with physical models, only a few studies have assessed the relevance of attributes derived from the analysis of stylistic fluctuations. Because fluctuations represent a pivotal factor for characterizing a myriad of real systems, this study focused on the analysis of the properties of stylistic fluctuations in texts via topological analysis of complex networks and intermittency measurements. The results showed that different authors display distinct fluctuation patterns. In particular, it was found that it is possible to identify the authorship of books using the intermittency of specific words. Taken together, the results described here suggest that the patterns found in stylistic fluctuations could be used to analyze other related complex systems. Furthermore, the discovery of novel patterns related to textual stylistic fluctuations indicates that these patterns could be useful to improve the state of the art of many stylistic-based natural language processing tasks.

show abstract

“…These properties are dependent on the parts of speech (POS) of the phrase constituents [2]. Few of the major linguistic patterns for a phrase in English are: [5]. He argues that the standard deviation of the distance between successive occurrences of a word is such a parameter to quantify this self-attraction.…”

Section: Some Linguistic Properties Of Keyphrasesmentioning

confidence: 99%

Keyword and Keyphrase Extraction Techniques: A Literature Review

Siddiqi¹,

Sharan²

2015

IJCA

156

View full text Add to dashboard Cite

In this paper we present a survey of various techniques available in text mining for keyword and keyphrase extraction. Keywords and keyphrases are very useful in analyzing large amount of textual material quickly and efficiently search over the internet besides being useful for many other purposes. Keywords and keyphrases are set of representative words of a document that give high-level specification of the content for interested readers. They are used highly in the field of Computer Science especially in Information Retrieval and Natural Language Processing and can be used for index generation, query refinement, text summarization, author assistance, etc. We have also discussed some important feature selection metrics generally employed by researchers to rank candidate keywords and keyphrases according to their importance.

show abstract

Keyword detection in natural languages and DNA

Cited by 107 publications

References 19 publications

Fitting Ranked Linguistic Data with Two-Parameter Functions

Fitting Ranked Linguistic Data with Two-Parameter Functions

Authorship recognition via fluctuation analysis of network topology and word intermittency

Keyword and Keyphrase Extraction Techniques: A Literature Review

Contact Info

Product

Resources

About