Can analysis of word frequency distinguish between writings of different authors?

Vilensky, Baruch

doi:10.1016/0378-4371(96)00109-4

Cited by 13 publications

(8 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [56], Meyer argues that on close inspection such claims turn out to be highly problematic, both on linguistic and on sciencetheoretical grounds. It has also been argued that it is possible to discriminate between human writings [57] and stochastic versions of texts precisely by looking at statistical properties of words. In contrast I argue here above that this statement can be extended to sentence statistics.…”

Section: Resultsmentioning

confidence: 99%

Punctuation effects in english and esperanto texts

Ausloos

2010

Physica A: Statistical Mechanics and its Applications

View full text Add to dashboard Cite

A statistical physics study of punctuation effects on sentence lengths is presented for written texts: Alice in wonderland and Through a looking glass. The translation of the first text into esperanto is also considered as a test for the role of punctuation in defining a style, and for contrasting natural and artificial, but written, languages. Several log-log plots of the sentence length-rank relationship are presented for the major punctuation marks. Different power laws are observed with characteristic exponents. The exponent can take a value much less than unity (ca. 0.50 or 0.30) depending on how a sentence is defined. The texts are also mapped into time series based on the word frequencies. The quantitative differences between the original and translated texts are very minutes, at the exponent level. It is argued that sentences seem to be more reliable than word distributions in discussing an author style.

show abstract

Section: Resultsmentioning

confidence: 99%

Punctuation effects in english and esperanto texts

Ausloos

2010

Physica A: Statistical Mechanics and its Applications

View full text Add to dashboard Cite

show abstract

“…Nevertheless, it has been argued that it is possible to discriminate between human writings [71] and stochastic versions of texts precisely by looking at statistical properties of words that fall where Eq. (1) does not hold [20].…”

Section: Zipf Methodsmentioning

confidence: 99%

Equilibrium and dynamic methods when comparing an English text and its Esperanto translation

Ausloos¹

2008

Physica A: Statistical Mechanics and its Applications

View full text Add to dashboard Cite

A comparison of two english texts from Lewis Carroll, one (Alice in wonderland), also translated into esperanto, the other (Through a looking glass) are discussed in order to observe whether natural and artificial languages significantly differ from each other. One dimensional time series like signals are constructed using only word frequencies (FTS) or word lengths (LTS). The data is studied through (i) a Zipf method for sorting out correlations in the FTS and (ii) a Grassberger-Procaccia (GP) technique based method for finding correlations in LTS. Features are compared : different power laws are observed with characteristic exponents for the ranking properties, and the phase space attractor dimensionality. The Zipf exponent can take values much less than unity (ca. 0.50 or 0.30) depending on how a sentence is defined. This non-universality is conjectured to be a measure of the author style. Moreover the attractor dimension r is a simple function of the so called phase space dimension n, i.e., r = n λ , with λ = 0.79. Such an exponent should also conjecture to be a measure of the author creativity. However, even though there are quantitative differences between the original english text and its esperanto translation, the qualitative differences are very minutes, indicating in this case a translation relatively well respecting, along our analysis lines, the content of the author writing.

show abstract

“…The aim is exploratory rather than hypothesis testing: to find out how word use in academic Web spaces differs from that in standard written English. Although word frequency analyses are capable of giving directly useful results (e.g., Boynton, Glanville, McDaid, & Lefebvre, 1998; Leydesdorff & Curran, 2000; Lindsay & Gordon, 1999; Oakes, 1998; Rousseau, 1999; Sanderson & van Rijsbergen, 1999; Vilensky, 1998), the primary aim of our research is to aid text‐based data cleansing. A second use is in building intuition for future scientific Web intelligence applications: Knowledge of the words used in academic Webs will suggest the kinds of information that can be extracted and perhaps also the problems that are likely to be encountered.…”

Section: Introductionmentioning

confidence: 99%

Text characteristics of English language university Web sites

Thelwall

2005

J. Am. Soc. Inf. Sci.

View full text Add to dashboard Cite

The nature of the contents of academic Web sites is of direct relevance to the new field of scientific Web intelligence, and for search engine and topic-specific crawler designers. We analyze word frequencies in national academic Webs using the Web sites of three Englishspeaking nations: Australia, New Zealand, and the United Kingdom. Strong regularities were found in page size and word frequency distributions, but with significant anomalies. At least 26% of pages contain no words. High frequency words include university names and acronyms, Internet terminology, and computing product names: not always words in common usage away from the Web. A minority of low frequency words are spelling mistakes, with other common types including nonwords, proper names, foreign language terms or computer science variable names. Based upon these findings, recommendations for data cleansing and filtering are made, particularly for clustering applications.

show abstract

Can analysis of word frequency distinguish between writings of different authors?

Cited by 13 publications

References 11 publications

Punctuation effects in english and esperanto texts

Punctuation effects in english and esperanto texts

Equilibrium and dynamic methods when comparing an English text and its Esperanto translation

Text characteristics of English language university Web sites

Contact Info

Product

Resources

About