Rank-frequency relation for Chinese characters

Deng, Weibing; Allahverdyan, Armen E.; Li, Bo; Wang, Qiuping A.

doi:10.1140/epjb/e2014-40805-2

Cited by 20 publications

(29 citation statements)

References 60 publications

(134 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, Deng et al found that the most used Chinese words R i (w k ) < 80 ''contain most functional characters'', and have different statistics in this ''pre-Zipfian region'' [11].…”

Section: Data and Calculation Resultsmentioning

confidence: 99%

Word population analysis and other evidences indicate that Shiji was amended by Liu Xiang

Chi

Wang

2015

Physica A: Statistical Mechanics and its Applications

View full text Add to dashboard Cite

“…Recently, Deng et al found that the most used Chinese words R i (w k ) < 80 ''contain most functional characters'', and have different statistics in this ''pre-Zipfian region'' [11].…”

Section: Data and Calculation Resultsmentioning

confidence: 99%

Word population analysis and other evidences indicate that Shiji was amended by Liu Xiang

Chi

Wang

2015

Physica A: Statistical Mechanics and its Applications

View full text Add to dashboard Cite

“…Note, however, that matching regime γ tailed exponents is non-trivial. For example, ranked frequency distributions of Chinese texts may be better fitted by stretched exponential functions than by power laws ( Deng et al, 2014 ). In addition, languages show “size-effects” generally present in the tails of frequency distributions that are dependent on the size of text corpora.…”

Section: Language Laws Are Constrained By the Engineering Of Biological Systemsmentioning

confidence: 99%

The Compressed Vocabulary of Microbial Life

Caetano‐Anollés

2021

Front. Microbiol.

View full text Add to dashboard Cite

Communication is an undisputed central activity of life that requires an evolving molecular language. It conveys meaning through messages and vocabularies. Here, I explore the existence of a growing vocabulary in the molecules and molecular functions of the microbial world. There are clear correspondences between the lexicon, syntax, semantics, and pragmatics of language organization and the module, structure, function, and fitness paradigms of molecular biology. These correspondences are constrained by universal laws and engineering principles. Macromolecular structure, for example, follows quantitative linguistic patterns arising from statistical laws that are likely universal, including the Zipf’s law, a special case of the scale-free distribution, the Heaps’ law describing sublinear growth typical of economies of scales, and the Menzerath–Altmann’s law, which imposes size-dependent patterns of decreasing returns. Trade-off solutions between principles of economy, flexibility, and robustness define a “triangle of persistence” describing the impact of the environment on a biological system. The pragmatic landscape of the triangle interfaces with the syntax and semantics of molecular languages, which together with comparative and evolutionary genomic data can explain global patterns of diversification of cellular life. The vocabularies of proteins (proteomes) and functions (functionomes) revealed a significant universal lexical core supporting a universal common ancestor, an ancestral evolutionary link between Bacteria and Eukarya, and distinct reductive evolutionary strategies of language compression in Archaea and Bacteria. A “causal” word cloud strategy inspired by the dependency grammar paradigm used in catenae unfolded the evolution of lexical units associated with Gene Ontology terms at different levels of ontological abstraction. While Archaea holds the smallest, oldest, and most homogeneous vocabulary of all superkingdoms, Bacteria heterogeneously apportions a more complex vocabulary, and Eukarya pushes functional innovation through mechanisms of flexibility and robustness.

show abstract

“…Given word rank u and frequency F ( u ) for a word of rank u , Zipf's law suggests the following proportionality formula:

As shown here for Les Misérables , the plot typically follows formula (3) only approximately. There have been discussions on how to improve the Zipf model by incorporating such bias (Mandelbrot, 1952 , 1965 ; Gerlach and Altmann, 2013 ; Deng et al, 2014 ). To the best of the author's knowledge, however, the question of a mathematical model that fully explains the bias is still under debate.…”

Section: Quantification Of Long-range Correlationmentioning

confidence: 99%

Long-Range Correlation Underlying Childhood Language and Generative Models

Tanaka-Ishii

2018

Front. Psychol.

View full text Add to dashboard Cite

Long-range correlation, a property of time series exhibiting relevant statistical dependence between two distant subsequences, is mainly studied in the statistical physics domain and has been reported to exist in natural language. By using a state-of-the-art method for such analysis, long-range correlation is first shown to occur in long CHILDES data sets. To understand why, generative stochastic models of language, originally proposed in the cognitive scientific domain, are investigated. Among representative models, the Simon model is found to exhibit surprisingly good long-range correlation, but not the Pitman-Yor model. Because the Simon model is known not to correctly reflect the vocabulary growth of natural languages, a simple new model is devised as a conjunct of the Simon and Pitman-Yor models, such that long-range correlation holds with a correct vocabulary growth rate. The investigation overall suggests that uniform sampling is one cause of long-range correlation and could thus have some relation with actual linguistic processes.

show abstract

Rank-frequency relation for Chinese characters

Cited by 20 publications

References 60 publications

Word population analysis and other evidences indicate that Shiji was amended by Liu Xiang

Word population analysis and other evidences indicate that Shiji was amended by Liu Xiang

The Compressed Vocabulary of Microbial Life

Long-Range Correlation Underlying Childhood Language and Generative Models

Contact Info

Product

Resources

About