Probability distribution in a quantitative linguistic problem

Abstract

In the present contribution, we propose a possible way to discuss the distributions of words in a given text. We have devoted our study to discuss some relevant properties observed in Spanish texts of Latin-American writers. We start analyzing the appearance of distributions of the frequency of occurrence in the Zipf perspective. We identify two regions of behavior separated by a special point. In order to correctly define such a point, we work beyond the Zipf law, defining other probability distribution that takes the frequency of repetition of a particular word among other different words into account. At this point, we take the linguistic problem to a statistical level. We make an effort to characterize the point of separation between two regions, via the Binder cumulant of fourth order, as it is made in the characterization of critical points in phase transitions of physical systems.

Stochastic Orderings and Parameter Estimation for the Hurwitz-Lerch Zeta Distribution

“…The Zipf's law or the power law that describes decay which is slower than the exponential rate has been applied to various fields where ranking of sizes or frequencies are of particular interest, such as quantitative linguistics [2], human demography [3], economics [4], citations [5] and internet traffic modelling [6]. Discrete distributions that are related to the power law include the Zipf-Mandelbrot distribution, Lotka distribution and so on.…”