Study on Constants of Natural Language Texts

Abstract

This paper considers different measures that might become constants for any length of a given natural language text. Such measures indicate a potential for studying the complexity of natural language but have previously only been studied using relatively small English texts. In this study, we consider measures for texts in languages other than English, and for large-scale texts. Among the candidate measures, we consider Yule's K, Orlov's Z, and Golcher's VM , each of whose convergence has been previously argued empirically. Furthermore, we introduce entropy H, and a measure, r, related to the scale-free property of language. Our experiments show that both K and VM are convergent for texts in various languages, whereas the other measures are not.

The Impact of Depression and Apathy on Sensory Language

“…The type-token ratio (TTR) can also be considered a variant of the species diversity equation, and is text size dependent, however, it is a popular metric, and we mitigate any size impacts and avoid issues near the asymptote of word counts by keeping all the samples of each novel of equal size and at 4000 words, well below 10,000 where other techniques perform better; further, we use Richness as part of a larger multivariate technique (Juola & Mikros, 2016;Kimura & Tanaka-Ishii, 2014;Kubát & Milička, 2013;Tanaka-Ishii & Aihara, 2015;Van Gijsel, Speelman, & Geeraerts, 2005;Vermeer, 2000).…”