Shannon entropy as a robust estimator of Zipf's Law in animal vocal communication repertoires

Abstract

Information complexity in animals is an indicator of advanced communication and an intricate socio‐ecology. Zipf's Law of least effort has been used to assess the potential information content of animal repertoires, including whether or not a particular animal communication could be ‘language‐like’. As all human languages follow Zipf's law, with a power law coefficient (PLC) close to −1, animal signals with similar probability distributions are postulated to possess similar information characteristics to language. However, estimation of the PLC from limited empirical datasets (e.g. most animal communication studies) is problematic because of biases from small sample sizes. The traditional approach to estimating Zipf's law PLC is to find the slope of a log–log rank‐frequency plot. Our alternative option uses the underlying equivalence between Shannon entropy (i.e. whether successive elements of a sequence are unpredictable, or repetitive) and PLC. Here, we test whether an entropy approach yields more robust estimates of Zipf's law PLC than the traditional approach. We examined the efficacy of the entropy approach in two ways. First, we estimated the PLC from synthetic datasets generated with a priori known power law probability distributions. This revealed that the estimated PLC using the traditional method is particularly inaccurate for highly stereotyped sequences, even at modest repertoire sizes. Estimation via Shannon entropy is accurate with modest sample sizes even for repertoires with thousands of distinct elements. Second, we applied these approaches to empirical data taken from 11 animal species. Shannon entropy produced a more robust estimate of PLC with lower variance than the traditional method, even when the true PLC is unknown. Our approach for the first time reveals Zipf's law operating in the vocal systems of multiple lineages: songbirds, hyraxes and cetaceans. As different methods of estimating the PLC can lead to misleading results in real data, estimating the balance of a communication system between simplicity and complexity is best performed using the entropy approach. This provides a more robust way to investigate the evolutionary constraints and processes that have acted on animal communication systems, and the parallels between these processes and the evolution of language.

High plasticity in marmoset monkey vocal development from infancy to adulthood

“…In a recent study, Kershenbaum and colleagues ( 59 ) compared Zipf estimates of different synthetic and empirical datasets, showing that these estimates could be a helpful way to describe the communicative complexity of vocal sequences in animals.…”

Information Theory Opens New Dimensions in Experimental Studies of Animal Behaviour and Communication

“…There are two aspects of the information theory approach to studying animal language behaviour: the use of information theory to assess the diversity, complexity, and development of communicative repertoires (see, for example [ 60 , 63 , 114 , 115 ]), and the experimental way to studying communications based on Shannon entropy [ 41 ] and Kolmogorov complexity [ 73 ], which I analyse in this review.…”

“…The informal pattern is quite simple: the more frequently a message is used in a language, the shorter the word or the phrase coding it [ 141 ]. This phenomenon is manifested in all known human languages [ 142 ] and recently has been analysed in animal communication [ 115 ].…”

Plant-insect chemical communication in ecological communities: an information theory perspective

“…Among these, the "least effort" hypothesis (i Cancho and Solé, 2003;Zipf, 1949) is a compelling hypothesis that nicely recovers the Zipf's law distribution as a natural consequence from the conflicting interests between speakers (who are thought to aim for "brevity and phonological reduction" to minimize their effort to speak) and listeners (who should desire "explicitness and clarity" to minimize their effort to understand). The successful application of information theory in studying the structure and emergence of human language has inspired studies on animal vocal communication (e.g., reviews in Kershenbaum et al (2021); McCowan et al (2008); McCOWAN et al (1999)), and indicates its potential to be extended in studying the chemical "language" in plant-insect communities (Zu et al, 2020). Indeed, Zu et al (2020) aimed to borrow the framework to test whether plant-herbivore chemical communication patterns can be explained by conflicting interests of plants (speakers) and herbivores (listeners).…”