Meaning and Measures: Interpreting and Evaluating Complexity Metrics

Abstract

Research on language complexity has been abundant and manifold in the past two decades. Within typology, it has to a very large extent been motivated by the question of whether all languages are equally complex, and if not, which language-external factors affect the distribution of complexity across languages. To address this and other questions, a plethora of different metrics and approaches has been put forward to measure the complexity of languages and language varieties. Against this backdrop we address three major gaps in the literature by discussing statistical, theoretical, and methodological problems related to the interpretation of complexity measures. First, we explore core statistical concepts to assess the meaningfulness of measured differences and distributions in complexity based on two case studies. In other words, we assess whether observed measurements are neither random nor negligible. Second, we discuss the common mismatch between measures and their intended meaning, namely, the fact that absolute complexity measures are often used to address hypotheses on relative complexity. Third, in the absence of a gold standard for complexity metrics, we suggest that existing measures be evaluated by drawing on cognitive methods and relating them to real-world cognitive phenomena. We conclude by highlighting the theoretical and methodological implications for future complexity research.

Interindividual Variation Refuses to Go Away: A Bayesian Computer Model of Language Change in Communicative Networks

“…Needless to say, the fact that heated debates have been flourishing for at least two decades suggests that these assumptions have led to multiple interpretations of how complexity should be defined and how it should be considered, and consequently that the complexity jigsaw puzzle has still to be solved. Several contributions to this Research Topic specifically address these aspects, e.g., Ehret et al (2021) on the equal complexity aspect, or Ehret et al (2021) and Joseph (2021) on measuring complexity, to name just a few. Another heated debate is about the existence of putative complexity trade-offs within each language (i.e., do phonological, morphological, and syntactic complexities interact and compensate or combine?…”

Natural language syntax complies with the free-energy principle

“…Instead, we are interested in underlying representational issues that pertain to syntax-semantics. Even the most recent explorations of simplicity in language, such as the volume on simplicity in grammar learning in Katzir et al (2021), focus on modelling minimum description length in phonology and morphology, or morphosyntactic complexity across distinct languages (Ehret et al, 2023), but not processes pertaining to the internal derivation of syntactic objects. Much of this work fruitfully explores complexity and simplicity across languages (Ehret et al, 2023), using measures such as second language acquisition difficulty and situational diversity (counting the range of communicative contexts a language can be used in), as opposed to the computational architecture of the language faculty itself.…”

Structural complexity reduction in English–French bilingual children’s event encoding

“…A second major claim by Beavers et al (2010) relevant to the present study is that intra-typological (dis-)preferences for certain encoding options over others can be explained as the result of a universal bias to prefer morphosyntactically less complex over more complex structures owing to ease of processing (Beavers et al, 2010, p. 36). It is important to note at this point that linguistic complexity is a multi-faceted notion and that there is no broad consensus on how to define or measure it (Ehret et al, 2021). An important distinction conventionally drawn in the literature is that between absolute and relative complexity (e.g., Dahl, 2004;Miestamo, 2008).…”

“…In the present study, syntactic complexity is defined as the sub-construct of subordination, which was operationalised as the proportion of responses involving the presence of at least one subordinate clause. The choice of subordination as a measure was motivated by its developmental relevance for L1 acquisition data generally, in light of research demonstrating that subordination is acquired late and well after coordination, (e.g., Crosson et al, 2008;Diessel, 2004) and has been used as a metric to account for timing patterns in child acquisition contexts (e.g., Tsimpli, 2014). The notion of subordination is also relevant for the acquisition of motion event encoding in particular, where subordination (but not coordination) has been shown to be associated with protracted acquisition trajectories (see Hickmann et al, 2018).…”