Info/information theory: Speakers choose shorter words in predictive contexts

Mahowald, Kyle; Fedorenko, Evelina; Piantadosi, Steven T.; Gibson, Edward

doi:10.1016/j.cognition.2012.09.010

Cited by 175 publications

(144 citation statements)

References 11 publications

Supporting

Mentioning

135

Contrasting

Unclassified

Order By: Relevance

“…For example, speakers are more likely to omit optional arguments when their semantics is inferable from the verb (Resnik, 1996 ; see also Brown & Dell, 1987 ). Similarly, speakers are more likely to choose more reduced expressions when the referent is expected in context (e.g., pronoun vs. lexical noun: Tily & Piantadosi, 2009 ; abbreviated vs. full nouns, like math vs. mathematics : Mahowald, Fedorenko, Piantadosi, & Gibson, 2013 ).…”

Section: Introductionmentioning

confidence: 99%

Predicting head-marking variability in Yucatec Maya relative clause production

Norcliffe

Jaeger

2014

Lang. cogn.

View full text Add to dashboard Cite

a b st r a c t Recent proposals hold that the cognitive systems underlying language production exhibit computational properties that facilitate communicative effi ciency, i.e., an effi cient trade-off between production ease and robust information transmission. We contribute to the cross-linguistic evaluation of the communicative effi ciency hypothesis by investigating speakers' preferences in the production of a typologically rare head-marking alternation that occurs in relative clause constructions in Yucatec Maya. In a sentence recall study, we fi nd that speakers of Yucatec Maya prefer to use reduced forms of relative clause verbs when the relative clause is more contextually expected. This result is consistent with communicative effi ciency and thus supports its typological generalizability. We compare[ * ] We thank Serapio Canul Dzib for Yucatec Maya language consulting and for assistance in preparing, translating, and recording experimental stimuli; Marcelina Chan, Miguel Sosoya, José Cipriano Dzib Uitzil, and Lorena Pool Balam for transcribing the Yucatec data; Andrew Watts for programming assistance; and Juergen Bohnemeyer for sharing his knowledge of and insights into Yucatec grammar with us. We are also very grateful to Mtra. Leyla Gisela Leo Peraza and Ddra. Graciela Cortés Camarillo at the Universidad de Oriente (UNO), for their offi cial support of the project, and to Betsy Kraft and Marta Beatriz Poot Nahuat for their invaluable assistance with participant recruiting and administration. Finally, our thanks to all the students at the UNO who took part in our experiment. This work was partially funded through National Science Foundation grants BCS-0848353 and CAREER IIS-1150028 to TFJ. The views expressed here are those of the authors and do not necessarily refl ect those of the National Science Foundation. Address for correspondence: Elisabeth Norcliff e, Max Planck Institute for Psycholinguistics, PO Box 310, 6500 AH Nijmegen, The Netherlands. e-mail: elisabeth.norcliff e@mpi.nl nor cliffe and jaeger 168 two types of cue to the presence of a relative clause, pragmatic cues previously investigated in other languages and a highly predictive morphosyntactic cue specifi c to Yucatec. We fi nd that Yucatec speakers' preferences for a reduced verb form are primarily conditioned on the more informative cue. This demonstrates the role of both general principles of language production and their language-specifi c realizations.

show abstract

Section: Introductionmentioning

confidence: 99%

Predicting head-marking variability in Yucatec Maya relative clause production

Norcliffe

Jaeger

2014

Lang. cogn.

View full text Add to dashboard Cite

show abstract

“…In the context of quantitative linguistics, entropic measures are used to understand laws in natural languages, such as the relationship between word frequency, predictability and the length of words [24][25][26][27], or the trade-off between word structure and sentence structure [10,13,28]. Information theory can further help to understand the complexities involved when building words from the smallest meaningful units, i.e., morphemes [29,30].…”

Section: Introductionmentioning

confidence: 99%

The Entropy of Words—Learnability and Expressivity across More than 1000 Languages

Bentz

Alikaniotis

Cysouw

et al. 2017

Entropy

View full text Add to dashboard Cite

Abstract:The choice associated with words is a fundamental property of natural languages. It lies at the heart of quantitative linguistics, computational linguistics and language sciences more generally. Information theory gives us tools at hand to measure precisely the average amount of choice associated with words: the word entropy. Here, we use three parallel corpora, encompassing ca. 450 million words in 1916 texts and 1259 languages, to tackle some of the major conceptual and practical problems of word entropy estimation: dependence on text size, register, style and estimation method, as well as non-independence of words in co-text. We present two main findings: Firstly, word entropies display relatively narrow, unimodal distributions. There is no language in our sample with a unigram entropy of less than six bits/word. We argue that this is in line with information-theoretic models of communication. Languages are held in a narrow range by two fundamental pressures: word learnability and word expressivity, with a potential bias towards expressivity. Secondly, there is a strong linear relationship between unigram entropies and entropy rates. The entropy difference between words with and without co-textual information is narrowly distributed around ca. three bits/word. In other words, knowing the preceding text reduces the uncertainty of words by roughly the same amount across languages of the world.

show abstract

“…Our analyses are motivated by recent studies that show a message's Information-Theoretic structure is influenced at a variety of linguistic levels including syntactic variation and phonetic reduction (Aylett, 1999;Genzel & Charniak, 2002;Aylett & Turk, 2006;2004;Levy & Jaeger, 2006;Jaeger, 2010;Mahowald et al, 2013). Specifically, the amount of information present across a message is shown to increase over time, abiding by the entropy rate constancy principlea message's information density increases at a stable rate (Genzel & Charniak, 2002)-perhaps in an effort to provide relevant content against channel noise.…”

Section: Current Studymentioning

confidence: 99%

Efficient n-gram analysis in R with cmscu

et al. 2016

View full text Add to dashboard Cite

We present a new R package, cmscu, which implements a Count-Min-Sketch with conservative updating (Cormode and Muthukrishnan Journal of Algorithms, 55(1), 58-75, 2005), and its application to n-gram analyses (Goyal et al., 2012). By writing the core implementation in C++ and exposing it to R via Rcpp, we are able to provide a memory-efficient, high-throughput, and easy-to-use library. As a proof of concept, we implemented the computationally challenging (Heafield et al., 2013) modified Kneser-Ney n-gram smoothing algorithm using cmscu as the querying engine. We then explore information density measures (Jaeger Cognitive Psychology, 61(1), 23-62, 2010) from ngram frequencies (for n = 2, 3) derived from a corpus of over 2.2 million reviews provided by a Yelp, Inc. dataset. We demonstrate that these text data are at a scale beyond the reach of other more common, more general-purpose libraries available through CRAN. Using the cmscu library and the smoothing implementation, we find a positive relationship between review information density and reader review ratings. We end by highlighting the important use of new efficient tools to explore behavioral phenomena in large, relatively noisy data sets.

show abstract

Info/information theory: Speakers choose shorter words in predictive contexts

Cited by 175 publications

References 11 publications

Predicting head-marking variability in Yucatec Maya relative clause production

Predicting head-marking variability in Yucatec Maya relative clause production

The Entropy of Words—Learnability and Expressivity across More than 1000 Languages

Efficient n-gram analysis in R with cmscu

Contact Info

Product

Resources

About