Optimal encoding! - Information Theory constrains article omission
            in newspaper headlines

Lemke, Robin; Horch, Eva; Reich, Ingo

doi:10.18653/v1/e17-2021

Cited by 7 publications

(8 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The tendency to reduce predictable expressions has been observed on different levels of linguistic analysis, ranging from phonetics [1,[38][39][40][41][42][43][44][45][46][47] and morphology [48] to the omission of predictable words [4,[6][7][8][9]49]. If this principle applies to fragments as well, we expect that fragments are more strongly preferred over the corresponding full sentence if the omission of words that are predictable in a specific context results in a well-formed fragment.…”

Section: Predictability Effects On Omissions In Fragmentsmentioning

confidence: 99%

“…Throughout the last 15 years, predictability effects on the choice and realization of linguistic expressions have been evidenced for a variety of languages and levels of linguistic analysis. This encompasses the findings that predictable words are articulatorily reduced [1], read faster [2] and more often pronominalized [3] and omitted [4][5][6][7][8][9]. These predictability effects have motivated the more general insight that the probability of a word to appear in context is proportional to the effort required for processing it [10,11].…”

Section: Introductionmentioning

confidence: 95%

See 1 more Smart Citation

Modeling the predictive potential of extralinguistic context with script knowledge: The case of fragments

2021

Self Cite

View full text Add to dashboard Cite

We describe a novel approach to estimating the predictability of utterances given extralinguistic context in psycholinguistic research. Predictability effects on language production and comprehension are widely attested, but so far predictability has mostly been manipulated through local linguistic context, which is captured with n-gram language models. However, this method does not allow to investigate predictability effects driven by extralinguistic context. Modeling effects of extralinguistic context is particularly relevant to discourse-initial expressions, which can be predictable even if they lack linguistic context at all. We propose to use script knowledge as an approximation to extralinguistic context. Since the application of script knowledge involves the generation of prediction about upcoming events, we expect that scrips can be used to manipulate the likelihood of linguistic expressions referring to these events. Previous research has shown that script-based discourse expectations modulate the likelihood of linguistic expressions, but script knowledge has often been operationalized with stimuli which were based on researchers’ intuitions and/or expensive production and norming studies. We propose to quantify the likelihood of an utterance based on the probability of the event to which it refers. This probability is calculated with event language models trained on a script knowledge corpus and modulated with probabilistic event chains extracted from the corpus. We use the DeScript corpus of script knowledge to obtain empirically founded estimates of the likelihood of an event to occur in context without having to resort to expensive pre-tests of the stimuli. We exemplify our method at a case study on the usage of nonsentential expressions (fragments), which shows that utterances that are predictable given script-based extralinguistic context are more likely to be reduced.

show abstract

Section: Predictability Effects On Omissions In Fragmentsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 95%

Modeling the predictive potential of extralinguistic context with script knowledge: The case of fragments

2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…The Uniform Information Density (UID) hypothesis (Levy and Jaeger, 2007 ) has been successfully applied to account for a variety of omission phenomena from acoustic reduction (Aylett and Turk, 2004 ; see Jaeger and Buz, 2017 for an overview), to the omission of functional elements such as relativizers (Levy and Jaeger, 2007 ), complementizers (Jaeger, 2010 ) and discourse markers (Asr and Demberg, 2015 ) in English, case markers in Japanese (Kurumada and Jaeger, 2015 ) and articles in German newspaper articles (Lemke et al, 2017 ), to the omission of content words, for instance the deletion of parts of the utterance in German fragments (Lemke et al, 2020 ) and the omission of preverbal subjects in Russian (Kravtchenko, 2014 ). In a recent study, Lemke et al 2 found that UID also constrains other elliptical phenomena such as sluicing.…”

Section: Information-theoretic Account To Vp Ellipsismentioning

confidence: 99%

The Role of UID for the Usage of Verb Phrase Ellipsis: Psycholinguistic Evidence From Length and Context Effects

et al. 2021

Self Cite

View full text Add to dashboard Cite

We investigate the underexplored question of when speakers make use of the omission phenomenon verb phrase ellipsis (VPE) in English given that the full form is also available to them. We base the interpretation of our results on the well-established information-theoretic Uniform Information Density (UID) hypothesis: Speakers tend to distribute processing effort uniformly across utterances and avoid regions of low information by omitting redundant material through, e.g., VPE. We investigate the length of the omittable VP and its predictability in context as sources of redundancy which lead to larger or deeper regions of low information and an increased pressure to use ellipsis. We use both naturalness rating and self-paced reading studies in order to link naturalness patterns to potential processing difficulties. For the length effects our rating and reading results support a UID account. Surprisingly, we do not find an effect of the context on the naturalness and the processing of VPE. We suggest that our manipulation might have been too weak or not effective to evidence such an effect.

show abstract

“…Information-theoretic processing constraints have been shown to explain the distribution of a wide range of reduction phenomena. Their application ranges from phonological reduction (Bell et al, 2003(Bell et al, , 2009Aylett and Turk, 2004;Demberg et al, 2012;Kuperman and Bresnan, 2012;Seyfarth, 2014;Pate and Goldwater, 2015;Brandt et al, 2017Brandt et al, , 2018Malisz et al, 2018) to morphological effects on contraction (Frank and Jaeger, 2008) and case marker omission (Kurumada and Jaeger, 2015;Norcliffe and Jaeger, 2016) to pronominalization (Tily and Piantadosi, 2009), and, what is most closely related to omissions in fragments, optional omissions of various types of function words (Levy and Jaeger, 2007;Jaeger, 2010;Asr and Demberg, 2015;Lemke et al, 2017) and preverbal subjects (Kravtchenko, 2014;Schäfer, 2021).…”

Section: An Information-theoretic Account Of Fragment Usagementioning

confidence: 99%

“…However, Lemke et al (2021) did not look into the more fine-grained predictions of the information-theoretic account on the word level. Second, information-theoretic constraints have been shown to explain the distribution of omissions particularly on the word level, such as those of complementizers, pronouns, articles and case markers (Levy and Jaeger, 2007;Frank and Jaeger, 2008;Jaeger, 2010;Asr and Demberg, 2015;Kurumada and Jaeger, 2015;Norcliffe and Jaeger, 2016;Lemke et al, 2017). Even though most of these studies focused on semantically relatively empty function words, 2 information-theoretic constraints make similar predictions on the omission of content words.…”

Section: Introductionmentioning

confidence: 98%

Predictable Words Are More Likely to Be Omitted in Fragments–Evidence From Production Data

et al. 2021

Self Cite

View full text Add to dashboard Cite

Instead of a full sentence like Bring me to the university (uttered by the passenger to a taxi driver) speakers often use fragments like To the university to get their message across. So far there is no comprehensive and empirically supported account of why and under which circumstances speakers sometimes prefer a fragment over the corresponding full sentence. We propose an information-theoretic account to model this choice: A speaker chooses the encoding that distributes information most uniformly across the utterance in order to make the most efficient use of the hearer's processing resources (Uniform Information Density, Levy and Jaeger, 2007). Since processing effort is related to the predictability of words (Hale, 2001) our account predicts two effects of word probability on omissions: First, omitting predictable words (which are more easily processed), avoids underutilizing processing resources. Second, inserting words before very unpredictable words distributes otherwise excessively high processing effort more uniformly. We test these predictions with a production study that supports both of these predictions. Our study makes two main contributions: First we develop an empirically motivated and supported account of fragment usage. Second, we extend previous evidence for information-theoretic processing constraints on language in two ways: We find predictability effects on omissions driven by extralinguistic context, whereas previous research mostly focused on effects of local linguistic context. Furthermore, we show that omissions of content words are also subject to information-theoretic well-formedness considerations. Previously, this has been shown mostly for the omission of function words.

show abstract

Optimal encoding! - Information Theory constrains article omission in newspaper headlines

Cited by 7 publications

References 8 publications

Modeling the predictive potential of extralinguistic context with script knowledge: The case of fragments

Modeling the predictive potential of extralinguistic context with script knowledge: The case of fragments

The Role of UID for the Usage of Verb Phrase Ellipsis: Psycholinguistic Evidence From Length and Context Effects

Predictable Words Are More Likely to Be Omitted in Fragments–Evidence From Production Data

Contact Info

Product

Resources

About