Listening Effort: How the Cognitive Consequences of Acoustic Challenge Are Reflected in Brain and Behavior

Abstract

Everyday conversation frequently includes challenges to the clarity of the acoustic speech signal, including hearing impairment, background noise, and foreign accents. Although an obvious problem is the increased risk of making word identification errors, extracting meaning from a degraded acoustic signal is also cognitively demanding, which contributes to increased listening effort. The concepts of cognitive demand and listening effort are critical in understanding the challenges listeners face in comprehension, which are not fully predicted by audiometric measures. Here I review converging behavioral, pupillometric, and neuroimaging evidence that understanding acoustically degraded speech requires additional cognitive support, and that this cognitive load can interfere with other operations such as language processing and memory for what has been heard. Behaviorally, acoustic challenge is associated with increased errors in speech understanding, poorer performance on concurrent secondary tasks, more difficulty processing linguistically complex sentences, and reduced memory for verbal material. Measures of pupil dilation support the challenge associated with processing a degraded acoustic signal, indirectly reflecting an increase in neural activity. Finally, functional brain imaging reveals that the neural resources required to understand degraded speech extend beyond traditional perisylvian language networks, most commonly including regions of prefrontal cortex, premotor cortex, and the cingulo-opercular network. Far from being exclusively an auditory problem, acoustic degradation presents listeners with a systems-level challenge that requires the allocation of executive cognitive resources. An important point is that a number of dissociable processes can be engaged to understand degraded speech, including verbal working memory and attention-based performance monitoring. The specific resources required likely differ as a function of the acoustic, linguistic, and cognitive demands of the task, as well as individual differences in listeners' abilities. A greater appreciation of cognitive contributions to processing degraded speech is critical in understanding individual differences in comprehension ability, variability in the efficacy of assistive devices, and guiding rehabilitation approaches to reducing listening effort and facilitating communication.

Age-related differences in auditory cortex activity during spoken word recognition

“…Indeed, prior to conducting this study, we expected to observe increased activity (e.g., in prefrontal cortex) for older adults relative to young adults, reflecting top-down compensation for reduced auditory sensitivity. Such activity would be consistent with increased cognitive demand during speech perception in listeners with hearing loss or other acoustic challenge (Pichora-Fuller et al, 2016;Peelle, 2018) . Although we were somewhat surprised not to see this, in retrospect, perhaps it would be expected.…”

Degree of Language Experience Modulates Visual Attention to Visible Speech and Iconic Gestures During Clear and Degraded Speech Comprehension

“…In everyday conversational contexts, we often communicate in challenging or adverse listening conditions. These listener‐related challenges can emerge because of external factors, such as noise (Peelle, ), but also because of internal factors, such as when communicating in a non‐native language (Lecumberri, Cooke, & Cutler, ). Everyday conversational contexts are often multimodal and can include auditory inputs, such as speech, but also visual input, such as visible speech and gestures.…”

Modulation of the primary auditory thalamus when recognising speech in noise

“…Speech-in-noise recognition abilities are thought to rely (i) on additional cognitive resources that are recruited when recognising speech-in-noise (reviewed in Peelle, 2018) and (ii) on the fidelity of speech sound representation in brainstem nuclei, as measured by auditory brainstem response recordings (reviewed in . For example, studies investigating speech-in-noise recognition at the level of the cerebral cortex found networks that include areas pertaining to linguistic, attentional, working memory, and motor planning (Bishop and Miller, 2008;Salvi et al, 2002;Scott et al, 2004;Wong et al, 2008).…”

“…For example, studies investigating speech-in-noise recognition at the level of the cerebral cortex found networks that include areas pertaining to linguistic, attentional, working memory, and motor planning (Bishop and Miller, 2008;Salvi et al, 2002;Scott et al, 2004;Wong et al, 2008). These results suggest that during speech recognition in challenging listening conditions additional cerebral cortex regions are recruited that likely complement the processing of sound in the core speech network (reviewed in Peelle, 2018). The present study showed that besides the additional cerebral cortex region recruitment, a specific part of the sensory pathway is also modulated during speech-in-noise recognition, the left vMGB.…”

“…Understanding speech in noise is a complex task that involves both sensory and cognitive processes (Adank, 2012;Alavash et al, 2019;Best et al, 2007;Bregman, 1994;Brokx and Noteboom, 1982;Bronkhorst, 2015;Darwin and Hukin, 2000;Moore et al, 1985;Parikh and Loizou, 2005;Peelle, 2018;Sayles and Winter, 2008;Shinn-Cunningham and Best, 2008;Song et al, 2010). Difficulties in understanding speech in noise can occur in agerelated hearing impairment (Schoof and Rosen, 2016), as well as in developmental disorders like autism spectrum disorder (Alca ntara et al, 2004), auditory processing disorder (Iliadou et al, 2017), or developmental dyslexia (Chandrasekaran et al, 2009;Ziegler et al, 2009).…”