The costs (and benefits) of effortful listening on context processing: A simultaneous electrophysiology, pupillometry, and behavioral study

Abstract

There is a disparity between the fields of cognitive audiology and cognitive electrophysiology as to how linguistic context is utilized when listening to perceptually challenging speech. To gain a clearer picture of how listening effort impacts context use, we conducted a pre-registered study to simultaneously examine electrophysiological, pupillometric, and behavioral responses when listening to sentences varying in contextual constraint and acoustic challenge in the same sample. Participants (N = 44) listened to sentences that were highly constraining and completed with expected or unexpected sentence-final words (“The prisoners were planning their escape/party”) or were low-constraint sentences with unexpected sentence-final words (“All day she thought about the party”). Sentences were presented either in quiet or with +3 dB SNR background noise. Pupillometry and EEG were simultaneously recorded and subsequent sentence recognition and word recall were measured. While the N400 expectancy effect was diminished by noise, suggesting impaired real-time context use, we simultaneously observed a beneficial effect of constraint on subsequent recognition memory for degraded speech. Importantly, analyses of trial-to-trial coupling between pupil dilation and N400 amplitude showed that when participants’ showed increased listening effort (i.e., greater pupil dilation), there was a subsequent recovery of the N400 effect, but at the same time, higher effort was related to poorer subsequent sentence recognition and word recall. Collectively, these findings suggest divergent effects of acoustic challenge and listening effort on context use: while noise impairs the rapid use of context to facilitate lexical semantic processing in general, this negative effect is attenuated when listeners increase effort in response to noise. However, this effort-induced reliance on context for online word processing comes at the cost of poorer subsequent memory.

The persistence of distraction: The hidden costs of intermittent multitasking.

“…An eye tracker was utilized to measure the PDR of participants across conditions. PDR has been found to increase with increases in cognitive load (e.g., Ahern & Beatty, 1979; Beatty & Lucero-Wagoner, 2000; Kahneman & Beatty, 1966; Silcox & Payne, 2021; Zekveld & Kramer, 2014). Three participant’s eye-tracking data were removed due to a recording malfunction, leaving 43 participants.…”

“…Additionally, we collected eye-tracking measures to determine if there were changes in pupil diameter and the visual scanning patterns following a multitasking episode. The pupil dilation response (PDR) reflects increases in arousal and effort in response to increases in cognitive demand (e.g., Ahern & Beatty, 1979; Beatty & Lucero-Wagoner, 2000; Kahneman & Beatty, 1966; Silcox & Payne, 2021; Zekveld & Kramer, 2014). The PDR has been linked to phasic activity in the locus coeruleus norepinephrine system (Joshi & Gold, 2020; LoTemplio et al, 2021).…”