The Impact of Auditory Spectral Resolution on Listening Effort Revealed by Pupil Dilation

Abstract

Objectives This study measured the impact of auditory spectral resolution on listening effort. Systematic degradation in spectral resolution was hypothesized to elicit corresponding systematic increases in pupil dilation, consistent with the notion of pupil dilation as a marker of cognitive load. Design Spectral resolution of sentences was varied with 2 different vocoders: (1) a noise channel vocoder with a variable number of spectral channels; and (2) a vocoder designed to simulate front-end processing of a cochlear implant, including peak-picking channel selection with variable synthesis filter slopes to simulate spread of neural excitation. Pupil dilation was measured after subject-specific luminance adjustment and trial-specific baseline measures. Mixed-effects growth curve analysis was used to model pupillary responses over time. Results For both types of vocoder, pupil dilation grew with each successive degradation in spectral resolution. Within each condition, pupillary responses were not related to intelligibility scores, and the effect of spectral resolution on pupil dilation persisted even when only analyzing trials in which responses were 100% correct. Conclusions Intelligibility scores alone were not sufficient to quantify the effort required to understand speech with poor resolution. Degraded spectral resolution results in increased effort required to understand speech, even when intelligibility is at 100%. Pupillary responses were a sensitive and highly granular measurement to reveal changes in listening effort. Pupillary responses might potentially reveal the benefits of aural prostheses that are not captured by speech intelligibility performance alone, as well as the disadvantages that are overcome by increased listening effort.

Hearing Impairment and Cognitive Energy: The Framework for Understanding Effortful Listening (FUEL)

“…Kramer et al 2016, this issue, 126S-135S). Research on cognitive processing load during listening using pupillometry has shown that the pupil response during listening is sensitive to speech intelligibility (Zekveld et al 2010), type of background noise (Koelewijn et al 2012), syntactic complexity (Piquado et al 2010), auditory stimulus characteristics , degraded spectral resolution (Winn et al 2015), cognitive abilities (Zekveld et al 2011), and divided (versus focused) attention (Koelewijn et al 2014). Cardiac Responses • Two cardiac measures that may be related to listening effort are heart-rate variability (HRV) and the PEP.…”

Physiology, Psychoacoustics and Cognition in Normal and Impaired Hearing

“…This overall smaller increase in pupil dilation can be explained by the fact that these curves are normalized to a baseline of 200 ms preceding the presentation of each critical item per trial, participant and condition. Listening to degraded speech is by itself more effortful than listening to natural speech (e.g., Winn et al 2015), and therefore there is a difference in the baseline between DS and NS. These differences in the baseline can be explained by the difference in processing degraded versus natural speech, and are independent of the effects of semantic integration on lexical competition.…”

Low-frequency fine-structure cues allow for the online use of lexical stress during spoken-word recognition in spectrally degraded speech

“…The findings of the present study reinforce the need for new tools or evaluation metrics in capturing benefits or treatment effectiveness beyond the traditional measure of percentage correct for speech intelligibility. Recent studies have shown a reduction in listening effort (measured by the reaction time to the secondary task or by the change of pupil size) as the quality of the auditory signal improved beyond the asymptotic performance in a recognition task (e.g., Sarampalis et al, 2009;Pals et al, 2013;Winn et al, 2015).…”