No abstract

Although the P3b component of the event‐related brain potential is one of the most widely studied components, its underlying generators are not currently well understood. Recent theories have suggested that the P3b is triggered by phasic activation of the locus‐coeruleus norepinephrine (LC‐NE) system, an important control center implicated in facilitating optimal task‐relevant behavior. Previous research has reported strong correlations between pupil dilation and LC activity, suggesting that pupil diameter is a useful indicator for ongoing LC‐NE activity. Given the strong relationship between LC activity and pupil dilation, if the P3b is driven by phasic LC activity, there should be a robust trial‐to‐trial relationship with the phasic pupillary dilation response (PDR). However, previous work examining relationships between concurrently recorded pupillary and P3b responses has not supported this. One possibility is that the relationship between the measures might be carried primarily by either inter‐individual (i.e., between‐participant) or intra‐individual (i.e., within‐participant) contributions to coupling, and prior work has not systematically delineated these relationships. Doing so in the current study, we do not find evidence for either inter‐individual or intra‐individual relationships between the PDR and P3b responses. However, baseline pupil dilation did predict the P3b. Interestingly, both the PDR and P3b independently predicted inter‐individual and intra‐individual variability in decision response time. Implications for the LC‐P3b hypothesis are discussed.

There is an apparent disparity between the fields of cognitive audiology and cognitive electrophysiology as to how linguistic context is used when listening to perceptually challenging speech. To gain a clearer picture of how listening effort impacts context use, we conducted a preregistered study to simultaneously examine electrophysiological, pupillometric, and behavioral responses when listening to sentences varying in contextual constraint and acoustic challenge in the same sample. Participants (N = 44) listened to sentences that were highly constraining and completed with expected or unexpected sentence-final words ("The prisoners were planning their escape/party") or were low-constraint sentences with unexpected sentence-final words ("All day she thought about the party"). Sentences were presented either in quiet or with +3 dB SNR background noise. Pupillometry and EEG were simultaneously recorded and subsequent sentence recognition and word recall were measured. While the N400 expectancy effect was diminished by noise, suggesting impaired real-time context use, we simultaneously observed a beneficial effect of constraint on subsequent recognition memory for degraded speech. Importantly, analyses of trial-to-trial coupling between pupil dilation and N400 amplitude showed that when participants' showed increased listening effort (i.e., greater pupil dilation), there was a subsequent recovery of the N400 effect, but at the same time, higher effort was related to poorer subsequent sentence recognition and word recall. Collectively, these findings suggest divergent effects of acoustic challenge and listening effort on context use: while noise impairs the rapid use of context to facilitate lexical semantic processing in general, this negative effect is attenuated when listeners show increased effort in response to noise. However, this effort-induced reliance on context for online word processing comes at the cost of poorer subsequent memory.

Objectives. Everyday speech understanding frequently occurs in perceptually demanding environments, for example due to background noise and normal age-related hearing loss. The resulting degraded speech signals increase listening effort, which gives rise to negative downstream effects on subsequent memory and comprehension, even when speech is intelligible. In two experiments, we explored whether the presentation of realistic assistive text captioned speech offsets the negative effects of background noise and hearing impairment on multiple measures of speech memory.Design. In Experiment 1, young normal hearing adults (N = 48) listened to sentences for immediate recall and delayed recognition memory. Speech was presented in quiet or in two levels of background noise. Sentences were either presented as speech only or as text captioned speech. Thus, the experiment followed a 2 (caption vs no caption) x 3 (no noise, +7 dB SNR, +3 dB SNR) within-subjects design. In Experiment 2, a group of older adults (age range : 61 – 80, N = 31), with varying levels of hearing acuity completed the same experimental task as in Experiment 1. For both experiments, immediate recall, recognition memory accuracy, and recognition memory confidence were analyzed via general(ized) linear mixed effects models. In addition, we examined individual differences as a function of hearing acuity in Experiment 2.Results. In Experiment 1, we found that the presentation of realistic text-captioned speech in young normal-hearing listeners improved immediate recall, delayed recognition memory accuracy, and memory confidence compared to speech alone. Moreover, text captions attenuated the negative effects of background noise on all speech memory outcomes. In Experiment 2, we replicated the same pattern of results in a sample of older adults with varying levels of hearing acuity. Moreover, we showed that the negative effects of hearing loss on speech memory in older adulthood were attenuated by the presentation of text captions.Conclusion. Collectively, these findings suggest that listeners can rapidly integrate text and speech, and that the simultaneous presentation of text can offset the negative effects of effortful listening on speech memory.

No abstract

There is a disparity between the fields of cognitive audiology and cognitive electrophysiology as to how linguistic context is utilized when listening to perceptually challenging speech. To gain a clearer picture of how listening effort impacts context use, we conducted a pre-registered study to simultaneously examine electrophysiological, pupillometric, and behavioral responses when listening to sentences varying in contextual constraint and acoustic challenge in the same sample. Participants (N = 44) listened to sentences that were highly constraining and completed with expected or unexpected sentence-final words (“The prisoners were planning their escape/party”) or were low-constraint sentences with unexpected sentence-final words (“All day she thought about the party”). Sentences were presented either in quiet or with +3 dB SNR background noise. Pupillometry and EEG were simultaneously recorded and subsequent sentence recognition and word recall were measured. While the N400 expectancy effect was diminished by noise, suggesting impaired real-time context use, we simultaneously observed a beneficial effect of constraint on subsequent recognition memory for degraded speech. Importantly, analyses of trial-to-trial coupling between pupil dilation and N400 amplitude showed that when participants’ showed increased listening effort (i.e., greater pupil dilation), there was a subsequent recovery of the N400 effect, but at the same time, higher effort was related to poorer subsequent sentence recognition and word recall. Collectively, these findings suggest divergent effects of acoustic challenge and listening effort on context use: while noise impairs the rapid use of context to facilitate lexical semantic processing in general, this negative effect is attenuated when listeners increase effort in response to noise. However, this effort-induced reliance on context for online word processing comes at the cost of poorer subsequent memory.

Objectives. Everyday speech understanding frequently occurs in perceptually demanding environments, for example due to background noise and normal age-related hearing loss. The resulting degraded speech signals increase listening effort, which gives rise to negative downstream effects on subsequent memory and comprehension, even when speech is intelligible.In two experiments, we explored whether the presentation of realistic assistive text captioned speech offsets the negative effects of background noise and hearing impairment on multiple measures of speech memory. Design.In Experiment 1, young normal hearing adults (N = 48) listened to sentences for immediate recall and delayed recognition memory. Speech was presented in quiet or in two levels of background noise. Sentences were either presented as speech only or as text captioned speech. Thus, the experiment followed a 2 (caption vs no caption) x 3 (no noise, +7 dB SNR, +3 dB SNR) within-subjects design. In Experiment 2, a group of older adults (age range : 61 -80, N = 31), with varying levels of hearing acuity completed the same experimental task as in Experiment 1. For both experiments, immediate recall, recognition memory accuracy, and recognition memory confidence were analyzed via general(ized) linear mixed effects models. In addition, we examined individual differences as a function of hearing acuity in Experiment 2. Results.In Experiment 1, we found that the presentation of realistic text-captioned speech in young normal-hearing listeners improved immediate recall, delayed recognition memory accuracy, and memory confidence compared to speech alone. Moreover, text captions attenuated the negative effects of background noise on all speech memory outcomes. In Experiment 2, we replicated the same pattern of results in a sample of older adults with varying levels of hearing acuity. Moreover, we showed that the negative effects of hearing loss on speech memory in older adulthood were attenuated by the presentation of text captions. Conclusion.Collectively, these findings suggest that listeners can rapidly integrate text and speech, and that the simultaneous presentation of text can offset the negative effects of effortful listening on speech memory.

Purpose: Previous studies have suggested that the negative effects of acoustic challenge on speech memory can be attenuated with assistive text captions, particularly among older adults with hearing impairment. However, no studies have systematically examined the effects of text-captioning errors, which are common in automated speech recognition (ASR) systems. Method: In two experiments, we examined memory for text-captioned speech (with and without background noise) when captions had no errors (control) or had one of three common ASR errors: substitution, deletion, or insertion errors. Results: In both Experiment 1 (young adults with normal hearing) and Experiment 2 (older adults with varying hearing acuity), we observed similar additive effects of caption errors and background noise, such that increased background noise and the presence of captioning errors negatively impacted memory outcomes. Notably, the negative effects of captioning errors were largest among older adults with increased hearing thresholds, suggesting that older adults with hearing loss may show an increased reliance on text captions compared to adults with normal hearing. Conclusion: Our findings show that even a single-word error can be deleterious to memory for text-captioned speech, especially in older adults with hearing loss. Therefore, to produce the greatest benefit to memory, it is crucial that text captions are accurate.