The Effect of Combined Sensory and Semantic Components on Audio–Visual Speech Perception in Older Adults

Abstract

Previous studies have found that perception in older people benefits from multisensory over unisensory information. As normal speech recognition is affected by both the auditory input and the visual lip movements of the speaker, we investigated the efficiency of audio and visual integration in an older population by manipulating the relative reliability of the auditory and visual information in speech. We also investigated the role of the semantic context of the sentence to assess whether audio–visual integration is affected by top-down semantic processing. We presented participants with audio–visual sentences in which the visual component was either blurred or not blurred. We found that there was a greater cost in recall performance for semantically meaningless speech in the audio–visual ‘blur’ compared to audio–visual ‘no blur’ condition and this effect was specific to the older group. Our findings have implications for understanding how aging affects efficient multisensory integration for the perception of speech and suggests that multisensory inputs may benefit speech perception in older adults when the semantic content of the speech is unpredictable.

Task-Specific, Age Related Effects in the Cross-Modal Identification and Localisation of Objects

“…Audio-visual congruent targets were identified and located faster and more accurately than both incongruent (i.e., AntVtINC or AtVntINC) targets except in the localisation task where re sponse times to incongruent visual targets paired with an auditory non-target (AntV t^c ) were comparable to spatially congruent audio-visual (AtVtCON) targets again showing how visual input dominated in the object localisation task. Suprathreshold stimuli were used in the current study as a number of studies have shown that older adults demonstrate no audio-visual enhancement in tasks using semantic audio-visual stimuli where the visual and auditory components are degraded (Gordon and Allen, 2009;Maguinness et al, 2011), however, reducing the reliability of the visual stimuli may have reduced visual dominance effects found in the current study. We failed to find evidence in either the accuracy or response time data for a three-way interaction between the age group of the participants, the task they performed and the modality in which the target was presented.…”

“…However, Gordon and Allen (2009) found that older adult's performance on a speech task was not enhanced relative to younger adults when visual cues were blurred and audio speech was presented in noise. In addition, Maguinness et al (2011) reported relatively worse recall by older adults for semantically meaningless audio-visual speech, particularly when visual cues were blurred relative to non-blurred visual cues. These find ings are in contrast to what would be predicted by the principle of inverse effectiveness (Meredith and Stein, 1983) whereby multisensory enhancement is more pronounced when individual sensory inputs are less effective.…”

“…Research studies investigating the role of development on the emergence of efficient multisensory processing have used different tasks, experimental set-ups and stimuli, to assess how different age groups utilise and combine sensory information (see e.g., Barutchu et al, 2010;Brandwein et al, 2010;Gordon and Allen, 2009;Gori et al, 2008;Hillock et al, 2011;Maguinness et al, 2011;Nardini et al, 2008;Ross et al, 2011). This variation in stimuli and methodology makes it difficult to determine whether the ability to optimally combine multisensory information develops across the life-span at the same rate for different tasks or whether the developmental trajectory of this ability differs depending on the nature of the perceptual task itself (Barutchu et al, 2009;Neil et al, 2006).…”

Shifts in Audiovisual Processing in Healthy Aging

“…Greater WMC was associated with a lower (more noisy) SNR at which maximal gain was observed. Given the known decreases in WMC with age, these results provide strong evidence that cognitive decline, specifically declines in working memory, may influence task performance that is generally considered more sensory based – though it should be noted that perceptual and cognitive declines do not occur in isolation, and likely interact [96-98]. …”

Susceptibility to a multisensory speech illusion in older persons is driven by perceptual processes

“…Higher levels of noise can be tolerated when the semantic content of speech is predictable (Pichora-Fuller, 2008). Indeed when older adults cannot rely on the semantic predictability of a sentence, for example, because the sentence does not express a meaningful content, they benefit more from the addition of visual information to auditory speech than younger adults (Maguinness et al, 2011). …”