How can audiovisual pathways enhance the temporal resolution of time-compressed speech in blind subjects?

Abstract

In blind people, the visual channel cannot assist face-to-face communication via lipreading or visual prosody. Nevertheless, the visual system may enhance the evaluation of auditory information due to its cross-links to (1) the auditory system, (2) supramodal representations, and (3) frontal action-related areas. Apart from feedback or top-down support of, for example, the processing of spatial or phonological representations, experimental data have shown that the visual system can impact auditory perception at more basic computational stages such as temporal signal resolution. For example, blind as compared to sighted subjects are more resistant against backward masking, and this ability appears to be associated with activity in visual cortex. Regarding the comprehension of continuous speech, blind subjects can learn to use accelerated text-to-speech systems for “reading” texts at ultra-fast speaking rates (>16 syllables/s), exceeding by far the normal range of 6 syllables/s. A functional magnetic resonance imaging study has shown that this ability, among other brain regions, significantly covaries with BOLD responses in bilateral pulvinar, right visual cortex, and left supplementary motor area. Furthermore, magnetoencephalographic measurements revealed a particular component in right occipital cortex phase-locked to the syllable onsets of accelerated speech. In sighted people, the “bottleneck” for understanding time-compressed speech seems related to higher demands for buffering phonological material and is, presumably, linked to frontal brain structures. On the other hand, the neurophysiological correlates of functions overcoming this bottleneck, seem to depend upon early visual cortex activity. The present Hypothesis and Theory paper outlines a model that aims at binding these data together, based on early cross-modal pathways that are already known from various audiovisual experiments on cross-modal adjustments during space, time, and object recognition.

Training of ultra-fast speech comprehension induces functional reorganization of the central-visual system in late-blind humans

“…Against the background of our model of ultra-fast speech understanding derived from previous work (Hertrich et al, 2013b), only blind, but not sighted subjects were expected to recruit the central-visual system whereas, by contrast, participants with residual or normal vision should deactivate the visual-system during listening to speech. As concerns SMA, already identified as a “bottleneck” of time-compressed spoken language perception (Vagharchakian et al, 2012), hemodynamic activation was expected (i) to reflect gains in behavioral performance and (ii) to increase with speech rate in high-performing subjects.…”

“…These findings provide tentative support for a relationship between ultra-fast speech perception and right-hemispheric V1 activation—in case the central-visual system is deprived of modality-specific afferent input. Based on our previous studies, it has been suggested that collaboration of right auditory cortex and ipsilateral primary visual area allows for the implementation of a signal-driven timing mechanism related to syllabic modulation of the speech signal (Dietrich et al, 2013; Hertrich et al, 2013a,b). More specifically, subjects able to recruit the visual cortex during spoken language comprehension appear to install an additional information channel conveying temporal cues on syllable structure to the frontal lobe which then help to trigger phonological encoding processes and, as a consequence, to support verbal working memory operations (Hertrich et al, 2013b).…”

“…Based on our previous studies, it has been suggested that collaboration of right auditory cortex and ipsilateral primary visual area allows for the implementation of a signal-driven timing mechanism related to syllabic modulation of the speech signal (Dietrich et al, 2013; Hertrich et al, 2013a,b). More specifically, subjects able to recruit the visual cortex during spoken language comprehension appear to install an additional information channel conveying temporal cues on syllable structure to the frontal lobe which then help to trigger phonological encoding processes and, as a consequence, to support verbal working memory operations (Hertrich et al, 2013b). Since the visual system seems to compensate for the temporal limitations of the auditory system under these conditions, this strategy might allow for a further increase in ultra-fast speech comprehension capabilities beyond the level that was achieved in the present training study.…”

“…Besides the classical perisylvian “language zones” of the left hemisphere [inferior frontal gyrus (IFG)/superior temporal cortex] and the supplementary motor area (SMA), blind people highly skilled in ultra-fast speech perception showed significant hemodynamic activation of right-hemispheric primary visual cortex (V1), contralateral fusiform gyrus (FG), and bilateral pulvinar (Pv) (Dietrich et al, 2013). In a recent Hypothesis and Theory paper (Hertrich et al, 2013b), an expanded model of speech perception was introduced to describe how blind subjects might use their visual system for ultra-fast speech perception. Thereby, right visual cortex enhances time-critical speech processing due to its cross-links to (i) the afferent auditory pathway (e.g., via Pv) and (ii) to frontal action-related representations (e.g., SMA, IFG).…”

Audiovisual integration: an introduction to behavioral and neuro-cognitive methods

“…This study showed that visual cues sped-up linguistic recognition in both noisy and clear listening conditions. Finally, a review and hypothesis article by Hertrich et al (2013) proposes a brain network explaining how blind individuals, on average, are capable of perceiving auditory speech at a much faster rate compared to individuals with normal vision. Together, these articles will help constrain dynamic and neural-based theories regarding temporal aspects of audiovisual speech perception.…”