Perception drives production across sensory modalities: A network for sensorimotor integration of visual speech

Abstract

Sensory information is critical for movement control, both for defining the targets of actions and providing feedback during planning or ongoing movements. This holds for speech motor control as well, where both auditory and somatosensory information have been shown to play a key role. Recent clinical research demonstrates that individuals with severe speech production deficits can show a dramatic improvement in fluency during online mimicking of an audiovisual speech signal suggesting the existence of a visuomotor pathway for speech motor control. Here we used fMRI in healthy individuals to identify this new visuomotor circuit for speech production. Participants were asked to perceive and covertly rehearse nonsense syllable sequences presented auditorily, visually, or audiovisually. The motor act of rehearsal, which is prima facie the same whether or not it is cued with a visible talker, produced different patterns of sensorimotor activation when cued by visual or audiovisual speech (relative to auditory speech). In particular, a network of brain regions including the left posterior middle temporal gyrus and several frontoparietal sensorimotor areas activated more strongly during rehearsal cued by a visible talker versus rehearsal cued by auditory speech alone. Some of these brain regions responded exclusively to rehearsal cued by visual or audiovisual speech. This result has significant implications for models of speech motor control, for the treatment of speech output disorders, and for models of the role of speech gesture imitation in development.

Syntax-sensitive regions of Broca’s area and the posterior temporal lobe are differentially recruited by production and perception

“…In the present study, we decided to test the hypothesis offered in by assessing syntactic perception and production in the brain in the same study. We used simple linguistic materials consisting of sequences of two-word jabberwocky structures (e.g., this pand these clopes), and a "perceive & rehearse" paradigm used in previous studies to localize both speech production and perception (Buchsbaum et al 2001;Hickok et al 2003;Okada and Hickok 2009;Isenberg et al 2012;Venezia et al 2016). We expected that language-selective subregions of Broca's area and the PTL (identified in individual subjects) would exhibit increased activation for syntactically structured materials relative to unstructured word and nonword lists, consistent with previous findings.…”

Lower Beta: A Central Coordinator of Temporal Prediction in Multimodal Speech

“…In contrast, the hierarchical latency effect that depends upon the saliency of visual content suggests a super-additive effect of visual information that may trigger motor representations of corresponding syllable production. Further, another fMRI study showed that rehearsing nonsense sequences of syllables previously presented in audiovisual modality activated the left posterior middle temporal gyrus and some fronto-parietal sensorimotor areas more strongly than when previously perceived in auditory modality only (Venezia et al, 2016 ). Taken together, these results contributed to show how visual information impacts both “when” and “what” predictions types in multimodal speech processing.…”

Biologically-Inspired Computational Neural Mechanism for Human Action/activity Recognition: A Review

“…Some regions such as the RSC and PPA do not respond to the parallax [166]. Venezia et al (2016) analyzed the sensorimotor integration of visual speech through perception. The study used fMRI on healthy individuals to identify new visiomotor circuit speech production [167].…”

“…Venezia et al (2016) analyzed the sensorimotor integration of visual speech through perception. The study used fMRI on healthy individuals to identify new visiomotor circuit speech production [167]. A research on how visual motion affects neural receptive fields and fMRI response amplitudes was carried out to examine visual motion neural position preferences in the hierarchy of visual field maps using high-field fMRI and population receptive field.…”