Visual information about speech content from the talker’s mouth is often available before auditory information from the talker's voice. Here we examined perceptual and neural responses to words with and without this visual head start. For both types of words, perception was enhanced by viewing the talker's face, but the enhancement was significantly greater for words with a head start. Neural responses were measured from electrodes implanted over auditory association cortex in the posterior superior temporal gyrus (pSTG) of epileptic patients. The presence of visual speech suppressed responses to auditory speech, more so for words with a visual head start. We suggest that the head start inhibits representations of incompatible auditory phonemes, increasing perceptual accuracy and decreasing total neural responses. Together with previous work showing visual cortex modulation (Ozker et al., 2018b) these results from pSTG demonstrate that multisensory interactions are a powerful modulator of activity throughout the speech perception network.
The insula plays a fundamental role in a wide range of adaptive human behaviors, but its electrophysiological dynamics are poorly understood. Here we used human intracranial electroencephalographic recordings to investigate the electrophysiological properties and hierarchical organization of spontaneous neuronal oscillations within the insula. We analyzed the neuronal oscillations of the insula directly and found that rhythms in the theta and beta frequency oscillations are widespread and spontaneously present. These oscillations are largely organized along the anterior–posterior axis of the insula. Both the left and right insula showed anterior-to-posterior decreasing gradients for the power of oscillations in the beta frequency band. The left insula also showed a posterior-to-anterior decreasing frequency gradient and an anterior-to-posterior decreasing power gradient in the theta frequency band. In addition to measuring the power of these oscillations, we also examined the phase of these signals across simultaneous recording channels and found that the insula oscillations in the theta and beta bands are traveling waves. The strength of the traveling waves in each frequency was positively correlated with the amplitude of each oscillation. However, the theta and beta traveling waves were uncoupled to each other in terms of phase and amplitude, which suggested that insular traveling waves in the theta and beta bands operate independently. Our findings provide new insights into the spatiotemporal dynamics and hierarchical organization of neuronal oscillations within the insula, which, given its rich connectivity with widespread cortical regions, indicates that oscillations and traveling waves have an important role in intra- and inter-insular communication.
Experimentalists studying multisensory integration compare neural responses to multisensory stimuli with responses to the component modalities presented in isolation. This procedure is problematic for multisensory speech perception since audiovisual speech and auditory-only speech are easily intelligible but visual-only speech is not. To overcome this confound, we developed intracranial encephalography (iEEG) deconvolution. Individual stimuli always contained both auditory and visual speech, but jittering the onset asynchrony between modalities allowed for the time course of the unisensory responses and the interaction between them to be independently estimated. We applied this procedure to electrodes implanted in human epilepsy patients (both male and female) over the posterior superior temporal gyrus (pSTG), a brain area known to be important for speech perception. iEEG deconvolution revealed sustained positive responses to visual-only speech and larger, phasic responses to auditory-only speech. Confirming results from scalp EEG, responses to audiovisual speech were weaker than responses to auditory-only speech, demonstrating a subadditive multisensory neural computation. Leveraging the spatial resolution of iEEG, we extended these results to show that subadditivity is most pronounced in more posterior aspects of the pSTG. Across electrodes, subadditivity correlated with visual responsiveness, supporting a model in which visual speech enhances the efficiency of auditory speech processing in pSTG. The ability to separate neural processes may make iEEG deconvolution useful for studying a variety of complex cognitive and perceptual tasks.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.