Dissociated Roles of the Inferior Frontal Gyrus and Superior Temporal Sulcus in Audiovisual Processing: Top-Down and Bottom-Up Mismatch Detection

Abstract

Visual inputs can distort auditory perception, and accurate auditory processing requires the ability to detect and ignore visual input that is simultaneous and incongruent with auditory information. However, the neural basis of this auditory selection from audiovisual information is unknown, whereas integration process of audiovisual inputs is intensively researched. Here, we tested the hypothesis that the inferior frontal gyrus (IFG) and superior temporal sulcus (STS) are involved in top-down and bottom-up processing, respectively, of target auditory information from audiovisual inputs. We recorded high gamma activity (HGA), which is associated with neuronal firing in local brain regions, using electrocorticography while patients with epilepsy judged the syllable spoken by a voice while looking at a voice-congruent or -incongruent lip movement from the speaker. The STS exhibited stronger HGA if the patient was presented with information of large audiovisual incongruence than of small incongruence, especially if the auditory information was correctly identified. On the other hand, the IFG exhibited stronger HGA in trials with small audiovisual incongruence when patients correctly perceived the auditory information than when patients incorrectly perceived the auditory information due to the mismatched visual information. These results indicate that the IFG and STS have dissociated roles in selective auditory processing, and suggest that the neural basis of selective auditory processing changes dynamically in accordance with the degree of incongruity between auditory and visual information.

Electrocorticography reveals continuous auditory and visual speech tracking in temporal and occipital cortex

“…In general, HG band responses were enhanced compared to baseline and LF responses were decreased compared to baseline and the dynamic visual speech information increased the HG response and reduced the LF response (enhanced decrease). Enhanced HG band responses and decreased LF responses compared to baseline have been previously reported to audiovisual speech with ECoG (Rhone et al, 2016;Schepers et al, 2015;Uno et al, 2015) and seem to be a general response profile in population electrophysiological responses to sensory stimulation (e.g., Scheeringa, Koopmans, van Mourik, Jensen, & Norris, 2016;Siegel, Donner, Oostenveld, Fries, & Engel, 2008). Magnitude changes in neural brain responses reported here, particularly in the broad HG range, which has been linked to neuronal spiking activity (Ray & Maunsell, 2011), might underlie BOLD response changes seen in previous fMRI studies (Mukamel et al, 2005).…”

Coupling between Theta Oscillations and Cognitive Control Network during Cross-Modal Visual and Auditory Attention: Supramodal vs Modality-Specific Mechanisms

“…Cross-modal visual and auditory attention and their underlying neural characteristics have been extensively studied. Recent studies show that the modulation of both amplitude and phase of EEG oscillation could be important mechanisms to enable cross-modal interactions [ 7 – 9 ]. In addition, a recent EEG and fMRI study showed that when there is competition between visual and auditory stimuli, requiring active endogenous control of attention, there is modulation of activity within the sensory cortices, sensorimotor areas and the default mode network (DMN) [ 10 ].…”

Neural responses to action contingency error in different cortical areas are attributable to forward prediction or sensory processing

“…The account primarily assumes the involvement of the multimodal or motor-related areas 16,29,30,44 . Indeed, the regions implicated in this account in the current study (i.e., the right TPJ, SMA, bilateral IFG, and left temporal pole) have been reported to show contingency-error responses across multiple sensory modalities 12,18,26,27,45–49 . Specifically, activation in the bilateral IFG has been implicated in sustained attention as part of the frontoparietal network 26,50 , while activation in the right SMA has been implicated in the planning of movement, motor intentions, and motor preparation 51–53 , and activation in the right TPJ in the detection of action-contingency error 21,24,26,54 .…”