The intraparietal sulcus governs multisensory integration of audiovisual information based on task difficulty

Abstract

Object recognition benefits maximally from multimodal sensory input when stimulus presentation is noisy, or degraded. Whether this advantage can be attributed specifically to the extent of overlap in object-related information, or rather, to object-unspecific enhancement due to the mere presence of additional sensory stimulation, remains unclear. Further, the cortical processing differences driving increased multisensory integration (MSI) for degraded compared with clear information remain poorly understood. Here, two consecutive studies first compared behavioral benefits of audio-visual overlap of object-related information, relative to conditions where one channel carried information and the other carried noise. A hierarchical drift diffusion model indicated performance enhancement when auditory and visual object-related information was simultaneously present for degraded stimuli. A subsequent fMRI study revealed visual dominance on a behavioral and neural level for clear stimuli, while degraded stimulus processing was mainly characterized by activation of a frontoparietal multisensory network, including IPS. Connectivity analyses indicated that integration of degraded object-related information relied on IPS input, whereas clear stimuli were integrated through direct information exchange between visual and auditory sensory cortices. These results indicate that the inverse effectiveness observed for identification of degraded relative to clear objects in behavior and brain activation might be facilitated by selective recruitment of an executive cortical network which uses IPS as a relay mediating crossmodal sensory information exchange.

Audio–visual and olfactory–visual integration in healthy participants and subjects with autism spectrum disorder

“…The commonalities of the models reveal a driving input of bimodal stimulation to the IPS, which in turn exerts a modulatory effect on the reciprocal connections between the respective primary sensory regions (the amygdala and the visual cortex in olfactory–visual integration, or the STG and the visual cortex in audio–visual integration). With regard to audio–visual integration, our results are consistent with other evidence showing that the IPS exerts a top‐down control on the information exchange between the primary sensory regions (Regenbogen et al, ). Our study also corroborates the notion of the IPS playing a role in olfactory–visual integration (Gottfried & Dolan, ; Seubert et al, ), suggesting that, analogous to its role in audiovisual integration, the IPS is involved in a top‐down control for the processing of inputs from the primary sensory regions (Bressler, Tang, Sylvester, Shulman, & Corbetta, ; Hopfinger, Buonocore, & Mangun, ).…”

“…This indicates that the visual enhancement of other sensory processes is central to the understanding of sensory interaction, supporting the Colavita visual dominance effect (Colavita, ) and a resulting shift of attention toward the visual domain (Posner, Nissen, & Klein, ). For instance, in case of clear visual stimuli, the visual processing robustly mediates the processing of multisensory information (Regenbogen et al, ). Thus, our results suggest that the information exchange between the visual cortex and the auditory cortex represents an earlier processing phase of MSI, which has been already observed in audio–visual integration studies (Foxe & Schroeder, ; Kayser, Logothetis, & Panzeri, ; Lakatos, Chen, O'Connell, Mills, & Schroeder, ).…”

“…Thus, it is by combining information from complex, cross‐modal stimuli, that the human brain produces unitary perception (Stein & Stanford, ). The inferior parietal sulcus (IPS) and the superior temporal sulcus (STS) have been found to play key roles in mediating information exchange between the primary sensory cortices to facilitate conclusions regarding the identity of particular objects (Beauchamp, Lee, Argall, & Martin, ; Fairhall & Macaluso, ; Grefkes, Weiss, Zilles, & Fink, ; Regenbogen et al, ; Sereno & Huang, ).…”

“…The inferior parietal sulcus (IPS) and the superior temporal sulcus (STS) have been found to play key roles in Jessica Freiherr and Natalya Chechko should be considered joint senior authors. mediating information exchange between the primary sensory cortices to facilitate conclusions regarding the identity of particular objects (Beauchamp, Lee, Argall, & Martin, 2004;Fairhall & Macaluso, 2009;Grefkes, Weiss, Zilles, & Fink, 2002;Regenbogen et al, 2017;Sereno & Huang, 2014).…”

“…According to this so-called principle of inverse effectiveness, the combination of weak stimuli, which are not effective on their own, far exceeds the sum of their individual responses, thus producing the greatest enhancement in MSI through a superadditive combination (Nagy, Eördegh, Paróczy, Márkus, & Benedek, 2006;Stein, Stanford, & Rowland, 2014). In audiovisual integration studies, for instance, the presentation of degraded auditory stimuli has been found to enable MSI (Regenbogen et al, 2017). In humans, nontrigeminal odors produce a relatively weak subjective experience compared to other senses such as hearing or seeing (Licon, Manesse, Dantec, Fournel, & Bensafi, 2018;Moessnang et al, 2013).…”