Multisensory Interactions within Human Primary Cortices Revealed by BOLD Dynamics

Abstract

Whether signals from different sensory modalities converge and interact within primary cortices in humans is unresolved, despite emerging evidence in animals. This is partially because of debates concerning the appropriate analyses of functional magnetic resonance imaging (fMRI) data in response to multisensory phenomena. Using event-related fMRI, we observed that simple auditory stimuli (noise bursts) activated primary visual cortices and that simple visual stimuli (checkerboards) activated primary auditory cortices, indicative of multisensory convergence. Moreover, analyses of blood oxygen level-dependent response dynamics revealed facilitation of hemodynamic response peak latencies and slopes for multisensory auditory-visual stimuli versus either unisensory condition, indicative of multisensory interactions within primary sensory cortices. Neural processing at the lowest cortical levels can be modulated by interactions between the senses. Temporal information in fMRI data can reveal these modulations and overcome analytic and interpretational challenges of more traditional procedures. In addition to providing an essential translational link with animal models, these results suggest that longstanding notions of cortical organization need to be revised to include multisensory interactions as an inherent component of functional brain organization.

Multisensory anatomical pathways

“…As mentioned earlier (see also other chapters in this issue), many human studies conducted with EEG or fMRI have revealed multisensory interactions in low-level cortices such as V1 (see Ghazanfar and Schroeder, 2006 for a review and Martuzzi et al, 2007;Romei et al, 2007Romei et al, , 2008 for recent evidences) and/or at a latency that strongly suggests that such phenomena are supported by the heteromodal connections directly linking areas of different sensory modalities. For example, the visual information derived from the lip movements during speech processing can affect directly the responses recorded in the auditory cortex (Besle et al, 2004).…”

“…The notion that multisensory integration is restricted to higherorder areas has recently been challenged by human and animal studies that have revealed that crossmodal interactions can occur in unisensory areas at very low levels of cortical processing (Buchel et al, 1998;Calvert et al, 1999Calvert et al, , 2001Macaluso et al, 2000;Schroeder et al, 2001;Amedi et al, 2002;Ghazanfar et al, 2005;Kriegstein et al, 2005;Miller and D'Esposito, 2005;Watkins et al, 2006;Martuzzi et al, 2007;Kayser et al, 2007Kayser et al, , 2008Romei et al, 2007Romei et al, , 2008Wang et al, 2008) and more importantly at very short latencies (Giard and Peronnet, 1999;Foxe et al, 2000;Molholm et al, 2002;Murray et al, 2005;Senkowski et al, 2007;Sperdin et al, 2009). Such a fast timing of multisensory interactions rule out an origin in the multisensory areas mediated through backward projections, and instead favor direct heteromodal connections.…”

“…Considering that only the peripheral visual field representation of V1 receives significant projections from the auditory cortex, such congruency in the spatial features may serve to facilitate gaze orienting, and consequently the relocation of foveal vision to peripheral locations in the visual field. However, it should also be mentioned that facilitative effects have been observed in humans when stimuli were centrally presented in both auditory and visual modalities (Giard and Peronnet, 1999;Molholm et al, 2002;Martuzzi et al, 2007). Ethologically, a role in alertness for dangerous stimuli is highly probable, an interpretation that can also be attributed to the visuo-somatosensory projections: the specific link between the FST visual complex and the representation of the face in the somatosensory cortex could contribute to phenomena of avoidance of a ''dangerous" stimulus which may hit the body Graziano, 2003, 2004).…”

Projection from Visual Areas V2 and Prostriata to Caudal Auditory Cortex in the Monkey

“…In humans, fMRI studies demonstrated activation of the caudal superior temporal plane during tasks that required viewing of simple moving visual stimuli (Martuzzi et al 2007;Antal et al 2008), visual identification of graphically presented phonemes (van Atteveldt et al 2004;Bernstein et al 2008), and viewing speakers' facial movements during speech perception (Calvert 2001;Olson et al 2002;Calvert and Campbell 2003;Wright et al 2003;Pekkola et al 2006). In monkeys, visual stimuli ranging from naturalistic scenes to flashes effectively modulated auditory responses in auditory cortex (Kayser et al 2007).…”

Methods for Determining Frequency- and Region-Dependent Relationships Between Estimated LFPs and BOLD Responses in Humans

“…There were several bases for this choice. A passive protocol avoided contributions of task-and motor-related activity that might in turn influence activations within visual cortices (e.g., Martuzzi et al 2006Martuzzi et al , 2007van Atteveldt et al 2007). We also opted here for a visual paradigm as these cortical regions (in particular primary visual cortex) are larger than their auditory counterparts and generally follow anatomical landmarks, making activity within them easier to disambiguate with both fMRI and EEG methods.…”