Human brain activity associated with audiovisual perception and attention

Multisensory Congruency as a Mechanism for Attentional Control over Perceptual Selection

“…Although there are studies reporting that audiovisual stimulus combination is mandatory (Driver and Spence, 1998;Guttman et al, 2005), this is not a general view (Shiffrin and Grantham, 1974;Bonnel and Hafter, 1998;SotoFaraco et al, 2005;Alais et al, 2006;Hupé et al, 2008). Our experiments address this by using perceptually ambiguous competing auditory and visual stimuli, thereby dissociating attention and stimulation to reveal that active attention to both modalities promotes audiovisual combination, in line with other recent studies (Calvert et al, 1997;Gutfreund et al, 2002;Degerman et al, 2007;Mozolic et al, 2008).…”

“…Although there is support for the idea that auditory and visual attention are processed separately (Shiffrin and Grantham, 1974;Bonnel and Hafter, 1998;Soto-Faraco et al, 2005;Alais et al, 2006;Pressnitzer and Hupé, 2006;Hupé et al, 2008), our findings support the neurophysiological literature (Calvert et al, 1997;Gutfreund et al, 2002;Shomstein and Yantis, 2004;Amedi et al, 2005;Brosch et al, 2005;Budinger et al, 2006;Degerman et al, 2007;Lakatos et al, 2007Lakatos et al, , 2008Shinn-Cunningham, 2008) that the mechanisms mediating multisensory attentional control are intimately linked. To understand these seemingly disparate results, note first that psychophysical studies finding separate processing, focused on spatial attention, as opposed to our study.…”

“…Our data suggest a functional role for neurons recently found in human posterior parietal, superior prefrontal, and superior temporal cortices that combine voluntarily initiated attentional functions across sensory modalities (Gutfreund et al, 2002;Shomstein and Yantis, 2004;Degerman et al, 2007). We suggest that when the brain can detect a rhythm in a task, attention feeds back to unisensory cortex to enforce coherent and amplified output of the matching perceptual interpretation.…”

“…3d), our pilot work (supplemental Fig. 5d, available at www.jneurosci.org as supplemental material) and the available literature suggest that attention must be engaged to promote cross-modal interactions (Calvert et al, 1997;Gutfreund et al, 2002;Degerman et al, 2007;Mozolic et al, 2008;for review, see Shinn-Cunningham, 2008). Nevertheless, although a systematic investigation of temporal offset and automation for the cross-modal effects goes beyond the scope of the present paper, it is interesting to note that the underlying rhythm mechanism for our rhythm-based effect may be different from the mechanism underlying automatically occurring coincidence-based auditory-visual interactions (such as in the reported enhanced perception of visual change by a coincident auditory tone pip) (van der Burg et al, 2008).…”

“…Many neurons in human posterior parietal and superior prefrontal cortices are involved in voluntary attentional shifts between vision and audition (Shomstein and Yantis, 2004), and attention to audiovisual feature combinations produces stronger activity in the superior temporal cortices than does attention to only auditory or visual features (Degerman et al, 2007). It has also been shown that auditory cortex can be profoundly engaged in processing nonauditory signals, particularly when those signals are being attended (for review, see Shinn-Cunningham, 2008).…”