How automatic are crossmodal correspondences?

Spence, Charles; Deroy, Ophelia

doi:10.1016/j.concog.2012.12.006

Cited by 127 publications

(118 citation statements)

References 128 publications

(191 reference statements)

Supporting

Mentioning

103

Contrasting

Order By: Relevance

“…The findings of this experiment support the conclusions drawn by Navarra Alsius, SotoFaraco and Spence (2010) and Spence and Deroy (2013), who argued that the automaticity of audiovisual integration depends on the specific demands of a given task. With a low processing load, the results are consistent with the original McGurk findings.…”

Section: Discussionsupporting

confidence: 90%

Searching for a talking face: The effect of degrading the auditory signal.

Stacey¹,

Murphy²,

Sumner³

et al. 2014

Journal of Experimental Psychology: Human Perception and Perfor

View full text Add to dashboard Cite

Previous research (e.g., McGurk & MacDonald, 1976) suggests that faces and voices are bound automatically, but recent evidence suggests that attention is involved in a task of searching for a talking face (Alsius & Soto-Faraco, 2011). We hypothesized that the processing demands of the stimuli may affect the amount of attentional resources required, and investigated what effect degrading the auditory stimulus had on the time taken to locate a talking face. Twenty participants were presented with between 2 and 4 faces articulating different sentences, and had to decide which of these faces matched the sentence that they heard. The results showed that in the least demanding auditory condition (clear speech in quiet), search times did not significantly increase when the number of faces increased. However, when speech was presented in background noise or was processed to simulate the information provided by a cochlear implant, search times increased as the number of faces increased. Thus, it seems that the amount of attentional resources required vary according to the processing demands of the auditory stimuli, and when processing load is increased then faces need to be individually attended to in order to complete the task. Based on these results we would expect cochlear-implant users to find the task of locating a talking face more attentionally demanding than normal hearing listeners.

show abstract

Section: Discussionsupporting

confidence: 90%

Searching for a talking face: The effect of degrading the auditory signal.

Stacey¹,

Murphy²,

Sumner³

et al. 2014

Journal of Experimental Psychology: Human Perception and Perfor

View full text Add to dashboard Cite

show abstract

“…Previous psychophysical studies have reported similar results among nonmusicians, indicating that the identification of cross-modal correspondences between pitch height and the height of a visual stimulus may be innate (13). In contrast, a recent study by our group investigating the audiotactile integration (14) indicated that similar rules binding auditory and tactile stimuli were not intrinsically incorporated by the nonmusicians, but that musical expertise was essential for reaching correct identification of audiotactile incongruences.…”

Section: Discussionmentioning

confidence: 55%

Musical expertise is related to altered functional connectivity during audiovisual integration

Paraskevopoulos

Kraneburg

Herholz

et al. 2015

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

The present study investigated the cortical large-scale functional network underpinning audiovisual integration via magnetoencephalographic recordings. The reorganization of this network related to long-term musical training was investigated by comparing musicians to nonmusicians. Connectivity was calculated on the basis of the estimated mutual information of the sources' activity, and the corresponding networks were statistically compared. Nonmusicians' results indicated that the cortical network associated with audiovisual integration supports visuospatial processing and attentional shifting, whereas a sparser network, related to spatial awareness supports the identification of audiovisual incongruences. In contrast, musicians' results showed enhanced connectivity in regions related to the identification of auditory pattern violations. Hence, nonmusicians rely on the processing of visual clues for the integration of audiovisual information, whereas musicians rely mostly on the corresponding auditory information. The large-scale cortical network underpinning multisensory integration is reorganized due to expertise in a cognitive domain that largely involves audiovisual integration, indicating long-term training-related neuroplasticity.functional connectivity | MEG | multisensory integration | cortical plasticity | musical training M ultisensory integration is of such importance for our understanding of the surrounding world that its cortical correlates interact with most of the neocortical regions, including the ones traditionally considered as unisensory (1). The cortical areas that are usually referred to as multisensory include the superior temporal sulcus, the intraparietal sulcus, and the frontal cortex (2). Nevertheless, recent evidence suggests that multisensory perception engages more widespread areas than what the classic modular approaches have so far assumed (1). Moreover, audiovisual integration emerges from a dynamic interplay of distributed regions operating in large-scale networks (3).The amount of shared information within these large-scale networks of distributed neuronal groups conceptualizes the notion of functional connectivity (4). Recent methodological advances allow the identification of networks that emerge from whole-brain analyses of neuroimaging data such as functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) (5) and provide fertile ground for studying the functional connectivity patterns of higher cognitive processes. These kinds of networks are modeled as graphs composed of nodes, which represent the cortical areas contributing to the network, and by edges, which represent the connections between the nodes. Each network has specific attributes that quantify connectivity organization (6). These graphs depict the functional connectome of the respective cognitive processes. Recent evidence suggests that functional connectivity networks may be reorganized by factors mediating neuroplasticity such as learning and development (7), altering information processing ...

show abstract

“…In most, if not all, of the studies that have been reviewed here, the fact that the music was relevant to the wine tasting experience was made apparent to the participants/tasters. It must remain an open question, therefore, as to whether for any music that just so happens to be playing incidentally, similar crossmodal effects would be observed or whether instead these kinds of effects require the taster to actively link their tasting experience with what they are hearing ( [81]; see also [82,83], on the automaticity of crossmodal correspondences).…”

Section: Attention and Crossmodal Correspondencesmentioning

confidence: 99%

Wine and music (II): can you taste the music? Modulating the experience of wine through music and sound

Spence

Wang

2015

Flavour

Self Cite

View full text Add to dashboard Cite

A growing body of scientific evidence now shows that what people taste when evaluating a wine, and how much they enjoy the experience, can be influenced by the music that happens to be playing at the same time. It has long been known that what we hear can influence the hedonic aspects of tasting. However, what the latest research now shows is that by playing the "right" music one can also impact specific sensory-discriminative aspects of tasting as well. Music has been shown to influence the perceived acidity, sweetness, fruitiness, astringency, and length of wine. We argue against an account of such results in terms of synaesthesia, or "oenesthesia," as some have chosen to call it. Instead, we suggest that attention, directed via the crossmodal correspondences that exist between sound and taste (in the popular meaning of the term, i.e., flavor), can modify (perhaps enhance, or certainly highlight when attended, or suppress when unattended) certain elements in the complex tasting experience that is drinking wine. We also highlight the likely role played by any change in the mood or emotional state of the person listening to the music on taste/aroma perception as well. Finally, we highlight how the crossmodal masking of sweetness perception may come into effect if the music happens to be too loud (a form of crossmodal sensory masking). Taken together, the evidence reviewed here supports the claim that, strange though it may seem, what we hear (specifically in terms of music) really can change our perception of the taste of wine, not to mention how much we enjoy the experience. Several plausible mechanisms that may underlie such crossmodal effects are outlined.

show abstract

How automatic are crossmodal correspondences?

Cited by 127 publications

References 128 publications

Searching for a talking face: The effect of degrading the auditory signal.

Searching for a talking face: The effect of degrading the auditory signal.

Musical expertise is related to altered functional connectivity during audiovisual integration

Wine and music (II): can you taste the music? Modulating the experience of wine through music and sound

Contact Info

Product

Resources

About