Does crossmodal correspondence modulate the facilitatory effect of auditory cues on visual search?

Abstract

The "pip-and-pop effect" refers to the facilitation of search for a visual target (a horizontal or vertical bar whose color changes frequently) among multiple visual distractors (tilted bars also changing color unpredictably) by the presentation of a spatially uninformative auditory cue synchronized with the color change of the visual target. In the present study, the visual stimuli in the search display changed brightness instead of color, and the crossmodal congruency between the pitch of the auditory cue and the brightness of the visual target was manipulated. When cue presence and cue congruency were randomly varied between trials (Experiment 1), both congruent cues (low-frequency tones synchronized with dark target states or highfrequency tones synchronized with bright target states) and incongruent cues (the reversed mapping) facilitated visual search performance equally, relative to a no-cue baseline condition. However, when cue congruency was blocked and the participants were informed about the pitch-brightness mapping in the cue-present blocks (Experiment 2), performance was significantly enhanced when the cue and target were crossmodally congruent as compared to when they were incongruent. These results therefore suggest that the crossmodal congruency between auditory pitch and visual brightness can influence performance in the pip-and-pop task by means of top-down facilitation.

Wine and music (II): can you taste the music? Modulating the experience of wine through music and sound

“…In research conducted here in Oxford, we have found that the presentation of high-versus low-pitched sounds can facilitate people's ability to pick out a specific visual target from in-amongst a cluttered array of flickering visual distractors [79]. Both of these perceptual enhancement effects appeared to occur in a fairly bottom-up or stimulusdriven manner.…”

How automatic are crossmodal correspondences?

“…Recent studies have delivered seemingly contradictory evidence concerning the automaticity of crossmodal correspondences (see Chiou & Rich, 2012a;Evans & Treisman, 2010;Klapetek et al, 2012;Peiffer-Smadja, 2010). These studies have utilised a variety of different experimental paradigms, including speeded classification (Evans & Treisman, 2010), exogenous spatial attentional cuing (Chiou & Rich, 2012a;Mossbridge, Grabowecky, & Suzuki, 2011), visual search (Klapetek et al, 2012), and a simplified variant of the Implicit Association Test (IAT; Peiffer-Smadja, 2010).…”

“…However, the suggested automaticity of crossmodal correspondences has been questioned by a series of negative results from studies that have sometimes failed to show any difference in behaviour between those conditions in which congruent vs. incongruent pairs of visual and auditory stimuli have been presented (see also Chiou & Rich, 2012a;Heron et al, 2012;Klapetek, Ngo, & Spence, 2012;Klein, Brennan, D'Aloisio, D'Entremont, & Gilani, 1987;Sweeny, Guzman-Martinez, Ortega, Grabowecky, & Suzuki, 2012). Explaining why such differences between studies have been obtained represents a worthwhile endeavour: And, what is more, in answering the question of the degree of automaticity of crossmodal correspondences, two further related questions also come to the fore, as detailed below.…”

Crossmodal correspondences between odors and contingent features: odors, musical notes, and geometrical shapes

“…Now, this might be attributable to researchers having looked preferentially at those crossmodal influences that have been formed by associating various cues relative to the same object (e.g., following the unity assumption, considered as a crucial factor in multisensory research since Welch & Warren, 1986; see also Vatakis & Spence, 2007). However, effects that are not limited to these cases have been demonstrated in those broader crossmodal correspondences between sometimes distinctly presented objects whose dimensions happen to feel congruent: Higher-pitched sounds can facilitate the detection of brighter objects, even without coming from or being attributed to the same source or object (e.g., Klapetek, Ngo, & Spence, 2012;Ludwig, Adachi, & Matzuzawa, 2011). As we detail below, this draws attention to the need for further investigation in the case of associations holding between olfaction and musical features, and it indicates a few of the directions along which research could profitably be pursued (see Table 2 and below).…”