On cross-modal similarity: Auditory–visual interactions in speeded discrimination.

Abstract

A series of four experiments explored how cross-modal similarities between sensory attributes in vision and hearing reveal themselves in speeded, two-stimulus discrimination. When subjects responded differentially to stimuli on one modality, speed and accuracy of response were greater on trials accompanied by informationally irrelevant "matching" versus "mismatching" stimuli from the other modality. Cross-modal interactions appeared in (a) responses to dim/bright lights and to dark/light colors accompanied by low-pitched/high-pitched tones; (b) responses to low-pitched/high-pitched tones accompanied by dim/bright lights or by dark/light colors; (c) responses to dim/bright lights, but not to dark/light colors, accompanied by soft/loud sounds; and (d) responses to rounded/sharp forms accompanied by low-pitched/high-pitched tones. These results concur with findings on cross-modal perception, synesthesia, and synesthetic metaphor, which reveal similarities between pitch and brightness, pitch and lightness, loudness and brightness, and pitch and form. The cross-modal interactions in response speed and accuracy may take place at a sensory/perceptual level of processing or after sensory stimuli are encoded semantically.

That Sounds Sweet: Using Cross‐Modal Correspondences to Communicate Gustatory Attributes

“…For instance, angular shapes are reliably matched to higher (rather than lower) auditory pitch (Marks, 1987;Walker, 2012). Congruence Specific to audition and taste, recent cognitive psychology and neuroscience research has demonstrated that people will reliably match specific (psycho-)acoustic and musical parameters with different tastes, flavors, and oral-somatosensory food-related experiences (Crisinel, et al, 2012;Crisinel & Spence, 2010a, 2010b, 2011, 2012a, 2012bMesz, Sigman, & Trevisan, 2012;Mesz, Trevisan, & Sigman, 2011;Simner, Cuskley, & Kirby, 2010).…”

Behaviour Management Problems as Predictors of Psychotropic Medication and Use of Psychiatric Services in Adults with Autism

“…The same trend is found in non-synaesthetes. This is found if people are asked to select colours for a pitch (Marks, 1974), rate how well colours and pitches go together (Hubbard, 1996), or even in Stroop-like tasks in which participants make a light/dark judgement whilst listening to an irrelevant high or low pitched tone (Marks, 1987;Melara, 1989). It is tempting to conclude that the synaesthetes and nonsynaesthetes are using the same cognitive and neural mechanisms to derive these associations, but with only the former eliciting an overt, definite perceptual experience.…”

“…This could be particularly important for sound-colour mappings because these are not necessarily arbitrary. For example, most people tend to associate high pitch sounds with light colours (Hubbard, 1996;Marks, 1974Marks, , 1982Marks, , 1987Melara, 1989) and most people are able to generate, on demand, visual imagery to music (e.g., Karwoski and Odbert, 1938).…”

Abstract Concepts in Grounded Cognition

“…Marks (1978), on the other hand, has argued that the equivalences in cross-modal perception reflect intrinsic properties of the sensory systems and depend upon the neurophysiological mechanisms that code perceptual qualities. In a set of reaction time studies (Marks, 1987), participants received compound (sound-light) stimuli, and were asked to categorize the stimuli based on one dimension (e.g., sound) while ignoring the other component (e.g., light) by pressing one of two keys. The results revealed that responses were faster (and the accuracy higher) on those trials where the critical stimulus (e.g., high pitch) is accompanied by "matching" (e.g., bright light) rather than "mismatching" (e.g., dim light) stimuli from another modality.…”