The aim of the present study is to investigate whether the crossmodal correspondence robustly documented between auditory pitch and visual elevation has analogues in the audio-tactile domain. Across 4 experiments, the compatibility effects between intuitively congruent pairs of stimuli (i.e., outward tactile movement, going from the inside of the finger toward the fingertip and increasing pitch, or inward tactile movement and decreasing pitch) and incongruent pairs stimuli (i.e., the reverse associations) were measured. Two methods were compared to assess the behavioral effects of such a correspondence: One where participants have to respond to either the auditory or tactile stimulus presented simultaneously, while ignoring the other (speeded classification task), and the other where the auditory and tactile stimuli are presented sequentially and associated to different response buttons (implicit association test). No significant compatibility effect was observed under the speeded classification task. The implicit association test revealed a significant compatibility effect. This effect was similar in the conditions where the finger was placed vertically and horizontally. However, this implicit association between pitch and tactile movements was not observed in blind participants. These results have methodological implications for the explanation and testing of crossmodal correspondences, and the origin of the widely discussed association between pitch and vertical elevation.
We investigated some spontaneous crossmodal correspondences between audition and touch both in blind and sighted people. In four experiments, we tested the interactions between the direction of tactile movement (proximal–distal vs. distal–proximal movement on the fingertip) and change in auditory frequency (increasing vs. decreasing pitch). We measured the compatibility effect between congruent stimuli (proximal–distal tactile movements and increasing pitch, or distal–proximal tactile movement and decreasing pitch) and incongruent stimuli (i.e., the reverse association). The selective attention method, commonly used to test crossmodal correspondences, requires participants to focus on tactile or auditory signals while ignoring the other one presented simultaneously. The results with this method did not reveal any significant compatibility effect. However, a variant of the implicit association task (IAT, e.g., Parise and Spence, 2012) that relies on associations in the response buttons did reveal a significant compatibility effect. This effect was similar in the conditions where the arm was placed vertically and horizontally, that is whether or not the distal–proximal tactile movement corresponded to the free movement of an object subjected to gravity. Finally, in the IAT protocol, similar effects were obtained in blind and in sighted people, i.e., a crossmodal correspondence effect was obtained independently of the arm’s position. These results have methodological implications for the testing of crossmodal correspondences and for the design of sensory substitution devices. They indeed demonstrate the relevance of using spontaneous crossmodal correspondences, and not just arbitrary associations, in order to code aspects of the original signals in conversion systems for blind people.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.