Multisensory Perception

Abstract

There has been a huge growth of interest in the topic of multisensory perception over the past half century. The majority of this research has focused on the spatial senses of vision, audition, and touch. Multisensory interactions would appear to be the norm, not the exception. Cross‐modal interactions (which include examples of multisensory integration) are influenced by stimulus timing. Interactions between the senses tend to be maximal when the component stimuli fall within the temporal window of integration that is approximately centered on simultaneity. Spatial coincidence between component stimuli modulates multisensory integration only under conditions of target location uncertainty and/or when space is somehow relevant to the participant's task. Bayesian decision theory has largely replaced previous accounts of why the senses interact in quite the way that they do, and can explain why one sense often dominates over the others. The study of interactions between the senses is now a core component of research into perception both in humans and increasingly in other species as well.

Bimodal Presentation Speeds up Auditory Processing and Slows Down Visual Processing

“…Most of our experiences are multisensory in nature, and there is a growing body of research examining how sensory modalities interact while processing multisensory information (see Stein and Meredith, 1993; Bahrick et al, 2004; Calvert et al, 2004; Spence and Driver, 2004; Spence and McDonald, 2004; Talsma et al, 2010; Spence et al, 2012; van Atteveldt et al, 2014; Murray et al, 2016; Spence, 2018, for reviews). Simultaneously presenting information to multiple sensory modalities can sometimes facilitate learning and speed up responding (Miller, 1982; Giard and Peronnet, 1999; Bahrick and Lickliter, 2000; Bahrick et al, 2002; Fort et al, 2002; Colonius and Diederich, 2006; Sinnett et al, 2008), however, there are also many situations where stimuli in one modality interfere or alter perception in another modality (Colavita, 1974; McGurk and MacDonald, 1976; Shams et al, 2000, 2002; Sloutsky and Napolitano, 2003).…”

“…Most of our experiences are multisensory in nature and there is a growing body of research highlighting the complex nature of intersensory interactions (see Stein and Meredith, 1993; Bahrick et al, 2004; Calvert et al, 2004; Spence and Driver, 2004; Spence and McDonald, 2004; Robinson and Sloutsky, 2010a; Talsma et al, 2010; Spence et al, 2012; van Atteveldt et al, 2014; Murray et al, 2016; Spence, 2018, for reviews). For example, when stimuli from different sensory modalities provide complimentary information, bimodal presentation can often facilitate processing and speed up responding (Miller, 1982; Stein and Meredith, 1993; Giard and Peronnet, 1999; Bahrick and Lickliter, 2000; Bahrick et al, 2002; Fort et al, 2002; Colonius and Diederich, 2006; Sinnett et al, 2008).…”

“…Interestingly, participants often press the visual button when both stimuli are presented together (the correct response is pressing both buttons or a third button corresponding to a bimodal stimulus), suggesting that the visual modality dominated encoding and/or responding to multisensory information. Many studies using variations of the Colavita task have weakened but have failed to reverse this effect (see Sinnett et al, 2007; Spence, 2009, 2018; Spence et al, 2012, for reviews, but see Shams et al, 2000, 2002; Robinson and Sloutsky, 2013; Parker and Robinson, 2018, which found auditory dominance using temporal tasks which are better suited for the auditory modality).…”

The influence of audiovisual stimuli cuing temperature, carbonation, and color on the categorization of freshness in beverages

“…Interaction effects that can result from the joint presentation of various perceptual features belonging to different sensory modalities have often been studied using speeded classification tasks (see Marks, ; Spence, for reviews). People generally respond more rapidly to multisensory stimuli than to unisensory stimuli.…”

Listen to Your Heart: Examining Modality Dominance Using Cross-Modal Oddball Tasks

“…Over the last 40 years, there has been a growing body of research examining how sensory systems process and integrate incoming information ( Stein and Meredeth, 1993 ; Lewkowicz, 1994 , 2000 ; Lickliter and Bahrick, 2000 ; Calvert et al, 2004 ; Driver and Spence, 2004 ; Spence, 2009 , 2018 ; Robinson and Sloutsky, 2010a ; Spence et al, 2012 ). Intersensory interactions can result in both facilitation and interference, and these effects appear to interact with the to-be-learned information.…”