The “ventriloquist effect”: Visual dominance or response bias?

Abstract

The interaction between vision and audition was investigated using a signal detection method. A light and tone were presented either in the same location or in different locations along the horizontal plane, and the subjects responded with same-different judgments of stimulus location. Three modes of stimulus presentation were used: simultaneous presentation of the light and tone, tone first, and light first. For the latter two conditions, the interstimulus interval was either 0.7 or 2.0 sec. A statistical decision model was developed which distinguished between the perceptual and decision processes. The results analyzed within the framework of this model suggested that the apparent interaction between vision and audition is due to shifts in decision criteria rather than perceptual change.One of the most controversial issues in psychology concerns the genesis of perceptual abilities and the relations that exist among the sense modalities. Until recently. it was strongly believed that "vision is educated by touch." Many scholars, such as Berkeley and Helmholtz. argued that since visual perception is grossly different from the image on the retina, it must be acquired through tactual experience. Although many experiments were conducted to test the cmpirieistie doctrine of visual space perception (e.g., Stratton. 1~97), there has been no unequivocal evidence supporting the position. Indeed, the argument has been challenged by recent studies of intersensory relations. Harris (1965). for example, demonstrated that vision "dominates" and modities the proprioceptive sense when the two modalities arc made to provide discrepant information. Thus. a person viewing the image of his hand through deflecting prism lenses comes to feel his hand where he sees it rather than where it actually is. Rock and Harris (I % 7) reported several experiments which further demonstrated the dominance of vision over touch. They concluded that rather than touch educating vision. the reverse appears to be true.The lind ing of visual dominance over proprioception led other researchers to investigate conflict situat ions involving other sensory modalities. Pick, Wurrcn , and Hay (ll)6l» examined the interaction of vision and proprioception. proprioception and audition. and vision and audition. For vision and audition. the discrepancy was created by displacing

Multidimensional processing of dynamic sounds: more than meets the ear

“…We found that independent perceptual and decision-related processes affected cross-modal processing of sound changes. This finding is consistent with previous research showing that multiple processes independently affect audiovisual interactions in the perception of lateral motion (Alais et al, 2010;Sanabria et al, 2007b;SotoFaraco et al, 2004a, b) and static events (Bertelson & de Gelder, 2004;Choe et al, 1975). Sanabria et al (2007b), for example, asked listeners to detect or classify whether an auditory apparent motion stream moved in a predefined target direction (left or right).…”

“…In contrast, visual effects on auditory sensitivity remained consistent regardless of testing context. Our findings complement previous research demonstrating independent perceptual and decisional influences coexisting in the audiovisual perception of lateral motion (Bertelson & de Gelder, 2004;Choe et al, 1975;Sanabria et al, 2007b;Schirillo, 2011) and static audiovisual events (Bertelson & de Gelder, 2004;Choe et al, 1975;Sanabria et al, 2007b;Schirillo, 2011). In addition, they raise new questions about how changes in auditory and visual cues interact.…”

“…Other researchers have isolated perceptual from decision-related effects statistically. For example, Choe, Welch, Gilford & Juola (1975) investigated spatial ventriloquism by using signal detection measures. Listeners were presented with synchronous and asynchronous flashes and tones and were asked to judge whether the tone and flash came from the same location or not.…”

Binaural loudness summation for speech presented via earphones and loudspeaker with and without visual cues

“…The McGurk effect (McGurk and McDonald, 1976) is a well-known example of how perception can involve more than one sense; if a subject observes a talker articulating /ga/ and simultaneously hears /ba/, the subject reports perceiving /da/. Visual capture (aka ventriloquism) is a wellknown example of visual location taking precedence over the spatial location of an auditory sound source (i.e., the talking dummy is perceived, see Choe et al, 1975). Research on the influence of one modality on another abounds (e.g., vision on hearing; for a review of multisensory interactions in ratings of loudness see Fastl and Florentine, 2011).…”

Action potential influences spatial perception: Evidence for genuine top-down effects on perception

“…My colleagues and I have made a similar point with respect to perceptual and response-based biases analyzed using signal detection theory (Witt, Taylor, Sugovic, & Wixted, 2015). Some researchers have claimed, solely on the basis of an effect on the bias measure c, that the effect is due to response-based bias and is not perceptual (e.g., Choe, Welch, Gilford, & Juola, 1975;Grove, Ashton, Kawachi, & Sakurai, 2012). In contrast, we used simulations of the Müller-Lyer illusion to show that both a perceptual bias and a response-based bias can produce the exact same pattern in the measure c. In other words, when the Müller-Lyer illusion is modeled as a perceptual bias in which lines with tails oriented inward appear shorter than lines with tails outward (i.e., the classic interpretation of this bias), 1002 Psychon Bull Rev (2017) 24:999-1021 signal detection theory analyses showed a large effect on c but no effect on d'.…”