Comparison of congruence judgment and auditory localization tasks for assessing the spatial limits of visual capture

Abstract

Vision typically has better spatial accuracy and precision than audition, and as a result often captures auditory spatial perception when visual and auditory cues are presented together. One determinant of visual capture is the amount of spatial disparity between auditory and visual cues: when disparity is small visual capture is likely to occur, and when disparity is large visual capture is unlikely. Previous experiments have used two methods to probe how visual capture varies with spatial disparity. First, congruence judgment assesses perceived unity between cues by having subjects report whether or not auditory and visual targets came from the same location. Second, auditory localization assesses the graded influence of vision on auditory spatial perception by having subjects point to the remembered location of an auditory target presented with a visual target. Previous research has shown that when both tasks are performed concurrently they produce similar measures of visual capture, but this may not hold when tasks are performed independently. Here, subjects alternated between tasks independently across three sessions. A Bayesian inference model of visual capture was used to estimate perceptual parameters for each session, which were compared across tasks. Results demonstrated that the range of audio-visual disparities over which visual capture was likely to occur were narrower in auditory localization than in congruence judgment, which the model indicates was caused by subjects adjusting their prior expectation that targets originated from the same location in a task-dependent manner.

Multiple time scales of the ventriloquism aftereffect

“…On average, responses in the pre-disparity block showed a small but significant leftward bias of −1.6° across all experimental sessions (which is negative for the rightward arrows and positive for the leftward arrows plotted in Fig 4 ), as confirmed with a one-sample t-test ( t (23) = −2.78, p = 0.01). A small leftward bias has been previously observed in our lab [ 27 ] and in other labs [ 29 ]. As a result, the distribution of pre-disparity bias is segregated by the direction of the session in Fig 4 , which subsequently makes Δ Encoded uneven across leftward and rightward sessions.…”

“…As shown in Fig 2 , localization responses are subject to systematic inaccuracies (perfect performance would be a flat line with zero intercept). Specifically, localization responses are subject to both uniform errors across space (referred to as bias , μ A ) and a tendancy to overestimate target eccentricity (referred to as positive spatial gain , SG ) [ 9 , 15 , 26 , 27 ]. As a consequence, the disparity between the auditory and visual locations encoded by the subject’s sensory systems can differ from the physical disparity, as shown in Eq 1 .…”

“…Although we did not measure it in this experiment, we included visual capture in the model to facilitate the model’s use in future experiments. For simplicity, we assume that visual capture always occurs at a fixed proportion of Δ AV , k , denoted A Cap , rather than modeling the chance that visual capture does or does not occur on a trial-by-trial basis [ 19 , 27 , 31 ]. Experimentally, the distance between targets is kept relatively small( S A − S V = ±8° for all experiments), so capture should occur on most, if not all, audio-visual targets.…”

Modality-specific attention attenuates visual-tactile integration and recalibration effects by reducing prior expectations of a common source for vision and touch

“…14). Thus, our study indicates that priors about the location of events and objects in the world depend on the sense through which we perceive them, which is consistent with previous reports on discrepancies between visual and auditory spatial priors (Odegaard, Wozny, & Shams, 2015;Bosen et al, 2016). The modality-specificity of spatial priors is consistent with the variability of the common-cause prior revealed by our study, both results suggest priors to be situation-dependent.…”

“…Thus, these two measures, localization shifts and alignment judgments, provided related but distinct information. Consistently, in previous studies, in which both responses were collected and modeled separately, parameter estimates diverged with the measure that was fitted (Bosen et al, 2016;Acerbi, Dokka, Angelaki, & Ma, 2018) even though qualitatively the two measures agree widely (Wallace et al, 2004;Hairston et al, 2003;Rohe & Noppeney, 2015b). In the best-fitting model, spatial-alignment responses were based on comparisons of the optimal location estimates of the two sensory signals (a decision rule not included in previous models).…”