Interactions between space and effectiveness in human multisensory performance

Abstract

Several stimulus factors are important in multisensory integration, including the spatial and temporal relationships of the paired stimuli as well as their effectiveness. Changes in these factors have been shown to dramatically change the nature and magnitude of multisensory interactions. Typically, these factors are considered in isolation, although there is a growing appreciation for the fact that they are likely to be strongly interrelated. Here, we examined interactions between two of these factors – spatial location and effectiveness - in dictating performance in the localization of an audiovisual target. A psychophysical experiment was conducted in which participants reported the perceived location of visual flashes and auditory noise bursts presented alone and in combination. Stimuli were presented at four spatial locations relative to fixation (0°, 30°, 60°, 90°) and at two intensity levels (high, low). Multisensory combinations were always spatially coincident and of the matching intensity (high-high or low-low). In responding to visual stimuli alone, localization accuracy decreased and response times (RTs) increased as stimuli were presented at more eccentric locations. In responding to auditory stimuli, performance was poorest at the 30 and 60° locations. For both visual and auditory stimuli, accuracy was greater and RTs were faster for more intense stimuli. For responses to visual-auditory stimulus combinations, performance enhancements were found at locations in which the unisensory performance was lowest, results concordant with the concept of inverse effectiveness. RTs for these multisensory presentations frequently violated race-model predictions, implying integration of these inputs, and a significant location-by-intensity interaction was observed. Performance gains under multisensory conditions were larger as stimuli were positioned at more peripheral locations, and this increase was most pronounced for the low-intensity conditions. These results provide strong support that the effects of stimulus location and effectiveness on multisensory integration are interdependent, with both contributing to the overall effectiveness of the stimuli in driving the resultant multisensory response.

Stimulus intensity modulates multisensory temporal processing

“…Thus, a greater tolerance for temporal offsets with low-intensity sensory signals is necessary in order to properly reflect the natural statistics of the environment. Furthermore, the nervous system may also expand its temporal filter for less effective stimuli in order to compensate for the necessity to acquire more information toward a behavioral judgment, These results parallel previous work and work in this issue showing that more peripherally presented audiovisual stimuli are more likely to be perceptually bound at wider temporal offsets (Nidiffer, et al, 2015 (in this issue); Stevenson, Fister, et al, 2012). …”

“…Data from one additional subject was excluded for responding synchronous for all trials irrespective of SOA resulting in 45 subjects being included in all data analysis. The present study is part of a larger study investigating the interrelationship of stimulus effectiveness, and stimulus spatial and temporal factors (Nidiffer, et al, 2015 (in this issue); Stevenson, Fister, et al, 2012). …”

“…In fact, these stimulus factors are complexly intertwined, with manipulations in one having effects upon the other. For example, simply changing the spatial location of an identical stimulus will impact the effectiveness of that stimulus given the differences in spatial acuity for different regions of space (Nidiffer, Stevenson, Krueger Fister, Barnett, & Wallace, 2015 (in this issue); Stein, Meredith, Huneycutt, & McDade, 1989). Reinforcing the importance of examining these interactions in more detail, recent neurophysiological studies in animal models have shown that manipulating one aspect of a multisensory stimulus (e.g., spatial location) has consequent effects in both the temporal and effectiveness dimensions (Carriere, Royal, & Wallace, 2008; Ghose & Wallace, 2014; Krueger, et al, 2009; Royal, et al, 2009).…”

Shifts in Audiovisual Processing in Healthy Aging

“…The first two principles, the temporal and spatial principles, suggest that the more temporally synchronous [5-27] and spatially coincident [9, 14, 22, 28-35] two sensory inputs are, the more likely they are to be perceptually bound into a unified percept, and thus resulting in greater multisensory benefits. Given known decreases in temporal perception in aging in particular, the temporal aspect has been targeted in recent research.…”

Bridging the gap: Synaesthesia and multisensory processes

“…The manuscripts from Krueger Fister et al (2016) and from Nidiffer et al (2016) focus on the so-called principles of multisensory integration, and illustrate the strong interdependencies that take place across the dimensions of space, time, and effectiveness in dictating how human subjects -respond to various multisensory combinations. Building off of this, the contribution from Retsa et al (2016) highlights the presence of a compensatory mechanism for estimating stimulus duration under cross-modal circumstances, and suggest that this effect is the result of shifts in spatial attention.…”