Individual differences in the multisensory temporal binding window predict susceptibility to audiovisual illusions.

Abstract

Human multisensory systems are known to bind inputs from the different sensory modalities into a unified percept, a process that leads to measurable behavioral benefits. This integrative process can be observed through multisensory illusions, including the McGurk effect and the sound-induced flash illusion, both of which demonstrate the ability of one sensory modality to modulate perception in a second modality. Such multisensory integration is highly dependent upon the temporal relationship of the different sensory inputs, with perceptual binding occurring within a limited range of asynchronies known as the temporal binding window (TBW). Previous studies have shown that this window is highly variable across individuals, but it is unclear how these variations in the TBW relate to an individual’s ability to integrate multisensory cues. Here we provide evidence linking individual differences in multisensory temporal processes to differences in the individual’s audiovisual integration of illusory stimuli. Our data provide strong evidence that the temporal processing of multiple sensory signals and the merging of multiple signals into a single, unified perception, are highly related. Specifically, the width of right side of an individuals’ TBW, where the auditory stimulus follows the visual, is significantly correlated with the strength of illusory percepts, as indexed via both an increase in the strength of binding synchronous sensory signals and in an improvement in correctly dissociating asynchronous signals. These findings are discussed in terms of their possible neurobiological basis, relevance to the development of sensory integration, and possible importance for clinical conditions in which there is growing evidence that multisensory integration is compromised.

Testing Sensory and Multisensory Function in Children with Autism Spectrum Disorder

“…Thus, accounting for the differences in propagation times of the auditory and visual signals, the brain assesses audiovisual temporal structure in relation to this window, and thus makes a probabilistic judgment as to whether the stimuli belong together or not. These data strongly argue for the TBW as a measure of temporal acuity and strength of multisensory integration, and indeed it has been shown that the width of this window appears to be correlated with the magnitude of the binding process, with those with smaller windows having larger indices of integration 18,27 . In addition to be a probabilistic construct across individuals, the TBW is also very much dependent on stimulus and task.…”

“…As an example of the utility of this battery, individual differences in temporal processing (such as the width of the TBW) can be correlated with performance differences on a perceptual task like the McGurk illusion ( Figure 3B). Several studies have observed a link between temporal acuity in the SJ task and perceptual differences in speech perception in the McGurk task and other measures of multisensory integration 18,27 . Because of the variability of responses for the same stimulus both within individual subjects and across subjects in a group, responses are shown as the percent of trials that were perceived as each phoneme.…”

“…This task battery has proven very successful in measuring individual differences in temporal processing in individuals with and without ASD 17,18,23,27 . For the SJ task, plot the resulting data from each individual subject by first calculating the proportion of responses at each SOA that subject responded "synchronous" and then fitting the resulting response curve with a Gaussian curve.…”

“…The battery has wide applicability for a number of populations and has been used by our laboratory in order to characterize audiovisual temporal performance in typical adults 18 , children 10,39 , and in children and adults with autism 17,23 . In addition, it has been used to examine how various facets of the battery relate to one another in correlational analyses 27 , and is currently being used to relate sensory and multisensory performance measures to cognitive domains including language and communication, attention and executive function.…”

“…In such a task, even when the visual and auditory stimuli are presented at SOAs of 100 msec or more, subjects report that the pair was simultaneous on a large proportion of trials. The window of time in which two inputs can occur and have a high probability of being perceived as occurring simultaneously is known as the temporal binding window (TBW) [17][18][19] .…”