Self-motion through an environment involves a composite of signals such as visual and vestibular cues. Building upon previous results showing that visual and vestibular signals combine in a statistically optimal fashion, we investigated the relative weights of visual and vestibular cues during self-motion. This experiment was comprised of three experimental conditions: vestibular alone, visual alone (with four different standard heading values), and visual-vestibular combined. In the combined cue condition, inter-sensory conflicts were introduced (Δ = ±6° or ±10°). Participants performed a 2-interval forced choice task in all conditions and were asked to judge in which of the two intervals they moved more to the right. The cue-conflict condition revealed the relative weights associated with each modality. We found that even when there was a relatively large conflict between the visual and vestibular cues, participants exhibited a statistically optimal reduction in variance. On the other hand, we found that the pattern of results in the unimodal conditions did not predict the weights in the combined cue condition. Specifically, visual-vestibular cue combination was not predicted solely by the reliability of each cue, but rather more weight was given to the vestibular cue.

Successful integration of auditory and visual inputs is crucial for both basic perceptual functions and for higher-order processes related to social cognition. Autism spectrum disorders (ASD) are characterized by impairments in social cognition and are associated with abnormalities in sensory and perceptual processes. Several groups have reported that individuals with ASD are impaired in their ability to integrate socially relevant audiovisual (AV) information, and it has been suggested that this contributes to the higher-order social and cognitive deficits observed in ASD. However, successful integration of auditory and visual inputs also influences detection and perception of nonsocial stimuli, and integration deficits may impair earlier stages of information processing, with cascading downstream effects. To assess the integrity of basic AV integration, we recorded high-density electrophysiology from a cohort of high-functioning children with ASD (7-16 years) while they performed a simple AV reaction time task. Children with ASD showed considerably less behavioral facilitation to multisensory inputs, deficits that were paralleled by less effective neural integration. Evidence for processing differences relative to typically developing children was seen as early as 100 ms poststimulation, and topographic analysis suggested that children with ASD relied on different cortical networks during this early multisensory processing stage.

Congruent audiovisual speech enhances our ability to comprehend a speaker, even in noise-free conditions. When incongruent auditory and visual information is presented concurrently, it can hinder a listener's perception and even cause him or her to perceive information that was not presented in either modality. Efforts to investigate the neural basis of these effects have often focused on the special case of discrete audiovisual syllables that are spatially and temporally congruent, with less work done on the case of natural, continuous speech. Recent electrophysiological studies have demonstrated that cortical response measures to continuous auditory speech can be easily obtained using multivariate analysis methods. Here, we apply such methods to the case of audiovisual speech and, importantly, present a novel framework for indexing multisensory integration in the context of continuous speech. Specifically, we examine how the temporal and contextual congruency of ongoing audiovisual speech affects the cortical encoding of the speech envelope in humans using electroencephalography. We demonstrate that the cortical representation of the speech envelope is enhanced by the presentation of congruent audiovisual speech in noise-free conditions. Furthermore, we show that this is likely attributable to the contribution of neural generators that are not particularly active during unimodal stimulation and that it is most prominent at the temporal scale corresponding to syllabic rate (2-6 Hz). Finally, our data suggest that neural entrainment to the speech envelope is inhibited when the auditory and visual streams are incongruent both temporally and contextually.

Walking while simultaneously performing cognitively demanding tasks such as talking or texting are typical complex behaviors in our daily routines. Little is known about neural mechanisms underlying cortical resource allocation during such mobile actions, largely due to portability limitations of conventional neuroimaging technologies. We applied an EEG-based Mobile Brain-Body Imaging (MOBI) system that integrates high-density event-related potential (ERP) recordings with simultaneously acquired foot-force sensor data to monitor gait patterns and brain activity. We compared behavioral and ERP measures associated with performing a Go/NoGo response-inhibition task under conditions where participants (N=18) sat stationary, walked deliberately or walked briskly. This allowed for assessment of effects of increasing dual-task load (i.e. walking speed) on neural indices of inhibitory control. Stride time and variability were also measured during inhibitory task performance and compared to stride parameters without task performance, thereby assessing reciprocal dual-task effects on gait parameters. There were no task performance differences between sitting and either walking condition, indicating that participants could perform both tasks simultaneously without suffering dual-task costs. However, participants took longer strides under dual-task load, likely indicating an adaptive mechanism to reduce inter-task competition for cortical resources. We found robust differences in amplitude, latency and topography of ERP components (N2 and P3) associated with inhibitory control between the sitting and walking conditions. Considering that participants showed no dual-task performance costs, we suggest that observed neural alterations under increasing task-load represent adaptive recalibration of the inhibitory network towards a more controlled and effortful processing mode, thereby optimizing performance under dual-task situations.

The simultaneous presentation of a stimulus in one sensory modality often enhances target detection in another sensory modality, but the neural mechanisms that govern these effects are still under investigation. Here we test a hypothesis proposed in the neurophysiologic literature: that auditory facilitation of visual-target detection operates through cross-sensory phase reset of ongoing neural oscillations (see Lakatos et al., 2009). To date, measurement limitations have prevented this potentially powerful neural mechanism from being directly linked with its predicted behavioral consequences. The present experiment uses a psychophysical approach in humans to demonstrate, for the first time, stimulus-locked periodicity in visual-target detection, following a temporally informative sound. Our data further demonstrate that periodicity in behavioral performance is strongly influenced by the probability of audiovisual co-occurrence. We argue that fluctuations in visual-target detection result from cross-sensory phase reset, both at the moment it occurs and persisting for seconds thereafter. The precise frequency at which this periodicity operates remains to be determined through a method that allows for a higher sampling rate.

Atypical processing and integration of sensory inputs are hypothesized to play a role in unusual sensory reactions and social-cognitive deficits in autism spectrum disorder (ASD). Reports on the relationship between objective metrics of sensory processing and clinical symptoms, however, are surprisingly sparse. Here we examined the relationship between neurophysiological assays of sensory processing and 1) autism severity and 2) sensory sensitivities, in individuals with ASD aged 6–17. Multiple linear regression indicated significant associations between neural markers of auditory processing and multisensory integration, and autism severity. No such relationships were apparent for clinical measures of visual/auditory sensitivities. These data support that aberrant early sensory processing contributes to autism symptoms, and reveal the potential of electrophysiology to objectively subtype autism.

Even simple tasks rely on information exchange between functionally distinct and often relatively distant neuronal ensembles. Considerable work indicates oscillatory synchronization through phase alignment is a major agent of inter-regional communication. In the brain, different oscillatory phases correspond to low-and high-excitability states. Optimally aligned phases (or high-excitability states) promote inter-regional communication. Studies have also shown that sensory stimulation can modulate or reset the phase of ongoing cortical oscillations. For example, auditory stimuli can reset the phase of oscillations in visual cortex, influencing processing of a simultaneous visual stimulus. Such cross-regional phase reset represents a candidate mechanism for aligning oscillatory phase for inter-regional communication. Here, we explored the role of local and inter-regional phase alignment in driving a well established behavioral correlate of multisensory integration: the redundant target effect (RTE), which refers to the fact that responses to multisensory inputs are substantially faster than to unisensory stimuli. In a speeded detection task, human epileptic patients (N ϭ 3) responded to unisensory (auditory or visual) and multisensory (audiovisual) stimuli with a button press, while electrocorticography was recorded over auditory and motor regions. Visual stimulation significantly modulated auditory activity via phase reset in the delta and theta bands. During the period between stimulation and subsequent motor response, transient synchronization between auditory and motor regions was observed. Phase synchrony to multisensory inputs was faster than to unisensory stimulation. This sensorimotor phase alignment correlated with behavior such that stronger synchrony was associated with faster responses, linking the commonly observed RTE with phase alignment across a sensorimotor network.

Aging is associated with reduced abilities to selectively allocate attention across multiple domains. This may be particularly problematic during everyday multitasking situations when cognitively demanding tasks are performed while walking. Due to previous limitations in neuroimaging technology, much remains unknown about the cortical mechanisms underlying resource allocation during locomotion. Here, we utilized an EEG-based Mobile Brain/Body Imaging (MoBI) technique that integrates high-density event-related potential (ERP) recordings with simultaneously acquired foot-force sensor data to monitor gait patterns and brain activity concurrently. To assess effects of motor load on cognition we evaluated young (N=17; mean age=27.2) and older adults (N=16; mean age=63.9) and compared behavioral and ERP measures associated with performing a Go/No-Go response inhibition task as participants sat stationary or walked on a treadmill. Stride time and variability were also measured during task performance and compared to stride parameters obtained without task performance, thereby assessing effects of cognitive load on gait. Results showed that older, but not young adults’ accuracy dropped significantly when performing the inhibitory task while walking. Young adults revealed ERP modulations at relatively early (N2 amplitude reduction) and later (earlier P3 latency) stages within the processing stream as motor load increased while walking. In contrast, older adults’ ERP modulations were limited to later processing stages (increased P3 amplitude) of the inhibitory network. The relative delay and attenuation of ERP modulations accompanied by behavioral costs in older participants might indicate an age-associated loss in flexible resource allocation across multiple tasks. Better understanding of the neural underpinnings of these age-related changes may lead to improved strategies to reduce fall risk and enhance mobility in aging.