The brain acts as an integrated information processing system, which methods in cognitive neuroscience have so far depicted in a fragmented fashion. Here, we propose a simple and robust way to integrate functional MRI (fMRI) with single trial event-related potentials (ERP) to provide a more complete spatiotemporal characterization of evoked responses in the human brain. The idea behind the approach is to find brain regions whose fMRI responses can be predicted by paradigm-induced amplitude modulations of simultaneously acquired single trial ERPs. The method was used to study a variant of a two-stimulus auditory target detection (oddball) paradigm that manipulated predictability through alternations of stimulus sequences with random or regular target-to-target intervals. In addition to electrophysiologic and hemodynamic evoked responses to auditory targets per se, single-trial modulations were expressed during the latencies of the P2 (170-ms), N2 (200-ms), and P3 (320-ms) components and predicted spatially separated fMRI activation patterns. These spatiotemporal matches, i.e., the prediction of hemodynamic activation by time-variant information from single trial ERPs, permit inferences about regional responses using fMRI with the temporal resolution provided by electrophysiology.multimodal imaging ͉ P3 pattern learning ͉ target detection F unctional MRI (fMRI) of the blood oxygenation leveldependent (BOLD) response (BOLD-fMRI) measures local changes in brain hemodynamics associated with a cognitive process noninvasively with a high spatial resolution. However, an unsolved issue in fMRI research is the insufficient temporal resolution of the BOLD response. In contrast to the spatial resolution of BOLDfMRI, event-related potentials (ERP) access the current induced by synaptic activity instantaneously, with an effective temporal resolution on the order of tens to hundreds of milliseconds in case of long-latency cortical responses. However, the location of underlying generators cannot be inferred with certainty. In combination, these two complementary noninvasive methods would allow for joint high-resolution spatial and temporal mapping of the mental process under investigation and add to a more complete understanding of the neural correlates of perception and cognition (1-3). In humans, this integrated spatial and temporal precision could so far be obtained only in direct intracranial recordings, usually performed in patients receiving brain surgery for treatment of epilepsy (4-7).There are basically three approaches to multimodal integration: (i) through fusion, usually referring to the use of a common forward or generative model that can explain both the electroencephalogram (EEG) and fMRI data (8, 9); (ii) through constraints, where spatial information from the fMRI is used for a (spatiotemporal) source reconstruction of the EEG (10-12); and (iii) through prediction, where the fMRI signal is modeled as some measure of the EEG convolved with a hemodynamic response function, a principle used in our study.Invasive r...

To investigate the hypothesis of a right hemispheric superiority in negative emotional processing, event-related potentials (ERPs) were recorded from 17 sites (Fz, Cz, Pz, F3/4, F7/8, C3/4, T7/8, P3/4, P7/8, O1/2) in a visual half-field paradigm. While maintaining fixation, right-handed women viewed pictures of patients with dermatological diseases before (negative) and after (neutral) cosmetic surgery. A principal components analysis with Varimax rotation performed on ERPs revealed factors identified as N1, N2, early P3, late P3, and slow wave. Repeated measures analyses of variance performed on factor scores revealed a significant effect of emotional content for all factors except for N1. However, asymmetries in emotional processing were restricted to N2 and early P3, with maximal effects over the right parietal region. N2-P3 amplitude was augmented for negative and reduced for neutral stimuli over right hemisphere regions. Visual field presentation interacted with these asymmetries in enhancing amplitudes contralaterally for early but ipsilaterally for late ERP components. Overall, findings for N2 and P3 support theories of an asymmetry in emotional processing.

Concurrent event-related EEG-fMRI recordings pick up volume-conducted and hemodynamically convoluted signals from latent neural sources that are spatially and temporally mixed across the brain, i.e. the observed data in both modalities represent multiple, simultaneously active, regionally overlapping neuronal mass responses. This mixing process decreases the sensitivity of voxel-byvoxel prediction of hemodynamic activation by the EEG when multiple sources contribute to either the predictor and/or the response variables. In order to address this problem, we used independent component analysis (ICA) to recover maps from the fMRI and timecourses from the EEG, and matched these components across the modalities by correlating their trial-to-trial modulation. The analysis was implemented as a group-level ICA that extracts a single set of components from the data and directly allows for population inferences about consistently expressed function-relevant spatiotemporal responses. We illustrate the utility of this method by extracting a previously undetected but relevant EEG-fMRI component from a concurrent auditory target detection experiment. §Corresponding Author: