Inferring Macroscale Brain Dynamics via Fusion of Simultaneous EEG-fMRI

Abstract

Advances in the instrumentation and signal processing for simultaneously acquired electroencephalography and functional magnetic resonance imaging (EEG-fMRI) have enabled new ways to observe the spatiotemporal neural dynamics of the human brain. Central to the utility of EEG-fMRI neuroimaging systems are the methods for fusing the two data streams, with machine learning playing a key role. These methods can be dichotomized into those that are symmetric and asymmetric in terms of how the two modalities inform the fusion. Studies using these methods have shown that fusion yields new insights into brain function that are not possible when each modality is acquired separately. As technology improves and methods for fusion become more sophisticated, the future of EEG-fMRI for noninvasive measurement of brain dynamics includes mesoscale mapping at ultrahigh magnetic resonance fields, targeted perturbation-based neuroimaging, and using deep learning to uncover nonlinear representations that link the electrophysiological and hemodynamic measurements. Expected final online publication date for the Annual Review of Neuroscience, Volume 44 is July 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

On the Role of Neural Oscillations Across Timescales in Speech and Music Processing

“…Neuroimaging methods can be applied in tandem, compensating for individual methods’ shortcomings. For example, pairing EEG and fMRI allows for temporally precise localization of neural patterns ( Philiastides et al, 2021 ; cf. Chang and Chen, 2021 ; Scrivener, 2021 ).…”

Pupil-linked phasic arousal relates to the reduction of response inhibition: inferences from a simultaneous pupillometry and EEG-fMRI study

“…Compared with acquiring EEG or fMRI separately, simultaneous acquisition of EEG and fMRI has less popularity. This is in part due to the technical challenges involved in concurrent data acquisition, and partly due to the scarcity of effective data fusion methods 39 . Understandably, relative to simultaneous EEG-fMRI, due to the increased technical and methodological hurdles involved with the addition of a third modality, concurrent acquisitions combining three modalities (pupillometry, EEG, and fMRI) have so far been applied in very few instances.…”

Spatiotemporal dissociation of brain network connectivity in salience processing: A simultaneous pupillometry-EEG-fMRI study

“…Specifically, the STV temporal information in the EEG was extracted at different time windows spanning the trial to explain the variance in the fMRI signal. This approach has been widely used to study a broad range of cognitive functions and human behaviors [66, 72, 107]. Compared to conventional fMRI analyses, this method allows us to temporally dissociate the stimuli-evoked brain activation, or even identify regions absent in the conventional analyses (canceled out due to a temporal integration effect) [37, 66, 72].…”

“…This approach has been widely used to study a broad range of cognitive functions and human behaviors [66, 72, 107]. Compared to conventional fMRI analyses, this method allows us to temporally dissociate the stimuli-evoked brain activation, or even identify regions absent in the conventional analyses (canceled out due to a temporal integration effect) [37, 66, 72]. This work extends this approach by introducing the functional connectome into the framework for mapping the underlying spatiotemporal network organizations of these neural substrates.…”