A multimodal encoding model applied to imaging decision-related neural cascades in the human brain

Abstract

Perception and cognition in the brain are naturally characterized as spatiotemporal processes. Decision-making, for example, depends on coordinated patterns of neural activity cascading across the brain, running in time from stimulus to response and in space from primary sensory regions to the frontal lobe. Measuring this cascade is key to developing an understanding of brain function. Here we report on a novel methodology that employs multi-modal imaging for inferring this cascade in humans at unprecedented spatiotemporal resolution. Specifically, we develop an encoding model to link simultaneously measured electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) signals to infer high-resolution spatiotemporal brain dynamics during a perceptual decision. After demonstrating replication of results from the literature, we report previously unobserved sequential reactivation of a substantial fraction of the pre-response network whose magnitude correlates with a proxy for decision confidence. Our encoding model, which temporally tags BOLD activations using time localized EEG variability, identifies a coordinated and spatially distributed neural cascade that is associated with a perceptual decision. In general the methodology illuminates complex brain dynamics that would otherwise be unobservable using fMRI or EEG acquired separately.

Disentangling the origins of confidence in speeded perceptual judgments through multimodal imaging

“…Furthermore, Murphy et al (43) showed that similar error-related feedback signals from the pMFC inform metacognitive judgments through the modulation of parietal activity involved in evidence accumulation. Other regions including the IPL, precentral cortex, and aPFC were found specifically in the late decoding window, which hints at their involvement in late processes at play for the computation of graded confidence estimates (44,45).…”

Generalized neural decoders for transfer learning across participants and recording modalities

“…To our knowledge, HTNet is the first decoder that can generalize and transfer its learning across both ECoG participants and different recording modalities. Previous studies have implemented decoders that can transfer across different EEG devices [14, 46, 77, 78] or leverage data from concurrent recording modalities [79, 80], but none of these decoders have demonstrated the ability to generalize to an entirely different recording modality. As for generalizing across participants, many decoders can do this with EEG data [14], including EEGNet [55], but development of analogous ECoG decoders has been hindered by the high variation in electrode placement across ECoG patients.…”

Non-stationary Group-Level Connectivity Analysis for Enhanced Interpretability of Oddball Tasks

“…Because the study focused on task-related attentional states, subjects were asked to respond to target stimuli, using a button press with the right index finger on an MR (Magnetic Resonance) compatible button response pad. Stimuli were presented to subjects using E-Prime software (Psychology Software Tools) and a VisuaStim Digital System (Resonance Technology) comprising headphones and 600×800 goggle display, as detailed in Muraskin et al (2018). Scalp data were acquired at 1, 000 Hz sampling rate (that is, t = 0.001 s) using an EEG data acquisition system with a custom cap configuration of C = 34 channels, for which the following preprocessing Butterworth filters were used: 1-Hz high pass to remove direct current drift; notched filter (centered at 60 and 120 Hz) to eliminate the electrical power line and its first harmonic, respectively; and a low pass filter with a cut frequency at 120-Hz, excluding high-frequency artifacts without neurophysiological content.…”