Localized estimation of electromagnetic sources underlying event-related fields using recurrent neural networks

Abstract

Objective: To use a recurrent neural network (RNN) to reconstruct neural activity responsible for generating noninvasively measured electromagnetic signals.
Approach: Output weights of an RNN were fixed as the lead field matrix from volumetric source space computed using the boundary element method with co-registered structural magnetic resonance images and magnetoencephalography (MEG). Initially, the network was trained to minimize mean-squared-error loss between its outputs and MEG signals, causing activations in the penultimate layer to converge towards putative neural source activations. Subsequently, L1 regularization was applied to the final hidden layer, and the model was fine-tuned, causing it to favour more focused activations. Estimated source signals were then obtained from the outputs of the last hidden layer. We developed and validated this approach with simulations before applying it to real MEG data, comparing performance with beamformers, minimum-norm estimate, and mixed-norm estimate source reconstruction methods.
Main results: The proposed RNN method had higher output signal-to-noise ratios and comparable correlation and error between estimated and simulated sources. Reconstructed MEG signals were also equal or superior to the other methods regarding their similarity to ground-truth. When applied to MEG data recorded during an auditory roving oddball experiment, source signals estimated with the RNN were generally biophysically plausible and consistent with expectations from the literature.
Significance: This work builds on recent developments of RNNs for modelling event-related neural responses by incorporating biophysical constraints from the forward model, thus taking a significant step towards greater biological realism and introducing the possibility of exploring how input manipulations may influence localized neural activity.

Localized estimation of event-related neural source activity from simultaneous MEG-EEG with a recurrent neural network

“…As with previous RNN models of event-related neural signals (O'Reilly, 2022b;O'Reilly, Angsuwatanakul, et al, 2022;O'Reilly, Zhu, et al, 2023), the architecture consisted of an input, four hidden layers, and an output layer. Time-domain signals were passed to the input layer, transformed sequentially through four simple recurrent layers, and then fed through a feed-forward output layer.…”

“…The backpropagation through time algorithm (Werbos, 1990) with adaptive moment estimation optimizer (Kingma & Ba, 2015) and mean-squared error loss were used to train the RNN, all using default hyperparameter settings in TensorFlow (Abadi et al, 2016). As described previously (O'Reilly, Zhu, et al, 2023;Srivastava et al, 2023), to derive source estimates, the RNN was pre-trained to fit labels without additional constraints (training step 1) before discarding the bias parameter and introducing L1-norm activity regularization to the fourth recurrent layer (training step 2). This L1norm penalty was weighted by 10 −4 .…”

“…A long short-term memory (LSTM; Hochreiter & Schmidhuber, 1997) type of RNN has been used to refine MEG and EEG source estimates obtained by applying MNE to individual time samples, highlighting the importance of temporal context among neural signals for improving source reconstruction (Dinh et al, 2021). Simple RNN models have also been developed for estimating localized source signals from MEG (O'Reilly, Zhu, et al, 2023) and EEG (Srivastava et al, 2023) by fixing the weights of a feedforward output layer equal to the lead field matrix. By training the RNN in two steps: (1) pretraining and (2) transfer learning with L1 regularization, estimated source signals are derived from penultimate layer hidden unit activations (O'Reilly, Zhu, et al, 2023).…”

“…Simple RNN models have also been developed for estimating localized source signals from MEG (O'Reilly, Zhu, et al, 2023) and EEG (Srivastava et al, 2023) by fixing the weights of a feedforward output layer equal to the lead field matrix. By training the RNN in two steps: (1) pretraining and (2) transfer learning with L1 regularization, estimated source signals are derived from penultimate layer hidden unit activations (O'Reilly, Zhu, et al, 2023).…”

Blind source separation of event-related potentials using a recurrent neural network

“…RNNs for analysing ERP waveforms have recently been developed for modelling auditory evoked potentials from mice [19], [20], human ERPs [21], [22], and combining with a convolutional neural network (CNN) to study visual ERPs [23]. RNNs can also be used for distributed source reconstruction from MEG [24], EEG [25], and simultaneously recorded MEG-EEG [26]. These previous studies demonstrate some of the ways that RNNs can be used for analysing event-related neural signals.…”