How to Build a Hybrid Neurofeedback Platform Combining EEG and fMRI

Abstract

Multimodal neurofeedback estimates brain activity using information acquired with more than one neurosignal measurement technology. In this paper we describe how to set up and use a hybrid platform based on simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI), then we illustrate how to use it for conducting bimodal neurofeedback experiments. The paper is intended for those willing to build a multimodal neurofeedback system, to guide them through the different steps of the design, setup, and experimental applications, and help them choose a suitable hardware and software configuration. Furthermore, it reports practical information from bimodal neurofeedback experiments conducted in our lab. The platform presented here has a modular parallel processing architecture that promotes real-time signal processing performance and simple future addition and/or replacement of processing modules. Various unimodal and bimodal neurofeedback experiments conducted in our lab showed high performance and accuracy. Currently, the platform is able to provide neurofeedback based on electroencephalography and functional magnetic resonance imaging, but the architecture and the working principles described here are valid for any other combination of two or more real-time brain activity measurement technologies.

Simultaneous MRI-EEG during a motor imagery neurofeedback task: an open access brain imaging dataset for multi-modal data integration

“…The multimodal imaging NF platform used to acquire simultaneously EEG and fMRI data and to provide real-time NF has been described in 17 and a schematic is showed in Figure 3. It guarantees the acquisition, processing, NF features computation and visualization in real-time of the two data streams (EEG and fMRI).…”

“…A detailed description of bimodal EEG-fMRI NF platform is given in 17 : the platform software package for real-time analysis and visualization is well documented but not publicly available. Python pipelines for the analysis of structural and functional MRI ( Figure 8) are available on github (https://github.com/glioi/BIDS_fMRI_analysis_nipype), in form of commented jupyter notebooks.…”

“…The bimodal NF platform architecture for simultaneous EEG-fMRI recording is described in 17 , together with details about the two layers data synchronization and preprocessing implemented in order to guarantee real-time NF presentation. Results relative to Data Citation 1 were published in 8 , a study where the added value of bimodal NF with respect to the single modalities (EEG and fMRI only NF) was shown.…”

Real‐time fMRI neurofeedback of the mediodorsal and anterior thalamus enhances correlation between thalamic BOLD activity and alpha EEG rhythm

“…Therefore, the rtfMRI‐nf with simultaneous EEG offers new opportunities for studying relationships between electrophysiological and hemodynamic processes in the human brain. This approach can also help to identify promising target measures for EEG neurofeedback (EEG‐nf) (Gruzelier, ), brain‐computer interfaces (Cavazza et al, ), and simultaneous multimodal rtfMRI‐EEG‐nf (Mano et al, ; Perronnet et al, ; Zotev, Phillips, Yuan, Misaki, & Bodurka, ).…”

A sparse EEG-informed fMRI model for hybrid EEG-fMRI neurofeedback prediction

“…However, in the context of neurofeedback, using simultaneous recording of EEG-fMRI to estimate neurofeedback scores computed in real time from features of both modalities (NF-EEG-fMRI) is a recent application that have been first introduced, and its feasibility demonstrated by (Zotev et al, 2014;Perronnet et al, 2017;Mano et al, 2017). The recent methodology synchronising both signals for real time neurofeedback , allows the acquisition of a new kind of data named NF-EEG-fMRI data, such as the one used here, and first presented in .…”