Learning 2-in-1: Towards Integrated EEG-fMRI-Neurofeedback

Abstract

Neurofeedback (NF) allows to exert self-regulation over specific aspects of one's own brain function by returning information extracted in real-time from brain activity measures. These measures are usually acquired from a single modality, most commonly EEG or fMRI. EEG-fMRI-neurofeedback (EEG-fMRI-NF) is a new NF approach that consists of providing a NF based simultaneously on EEG and fMRI. By exploiting the complementarity of these two modalities, EEG-fMRI-NF opens a spectrum of possibilities for defining bimodal NF targets that could be more ro-bust, flexible and effective than unimodal ones. However facing a greater amount of information, the question arises of how to represent the EEG and fMRI features simultaneously in order to allow the subject to achieve better self-regulation. In this work, we propose that the EEG and fMRI features should be represented in a single bimodal feedback, which we refer to as integrated feedback. We then introduce two integrated feedback strategies for EEG-fMRI-NF and compare their early effects on a motor imagery task with a between-group design. The first group (BI DIM, n=10) was shown a two-dimensional (2D) feedback in which each dimension de-picted the information from one modality. The second group (UNI DIM, n=10) was shown a one-dimensional (1D) feedback that integrated both types of information even further by merging them into one. Online fMRI activations were significantly higher in the UNI DIM group than in the BI DIM group, which suggests that the 1D feedback is easier to control than the 2D feedback. However when looking at posthoc activation levels, the difference of fMRI activation levels between the NF runs and the preliminary motor imagery run without NF was more significant in the 2D group. Moreover, subjects from the BI DIM group produced more specific BOLD activations with a notably stronger activation in the right superior parietal lobule (BI DIM > UNI DIM, p < 0.001, uncorrected). These results suggest that the 2D feedback encourages subjects to explore their strategies to recruit more spe-cific brain patterns. To summarize, our study shows that 1D and 2D integrated feedbacks are effective but also appear to be complementary and could therefore be used in combination in a bimodal NF training program. Altogether, our study paves the way to novel integrated feedback strategies for the development of flexible and effective bimodal NF paradigms making the most of the bimodal information and better suited to clinical applications.

Simultaneous MRI-EEG during a motor imagery neurofeedback task: an open access brain imaging dataset for multi-modal data integration

“…Methods and protocol for the second study have been described in 18 and are schematically showed in Figure 2. As for the first study, the experimental procedure and the NF task were carefully explained to participants before and during the experiment, in the form of written and verbal instructions.…”

“…Data collected during the MIpre session were used to identify an optimal spatial filter for EEG and a ROI based on the maximum of activation in the target motor area. As described in detail in 18 , EEG data were pre-processed in Analyzer (Brain Product Software) and BCG artefacts were corrected with a semi-automated procedure. A Common Spatial Pattern (CSP) filter maximizing the difference in EEG power in the target band Hz) between rest and MI blocks was estimated from a selection of motor channels.…”

A sparse EEG-informed fMRI model for hybrid EEG-fMRI neurofeedback prediction

“…The neurofeedback display was uni-dimensional (1D) for 9 subjects (Figure 3 left), and bi-dimensional (2D) for 8 subjects (Figure 3 middle). For both, the goal was to bring the ball into the dark blue area (Perronnet et al, 2018).…”

“…The Figure 3. Bi-modal neurofeedback strategies (1D on the left, 2D on the middle), displayed during sessions (Perronnet et al, 2018). 1D: the ball's position represents the sum of y e and y f .…”

“…During the past few years, the use of simultaneous EEG-fMRI recording has been used to understand the links between EEG and fMRI in different states of the brain activity and received recognition as a promising multi-modal measurement of the brain activity (Perronnet et al, 2018;Abreu et al, 2018).…”