A New Generation of Brain-Computer Interfaces Driven by Discovery of Latent EEG-fMRI Linkages Using Tensor Decomposition

Abstract

A Brain-Computer Interface (BCI) is a setup permitting the control of external devices by decoding brain activity. Electroencephalography (EEG) has been extensively used for decoding brain activity since it is non-invasive, cheap, portable, and has high temporal resolution to allow real-time operation. Due to its poor spatial specificity, BCIs based on EEG can require extensive training and multiple trials to decode brain activity (consequently slowing down the operation of the BCI). On the other hand, BCIs based on functional magnetic resonance imaging (fMRI) are more accurate owing to its superior spatial resolution and sensitivity to underlying neuronal processes which are functionally localized. However, due to its relatively low temporal resolution, high cost, and lack of portability, fMRI is unlikely to be used for routine BCI. We propose a new approach for transferring the capabilities of fMRI to EEG, which includes simultaneous EEG/fMRI sessions for finding a mapping from EEG to fMRI, followed by a BCI run from only EEG data, but driven by fMRI-like features obtained from the mapping identified previously. Our novel data-driven method is likely to discover latent linkages between electrical and hemodynamic signatures of neural activity hitherto unexplored using model-driven methods, and is likely to serve as a template for a novel multi-modal strategy wherein cross-modal EEG-fMRI interactions are exploited for the operation of a unimodal EEG system, leading to a new generation of EEG-based BCIs.

BCIAUT-P300: A Multi-Session and Multi-Subject Benchmark Dataset on Autism for P300-Based Brain-Computer-Interfaces

“…Different types of neuroimaging techniques can be used to implement BCIs, i.e., electroencephalography (EEG), magnetoencephalography (MEG), functional Magnetic Resonance Imaging (fMRI), functional Near-Infrared Spectroscopy (fNIRS), among others ( Zou et al, 2019 ). The most common modality is the EEG, since it provides a portable, inexpensive, non-invasive solution to measure brain activity with high temporal resolution ( Sitaram et al, 2007 ; Bhattacharyya et al, 2017 ; Deshpande et al, 2017 ; Zou et al, 2019 ).…”

Extraction of common task features in EEG-fMRI data using coupled tensor-tensor decomposition

“…On the other hand, tensors conveniently present multidimensional data and tensor decomposition techniques such as PARAFAC or Tucker decomposition do not impose constraints in the optimization process. Tensor-based analysis of concurrent EEG-fMRI have received increasing attention in recent years (Vanderperren et al 2010;Karahan et al 2015;Ferdowsi et al 2015;Hunyadi et al 2016Hunyadi et al , 2017Acar et al 2017a,b;Deshpande et al 2017;Sen and Parhi 2017;Van Eyndhoven et al 2017;Chatzichristos et al 2018;Kinney-Lang et al 2019). However, all the tensor-based fusion of the EEG-fMRI methods, except (Chatzichristos et al 2018), has been in fact under the matrixtensor factorization framework, i.e., fMRI in a matrix and EEG in a 3rd order tensor, and only one mode of variability such as participant or time have been used as the common loading vectors for the decomposition of the two modalities.…”

Brain–Computer Interfaces in Neurorecovery and Neurorehabilitation

“…Examples of multimodal sensors include combinations of MEG and EEG, 53 or of EEG and fMRI. 54 Sensors can be distinguished by their location (e.g., implanted within deep brain structures, or within cortex, or subdural, epidural, intra-, or epicranial, on the scalp, or external to the head), temporal resolution (i.e., sensing speed), spatial resolution (i.e., sensing detail), signal-to-noise ratio, sensor size, and ability to record signals for an extended period of time. 55 Electrical sensors, and specifically EEG and MEGs, are the two most common types of BCI sensors used toward the restoration of movement and communication for people with neurologic disease.…”