Decoding EEG and LFP signals using deep learning

Nurse, Ewan S.; Mashford, Benjamin S.; Yepes, Antonio Jimeno; Kiral-Kornek, Isabell; Harrer, Stefan; Freestone, Dean R.

doi:10.1145/2903150.2903159

Cited by 71 publications

(46 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Non-EEG information, especially gaze data and data about the fixated objects, their environment, possible types of current activity of the user and so on can be used as additional features, or, in some cases, for selecting a classifier (e.g., specific classifiers can be trained for different steps in sequences of actions, as in action triplets in our game). If sufficiently large amounts of gaze-synchronized EEG data will be harvested during the use of EBCIs, it will become possible to apply deep learning algorithms (LeCun et al, 2015; see also Nurse et al, 2016, on deep learning implementation at TrueNorth chip for EEG/ECoG/LFP data) that are able to find hidden patterns in the data and strongly improve classification performance.…”

Section: Discussionmentioning

confidence: 99%

EEG Negativity in Fixations Used for Gaze-Based Control: Toward Converting Intentions into Actions with an Eye-Brain-Computer Interface

et al. 2016

View full text Add to dashboard Cite

We usually look at an object when we are going to manipulate it. Thus, eye tracking can be used to communicate intended actions. An effective human-machine interface, however, should be able to differentiate intentional and spontaneous eye movements. We report an electroencephalogram (EEG) marker that differentiates gaze fixations used for control from spontaneous fixations involved in visual exploration. Eight healthy participants played a game with their eye movements only. Their gaze-synchronized EEG data (fixation-related potentials, FRPs) were collected during game's control-on and control-off conditions. A slow negative wave with a maximum in the parietooccipital region was present in each participant's averaged FRPs in the control-on conditions and was absent or had much lower amplitude in the control-off condition. This wave was similar but not identical to stimulus-preceding negativity, a slow negative wave that can be observed during feedback expectation. Classification of intentional vs. spontaneous fixations was based on amplitude features from 13 EEG channels using 300 ms length segments free from electrooculogram contamination (200–500 ms relative to the fixation onset). For the first fixations in the fixation triplets required to make moves in the game, classified against control-off data, a committee of greedy classifiers provided 0.90 ± 0.07 specificity and 0.38 ± 0.14 sensitivity. Similar (slightly lower) results were obtained for the shrinkage Linear Discriminate Analysis (LDA) classifier. The second and third fixations in the triplets were classified at lower rate. We expect that, with improved feature sets and classifiers, a hybrid dwell-based Eye-Brain-Computer Interface (EBCI) can be built using the FRP difference between the intended and spontaneous fixations. If this direction of BCI development will be successful, such a multimodal interface may improve the fluency of interaction and can possibly become the basis for a new input device for paralyzed and healthy users, the EBCI “Wish Mouse.”

show abstract

Section: Discussionmentioning

confidence: 99%

EEG Negativity in Fixations Used for Gaze-Based Control: Toward Converting Intentions into Actions with an Eye-Brain-Computer Interface

et al. 2016

View full text Add to dashboard Cite

show abstract

“…Occlusion sensitivity techniques [92,26,175] use a similar idea, by which the decisions of the network when different parts of the input are occluded are analyzed. [135,211,86,34,87,200,182,122,170,228,164,109,204,85,25] Analysis of activations [212,194,87,83,208,167,154,109] Input-perturbation network-prediction correlation maps [149,191,67,16,150] Generating input to maximize activation [188,144,160,15] Occlusion of input [92,26,175] Several studies used backpropagation-based techniques to generate input maps that maximize activations of specific units [188,144,160,15]. These maps can then be used to infer the role of specific neurons, or the kind of input they are sensitive to.…”

Section: Inspection Of Trained Modelsmentioning

confidence: 99%

Deep learning-based electroencephalography analysis: a systematic review

et al. 2019

View full text Add to dashboard Cite

Context. Electroencephalography (EEG) is a complex signal and can require several years of training, as well as advanced signal processing and feature extraction methodologies to be correctly interpreted. Recently, deep learning (DL) has shown great promise in helping make sense of EEG signals due to its capacity to learn good feature representations from raw data. Whether DL truly presents advantages as compared to more traditional EEG processing approaches, however, remains an open question.Objective. In this work, we review 156 papers that apply DL to EEG, published between January 2010 and July 2018, and spanning different application domains such as epilepsy, sleep, braincomputer interfacing, and cognitive and affective monitoring. We extract trends and highlight interesting approaches from this large body of literature in order to inform future research and formulate recommendations.Methods. Major databases spanning the fields of science and engineering were queried to identify relevant studies published in scientific journals, conferences, and electronic preprint repositories. Various data items were extracted for each study pertaining to 1) the data, 2) the preprocessing methodology, 3) the DL design choices, 4) the results, and 5) the reproducibility of the experiments. These items were then analyzed one by one to uncover trends. * The first two authors contributed equally to this work.Significance. To help the community progress and share work more effectively, we provide a list of recommendations for future studies. We also make our summary table of DL and EEG papers available and invite authors of published work to contribute to it directly.

show abstract

“…While feature extraction is a prerequisite for most motor BCI decoding algorithms, end-to-end learning—that is, learning from row data without any prior feature extraction—has recently been reported in several offline motor BCI studies (Wang Z. et al, 2013 ; Nurse et al, 2015b , 2016 ; Schirrmeister et al, 2017 ). In these studies, raw or preprocessed neural signals are directly fed to decoders.…”

Section: Feature Extractionmentioning

confidence: 99%

“…These models then learn how to both extract and decode useful neural signal characteristics during model identification. End-to-end learning has been investigated for movement classification (Nurse et al, 2015b , 2016 ; Schirrmeister et al, 2017 ) and trajectory prediction (Wang Z. et al, 2013 ) from EEG (Nurse et al, 2015b , 2016 ; Schirrmeister et al, 2017 ) and ECoG neural signals (Wang Z. et al, 2013 ) acquired either during motor imagery tasks or movement execution. These models generally rely on deep learning decoders, such as multi-layer perceptrons (Nurse et al, 2015b ) or convolutional neural networks and their variants (Wang Z. et al, 2013 ; Nurse et al, 2016 ; Schirrmeister et al, 2017 ).…”

Section: Feature Extractionmentioning

confidence: 99%

“…End-to-end learning exhibits several advantages; for example, it can be implemented with minimal preprocessing procedures [e.g., centering (Wang Z. et al, 2013 ; Nurse et al, 2015b ; Schirrmeister et al, 2017 ), scaling (Wang Z. et al, 2013 ; Schirrmeister et al, 2017 ), outlier removal (Nurse et al, 2016 ), or band pass filtering (Yuksel and Olmez, 2015 ; Sturm et al, 2016 ; Tang et al, 2017 )]. It additionally holds the promise of highly accurate decoding because of the joint optimization of feature extraction and decoding.…”

Section: Feature Extractionmentioning

confidence: 99%

See 1 more Smart Citation

Data-Driven Transducer Design and Identification for Internally-Paced Motor Brain Computer Interfaces: A Review

Schaeffer

Aksenova

2018

Front. Neurosci.

View full text Add to dashboard Cite

Brain-Computer Interfaces (BCIs) are systems that establish a direct communication pathway between the users' brain activity and external effectors. They offer the potential to improve the quality of life of motor-impaired patients. Motor BCIs aim to permit severely motor-impaired users to regain limb mobility by controlling orthoses or prostheses. In particular, motor BCI systems benefit patients if the decoded actions reflect the users' intentions with an accuracy that enables them to efficiently interact with their environment. One of the main challenges of BCI systems is to adapt the BCI's signal translation blocks to the user to reach a high decoding accuracy. This paper will review the literature of data-driven and user-specific transducer design and identification approaches and it focuses on internally-paced motor BCIs. In particular, continuous kinematic biomimetic and mental-task decoders are reviewed. Furthermore, static and dynamic decoding approaches, linear and non-linear decoding, offline and real-time identification algorithms are considered. The current progress and challenges related to the design of clinical-compatible motor BCI transducers are additionally discussed.

show abstract

Decoding EEG and LFP signals using deep learning

Cited by 71 publications

References 26 publications

EEG Negativity in Fixations Used for Gaze-Based Control: Toward Converting Intentions into Actions with an Eye-Brain-Computer Interface

EEG Negativity in Fixations Used for Gaze-Based Control: Toward Converting Intentions into Actions with an Eye-Brain-Computer Interface

Deep learning-based electroencephalography analysis: a systematic review

Data-Driven Transducer Design and Identification for Internally-Paced Motor Brain Computer Interfaces: A Review

Contact Info

Product

Resources

About