Decoding P300 Variability Using Convolutional Neural Networks

Abstract

Deep convolutional neural networks (CNN) have previously been shown to be useful tools for signal decoding and analysis in a variety of complex domains, such as image processing and speech recognition. By learning from large amounts of data, the representations encoded by these deep networks are often invariant to moderate changes in the underlying feature spaces. Recently, we proposed a CNN architecture that could be applied to electroencephalogram (EEG) decoding and analysis. In this article, we train our CNN model using data from prior experiments in order to later decode the P300 evoked response from an unseen, hold-out experiment. We analyze the CNN output as a function of the underlying variability in the P300 response and demonstrate that the CNN output is sensitive to the experiment-induced changes in the neural response. We then assess the utility of our approach as a means of improving the overall signal-to-noise ratio in the EEG record. Finally, we show an example of how CNN-based decoding can be applied to the analysis of complex data.

Thinker invariance: enabling deep neural networks for BCI across more people

“…Recognizing the need here to capture a temporal signature is likely crucial, and therefore it is no surprise that recurrent neural network architectures feature more commonly [19,31,44] than with MI tasks, given their propensity for sequence modelling (although they find themselves presently out of fashion for this purpose). That said, shallow separated (mostly linear) temporal and spatial convolution architectures were also well represented [3,31,[45][46][47]. Throughout most of the work we encountered, aside from the work of Ditthapron et al, we find consistent use of P300 datasets recorded for BCI competitions II and III 6 , which feature only one and two subjects respectively, making them poor choices for thinker-invariant evaluation.…”

A Lightweight Multi-Scale Convolutional Neural Network for P300 Decoding: Analysis of Training Strategies and Uncovering of Network Decision

“…The first dataset is BCIAUT-P300, a public benchmark dataset released for the IFMBE 2019 scientific challenge (available at https://www.kaggle.com/disbeat/bciaut-p300 ) (Simões et al, 2020 ) consisting of a larger number of examples than other public benchmarks (Blankertz et al, 2004 , 2006 ) or private (Lawhern et al, 2018 ; Farahat et al, 2019 ; Solon et al, 2019 ) datasets. Signals were recorded from 15 participants (all males, age of 22 ± 5 years, mean ± standard deviation) with ASD during seven recording sessions (for a total of 4 months) while testing a P300-based BCI (Amaral et al, 2018 ).…”

Motor decoding from the posterior parietal cortex using deep neural networks

“…trial-level decoding). The sliding window decoding procedure is often adopted in the literature [41][42][43][44]62] since it exhibits three main advantages. First, learning systems are forced to associate the proper label (in case of classification tasks) to a few hundreds of milliseconds of signals instead of to the entire trial (singletrial decoding), thus, providing a faster and earlier inference over the trial course.…”

“…In this regard, future developments could benefit from the adoption of post-hoc explanation techniques [86] (e.g. saliency maps or layer-wise relevance propagation), devoted to highlight useful features in a domain under investigation, as recently obtained in CNNs for EEG processing [15,16,49,53,62,[87][88][89]. Second, all the performed analyses were conducted offline; thus, the insights provided from the performed offline investigations should be validated online in future studies.…”