Development of a brain computer interface interface using multi-frequency visual stimulation and deep neural networks

Perez-Benitez, J.L.; Benítez, José Alberto Pérez; Espina-Hernández, J.H.

doi:10.1109/conielecomp.2018.8327170

Cited by 12 publications

(17 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Occlusion sensitivity techniques [92,26,175] use a similar idea, by which the decisions of the network when different parts of the input are occluded are analyzed. [135,211,86,34,87,200,182,122,170,228,164,109,204,85,25] Analysis of activations [212,194,87,83,208,167,154,109] Input-perturbation network-prediction correlation maps [149,191,67,16,150] Generating input to maximize activation [188,144,160,15] Occlusion of input [92,26,175] Several studies used backpropagation-based techniques to generate input maps that maximize activations of specific units [188,144,160,15]. These maps can then be used to infer the role of specific neurons, or the kind of input they are sensitive to.…”

Section: Inspection Of Trained Modelsmentioning

confidence: 99%

Deep learning-based electroencephalography analysis: a systematic review

et al. 2019

View full text Add to dashboard Cite

Context. Electroencephalography (EEG) is a complex signal and can require several years of training, as well as advanced signal processing and feature extraction methodologies to be correctly interpreted. Recently, deep learning (DL) has shown great promise in helping make sense of EEG signals due to its capacity to learn good feature representations from raw data. Whether DL truly presents advantages as compared to more traditional EEG processing approaches, however, remains an open question.Objective. In this work, we review 156 papers that apply DL to EEG, published between January 2010 and July 2018, and spanning different application domains such as epilepsy, sleep, braincomputer interfacing, and cognitive and affective monitoring. We extract trends and highlight interesting approaches from this large body of literature in order to inform future research and formulate recommendations.Methods. Major databases spanning the fields of science and engineering were queried to identify relevant studies published in scientific journals, conferences, and electronic preprint repositories. Various data items were extracted for each study pertaining to 1) the data, 2) the preprocessing methodology, 3) the DL design choices, 4) the results, and 5) the reproducibility of the experiments. These items were then analyzed one by one to uncover trends. * The first two authors contributed equally to this work.Significance. To help the community progress and share work more effectively, we provide a list of recommendations for future studies. We also make our summary table of DL and EEG papers available and invite authors of published work to contribute to it directly.

show abstract

Section: Inspection Of Trained Modelsmentioning

confidence: 99%

Deep learning-based electroencephalography analysis: a systematic review

et al. 2019

View full text Add to dashboard Cite

show abstract

“…This technique augments data by artificially generating new samples based on existing training data [ 19 ]. Typical methods of DA include geometric transformation (GT), noise addition (NA) [ 20 ], and generative models [ 21 , 22 , 23 ]. DA using GT and NA is achieved by changing the geometric features of the data, and generative models use a hidden model to create generated data (GD) that have a similar distribution to the real data (RD) [ 24 ].…”

Section: Introductionmentioning

confidence: 99%

Data Augmentation for Motor Imagery Signal Classification Based on a Hybrid Neural Network

Zhang

Han

et al. 2020

Sensors

View full text Add to dashboard Cite

As an important paradigm of spontaneous brain-computer interfaces (BCIs), motor imagery (MI) has been widely used in the fields of neurological rehabilitation and robot control. Recently, researchers have proposed various methods for feature extraction and classification based on MI signals. The decoding model based on deep neural networks (DNNs) has attracted significant attention in the field of MI signal processing. Due to the strict requirements for subjects and experimental environments, it is difficult to collect large-scale and high-quality electroencephalogram (EEG) data. However, the performance of a deep learning model depends directly on the size of the datasets. Therefore, the decoding of MI-EEG signals based on a DNN has proven highly challenging in practice. Based on this, we investigated the performance of different data augmentation (DA) methods for the classification of MI data using a DNN. First, we transformed the time series signals into spectrogram images using a short-time Fourier transform (STFT). Then, we evaluated and compared the performance of different DA methods for this spectrogram data. Next, we developed a convolutional neural network (CNN) to classify the MI signals and compared the classification performance of after DA. The Fréchet inception distance (FID) was used to evaluate the quality of the generated data (GD) and the classification accuracy, and mean kappa values were used to explore the best CNN-DA method. In addition, analysis of variance (ANOVA) and paired t-tests were used to assess the significance of the results. The results showed that the deep convolutional generative adversarial network (DCGAN) provided better augmentation performance than traditional DA methods: geometric transformation (GT), autoencoder (AE), and variational autoencoder (VAE) (p < 0.01). Public datasets of the BCI competition IV (datasets 1 and 2b) were used to verify the classification performance. Improvements in the classification accuracies of 17% and 21% (p < 0.01) were observed after DA for the two datasets. In addition, the hybrid network CNN-DCGAN outperformed the other classification methods, with average kappa values of 0.564 and 0.677 for the two datasets.

show abstract

“…1 for a survey), so that motor neural impulses can be mapped and controlled with the ultimate goal of assisting, augmenting, or repairing human cognitive or sensory-motor functions. Industry is also making efforts in the design of "mind reading" devices in order to let users monitor their well-being 2 , for example, to support meditation or facilitate the execution of daily activities [3][4][5][6][7][8][9][10][11][12][13] .…”

mentioning

confidence: 99%

“…4 Department of Computer Science, University of Verona, Strada Le Grazie 15, 37134 Verona, Italy. 5 Naver Labs Europe, 6 Chemin de Maupertuis, Meylan, 38240 Grenoble, France. * email: pietro.morerio@iit.it (a) Exemplar footages for Action Concepts Understanding (images are selected from the public dataset [17]).…”

mentioning

confidence: 99%

“…For instance, in Ref. 3 , a system for rapid image search is devised, whereas EEG and motion capture can be combined in a multi-modal BCI 4 with the optional usage of deep learning to better fuse the two modalities 5 . Classical computer vision problems, such as object classification in images can be boosted when having access to an ancillary data modality like EEG.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Understanding action concepts from videos and brain activity through subjects’ consensus

Cavazza

Ahmed

Volpi

et al. 2022

Sci Rep

View full text Add to dashboard Cite

In this paper, we investigate brain activity associated with complex visual tasks, showing that electroencephalography (EEG) data can help computer vision in reliably recognizing actions from video footage that is used to stimulate human observers. Notably, we consider not only typical “explicit” video action benchmarks, but also more complex data sequences in which action concepts are only referred to, implicitly. To this end, we consider a challenging action recognition benchmark dataset—Moments in Time—whose video sequences do not explicitly visualize actions, but only implicitly refer to them (e.g., fireworks in the sky as an extreme example of “flying”). We employ such videos as stimuli and involve a large sample of subjects to collect a high-definition, multi-modal EEG and video data, designed for understanding action concepts. We discover an agreement among brain activities of different subjects stimulated by the same video footage. We name it as subjects consensus, and we design a computational pipeline to transfer knowledge from EEG to video, sharply boosting the recognition performance.

show abstract

Development of a brain computer interface interface using multi-frequency visual stimulation and deep neural networks

Cited by 12 publications

References 13 publications

Deep learning-based electroencephalography analysis: a systematic review

Deep learning-based electroencephalography analysis: a systematic review

Data Augmentation for Motor Imagery Signal Classification Based on a Hybrid Neural Network

Understanding action concepts from videos and brain activity through subjects’ consensus

Contact Info

Product

Resources

About