Hybrid Deep Learning (hDL)-Based Brain-Computer Interface (BCI) Systems: A Systematic Review

Abstract

Background: Brain-Computer Interface (BCI) is becoming more reliable, thanks to the advantages of Artificial Intelligence (AI). Recently, hybrid Deep Learning (hDL), which combines different DL algorithms, has gained momentum over the past five years. In this work, we proposed a review on hDL-based BCI starting from the seminal studies in 2015. Objectives: We have reviewed 47 papers that apply hDL to the BCI system published between 2015 and 2020 extracting trends and highlighting relevant aspects to the topic. Methods: We have queried four scientific search engines (Google Scholar, PubMed, IEEE Xplore and Elsevier Science Direct) and different data items were extracted from each paper such as the database used, kind of application, online/offline training, tasks used for the BCI, pre-processing methodology adopted, type of normalization used, which kind of features were extracted, type of DL architecture used, number of layers implemented and which optimization approach were used as well. All these items were then investigated one by one to uncover trends. Results: Our investigation reveals that Electroencephalography (EEG) has been the most used technique. Interestingly, despite the lower Signal-to-Noise Ratio (SNR) of the EEG data that makes pre-processing of that data mandatory, we have found that the pre-processing has only been used in 21.28% of the cases by showing that hDL seems to be able to overcome this intrinsic drawback of the EEG data. Temporal-features seem to be the most effective with 93.94% accuracy, while spatial-temporal features are the most used with 33.33% of the cases investigated. The most used architecture has been Convolutional Neural Network-Recurrent Neural Network CNN-RNN with 47% of the cases. Moreover, half of the studies have used a low number of layers to achieve a good compromise between the complexity of the network and computational efficiency. Significance: To give useful information to the scientific community, we make our summary table of hDL-based BCI papers available and invite the community to published work to contribute to it directly. We have indicated a list of open challenges, emphasizing the need to use neuroimaging techniques other than EEG, such as functional Near-Infrared Spectroscopy (fNIRS), deeper investigate the advantages and disadvantages of using pre-processing and the relationship with the accuracy obtained. To implement new combinations of architectures, such as RNN-based and Deep Belief Network DBN-based, it is necessary to better explore the frequency and temporal-frequency features of the data at hand.

Comparing between Different Sets of Preprocessing, Classifiers, and Channels Selection Techniques to Optimise Motor Imagery Pattern Classification System from EEG Pattern Recognition

“…Moreover, regarding the necessity of cleaning the data, different pre-processing techniques, referred to supervised ICA approach, were highly time-consuming. On this aspect, neural networks can help since they do not need a strong pre-processing as shown in some studies [ 49 ]. In those studies, only minimal pre-processing such as removing or interpolating bad channels were used and all the rest was left to the burden of learning from a potentially noisy signal to the neural network [ 49 ].…”

“…On this aspect, neural networks can help since they do not need a strong pre-processing as shown in some studies [ 49 ]. In those studies, only minimal pre-processing such as removing or interpolating bad channels were used and all the rest was left to the burden of learning from a potentially noisy signal to the neural network [ 49 ]. On the other hand, machine learning methods based on artificial neural networks with the ability to use techniques that allow a system to automatically detect and classify features from raw data, including unsupervised training of raw input data (i.e., automatic feature selection and dimensionality reduction), are highly computationally expensive to train and to determine the hyperparameters.…”

“…On the other hand, machine learning methods based on artificial neural networks with the ability to use techniques that allow a system to automatically detect and classify features from raw data, including unsupervised training of raw input data (i.e., automatic feature selection and dimensionality reduction), are highly computationally expensive to train and to determine the hyperparameters. Consequently, the choice of the hyperparameters is mainly user-dependent [ 49 ].…”

Artificial intelligence and machine learning approaches for drug design: challenges and opportunities for the pharmaceutical industries

“…Artificial neural networks (ANNs) are favourably recognized computer models developed based on the Homo sapiens brain and its networking trends [ 16 ]. The simplest case shows a fully connected network or feed-forward that shapes the computation chart with three layers (input layer, hidden layer, and output layer) [ 17 ]. The layer-wise single computing unit called neurons works as a nonlinear transformation to the input data.…”

A Recent Investigation on Detection and Classification of Epileptic Seizure Techniques Using EEG Signal

“…The most common approach is a convolutional neural network with a variety of architectures such as temporal CNNs (TCNNs), temporal graph convolutional networks (TGCNs) and CNN-recurrent neural networks (RNNs) [161]. A CNN's basic structure consists of convolutional layers, max pooling layers, fully connected layers and softmax layers [162,163].…”