Enhanced Accuracy for Multiclass Mental Workload Detection Using Long Short-Term Memory for Brain–Computer Interface

Abstract

Cognitive workload is one of the widely invoked human factors in the areas of humanmachine interaction (HMI) and neuroergonomics. The precise assessment of cognitive and mental workload (MWL) is vital and requires accurate neuroimaging to monitor and evaluate the cognitive states of the brain. In this study, we have decoded four classes of MWL using long short-term memory (LSTM) with 89.31% average accuracy for brain-computer interface (BCI). The brain activity signals are acquired using functional near-infrared spectroscopy (fNIRS) from the prefrontal cortex (PFC) region of the brain. We performed a supervised MWL experimentation with four varying MWL levels on 15 participants (both male and female) and 10 trials of each MWL per participant. Realtime four-level MWL states are assessed using fNIRS system, and initial classification is performed using three strong machine learning (ML) techniques, support vector machine (SVM), k-nearest neighbor (k-NN), and artificial neural network (ANN) with obtained average accuracies of 54.33, 54.31, and 69.36%, respectively. In this study, novel deep learning (DL) frameworks are proposed, which utilizes convolutional neural network (CNN) and LSTM with 87.45 and 89.31% average accuracies, respectively, to solve high-dimensional four-level cognitive states classification problem. Statistical analysis, t-test, and one-way F-test (ANOVA) are also performed on accuracies obtained through ML and DL algorithms. Results show that the proposed DL (LSTM and CNN) algorithms significantly improve classification performance as compared with ML (SVM, ANN, and k-NN) algorithms.

Subject-Independent Functional Near-Infrared Spectroscopy-Based Brain–Computer Interfaces Based on Convolutional Neural Networks

“…First, the proposed deep learning approach is based on CNNs, but there are other promising neural network models-such as long short-term memory (LSTM)-which are known to be particularly effective for dealing with time-series data. Asgher et al (2020) reported that the deep learning framework based on LSTM outperformed conventional machine learning and CNNbased algorithms in the assessment of cognitive and mental workload using fNIRS. Therefore, it would be worthwhile to compare the performance of various deep learning approaches in the implementation of subject-independent fNIRS-based BCI.…”

Hybrid EEG-fNIRS BCI Fusion Using Multi-Resolution Singular Value Decomposition (MSVD)

“…Recently, deep learning techniques, such as convolution neural network (CNN) and recurrent neural network (RNN) have been utilized for automatic feature extraction, pre-processing, and classification (Zhang et al, 2017;Yang et al, 2018;Tayeb et al, 2019). The obtained results have been promising when compared to the conventional classifiers (Trakoolwilaiwan et al, 2017;Chiarelli et al, 2018;Kumar et al, 2019;Asgher et al, 2020;Ghonchi et al, 2020). Considering the improvement in accuracy obtained using deep learning techniques, even in light of the limited amount of data and fewer pre-processing requirements, this improvement motivates us to work upon the combination of such techniques with MSVD in the future.…”

A Bipolar-Channel Hybrid Brain-Computer Interface System for Home Automation Control Utilizing Steady-State Visually Evoked Potential and Eye-Blink Signals

“…This finding is consistent with the pioneer study [ 45 ], which could achieve 100% by conducting the task with a 20 s time window. With the development of deep learning, an improved classification may be achieved by utilizing the hybrid modality (i.e., EEG and fNIRS) [ 49 , 50 ] advanced machine learning algorithms, such as long-short team memory [ 51 ] and deep neural network [ 52 ]. In addition, in this study, we applied red squares with the text indicators to guide the participants to select the corresponding control commands.…”