EEG-Based Cross-Subject Driver Drowsiness Recognition With an Interpretable Convolutional Neural Network

Abstract

In the context of electroencephalogram (EEG)-based driver drowsiness recognition, it is still challenging to design a calibration-free system, since EEG signals vary significantly among different subjects and recording sessions. Many efforts have been made to use deep learning methods for mental state recognition from EEG signals. However, existing work mostly treats deep learning models as black-box classifiers, while what have been learned by the models and to which extent they are affected by the noise in EEG data are still underexplored. In this paper, we develop a novel convolutional neural network combined with an interpretation technique that allows sample-wise analysis of important features for classification. The network has a compact structure and takes advantage of separable convolutions to process the EEG signals in a spatial-temporal sequence. Results show that the model achieves an average accuracy of 78.35% on 11 subjects for leave-one-out cross-subject drowsiness recognition, which is higher than the conventional baseline methods of 53.40%-72.68% and state-of-the-art deep learning methods of 71.75%-75.19%. Interpretation results indicate the model has learned to recognize biologically meaningful features from EEG signals, e.g., Alpha spindles, as strong indicators of drowsiness across different subjects.In addition, we also explore reasons behind some wrongly classified samples with the interpretation technique and discuss potential ways to improve the recognition accuracy. Our work illustrates a promising direction on using interpretable deep learning models to discover meaningful patterns related to different mental states from complex EEG signals.

A Decomposition-Based Hybrid Ensemble CNN Framework for Improving Cross-Subject EEG Decoding Performance

“…In the proposed decomposition-based hybrid ensemble CNN framework, we adopted InterpretableCNN [16] as the backbone network. For convenience, we denoted the InterpretableCNN as ICNN and ensemble InterpretableCNN as EICNN.…”

“…Specifically, the PSD features in the frequency bands of alpha (8-12 Hz), theta (4-7 Hz), and delta (1-3 Hz) were utilized. 2) EEGNet-8,2 [15], 3) ConvNet [11], 4) InterpretableCNN [16] and 5) subjectmatching model [28] which is a type of domain generalization framework to reduce the subject-variability problem by exploiting the similarity between subjects data.…”

“…In 2017, Lawhern et al [15] proposed a compact CNN named EEGNet that can be applied on different braincomputer interface (BCI) paradigms and achieved good performance in subject-dependent and subject-independent settings. Moreover, Cui et al [16] proposed an InterpretableCNN which performed spatial and temporal convolution operations, which had been shown the superiority on drowsiness EEG signal decoding and improved the subject-independent recognition accuracy compared with EEGNet. Apart from commonly used 2-dimension CNN, other architectures, such as long-short term memory (LSTM) networks [17], and 3D CNN [18], have presented promising results for corresponding BCI paradigms as well.…”

Recent Advances in Fatigue Detection Algorithm Based on EEG

“…As a result, they achieved an average precision of 75.87% on 27 subjects for leave-one-out cross-validation. Cui et al [56] tried to improved an interpretation technique called class activation map (CAM) to discover the factors that favor and disfavor classification. Separable convolution was utilized to reduce parameters and speed up network convergence.…”

TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibration

“…The extracted feature vectors originate from hand-and body poses [14], [19], eyerelated inputs like driver gaze [20], [21], head patterns [17], [21], as well as foot dynamics [22]. Object recognition cues [23] and physiological signals [24], [25] are also associated for driver behavior observation.…”