EEG-TNet: An End-To-End Brain Computer Interface Framework for Mental Workload Estimation

Abstract

The mental workload (MWL) of different occupational groups' workers is the main and direct factor of unsafe behavior, which may cause serious accidents. One of the new and useful technologies to estimate MWL is the Brain computer interface (BCI) based on EEG signals, which is regarded as the gold standard of cognitive status. However, estimation systems involving handcrafted EEG features are time-consuming and unsuitable to apply in real-time. The purpose of this study was to propose an end-to-end BCI framework for MWL estimation. First, a new automated data preprocessing method was proposed to remove the artifact without human interference. Then a new neural network structure named EEG-TNet was designed to extract both the temporal and frequency information from the original EEG. Furthermore, two types of experiments and ablation studies were performed to prove the effectiveness of this model. In the subject-dependent experiment, the estimation accuracy of dual-task estimation (No task vs. TASK) and triple-task estimation (Lo vs. Mi vs. Hi) reached 99.82 and 99.21%, respectively. In contrast, the accuracy of different tasks reached 82.78 and 66.83% in subject-independent experiments. Additionally, the ablation studies proved that preprocessing method and network structure had significant contributions to estimation MWL. The proposed method is convenient without any human intervention and outperforms other related studies, which becomes an effective way to reduce human factor risks.

Survey on the research direction of EEG-based signal processing

“…Singh et al ( 2021 ) has highlighted the importance of subject-independent tasks— they play a crucial role in designing a plug-and-play calibration-free BCI device. While many studies report high within-subject accuracy, which can exceed 90% in several cases, cross-subject accuracy is still lower (mostly around 50%; Lashgari et al, 2021 ; Fan et al, 2022 ). Improving cross-subject accuracy has become a significant direction.…”

“…A detailed table with information on the directions and performance of each paper can be found in the Supplementary material . The following reviewed papers are presented in ascending order of their published date (Aellen et al, 2021 ; Asheri et al, 2021 ; Ashwini and Nagaraj, 2021 ; Awais et al, 2021 ; Cai et al, 2021 ; Dagdevir and Tokmakci, 2021 ; De Venuto and Mezzina, 2021 ; Du et al, 2021 ; Fan et al, 2021 , 2022 ; Ferracuti et al, 2021 ; Gao N. et al, 2021 ; Gao Z. et al, 2021 ; Gaur et al, 2021 ; Lashgari et al, 2021 ; Lian et al, 2021 ; Liu and Jin, 2021 ; Liu and Yang, 2021 ; Liu et al, 2021 ; Qi et al, 2021 ; Rashid et al, 2021 ; Sun et al, 2021 ; Varsehi and Firoozabadi, 2021 ; Vega et al, 2021 ; Vorontsova et al, 2021 ; Wahid and Tafreshi, 2021 ; Wang and Quan, 2021 ; Xu C. et al, 2021 ; Xu F. et al, 2021 ; Yin et al, 2021 ; Zhang K. et al, 2021 ; Zhang Y. et al, 2021 ; Algarni et al, 2022 ; Ali et al, 2022 ; Asadzadeh et al, 2022 ; Ayoobi and Sadeghian, 2022 ; Bagchi and Bathula, 2022 ; Chang et al, 2022 ; Chen J. et al, 2022 ; Chen L. et al, 2022 ; Cui et al, 2022 ; Geng et al, 2022 ; Islam et al, 2022 ; Jia et al, 2022 ; Kim et al, 2022 ; Ko et al, 2022 ; Li and Sun, 2022 ; Li H. et al, 2022 ; Lin et al, 2022 ; Li Q. et al, 2022 ; Lu et al, 2022 ; Ma et al, 2022 ; Mattioli et al, 2022 ;...…”

Influencing factors and mechanism of high-speed railway passenger overall comfort: Insights from source functional brain network and subjective report

“…As EEG is one of the best objective criteria for evaluating comfort, many scholars have used EEG in recent years to study the influencing factors and mechanisms of comfort (13,32,33). For example, Fukai et al proposed a method to assess the ride comfort of cars with different tires using EEG (34).…”

The Application of Electroencephalogram in Driving Safety: Current Status and Future Prospects

“…Moreover, EEG uses a simple and subject-acceptable method to obtain data that can be used for driver state perception analysis. Therefore, EEG signals have become a common focus for future intelligent transportation-assisted driving and brain-computer interface fields (Cheng et al, 2022a;Fan et al, 2022). An optimal human-machine symbiotic interaction, where the vehicle can consider the driver's goals and preferences, can be achieved by fully exploring the intrinsic correlation between driving states and physiological/psychological signals.…”