Electroencephalogram-based mental workload prediction for using Augmented Reality head mounted display in construction assembly: A deep learning approach

Qin, Yimin; Bulbul, Tanyel

doi:10.1016/j.autcon.2023.104892

Cited by 8 publications

(5 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The result suggest that it is possible to predict mental workload during exoskeleton-use for construction work. Previous studies have corroborated the assertion that mental workload can be predicted (Borghini et al, 2014;Missonnier et al, 2006;Qin & Bulbul, 2023). exoskeleton and active exoskeleton conditions.…”

Section: Mental Workloadsupporting

confidence: 60%

“…On the other hand, objective methods include the use of data collection instruments such as functional magnetic resonance imaging-fMRI, and electroencephalography (Ryu & Myung, 2005). However, EEG has been touted as one of the most suitable devices for measuring brain activities to infer mental workload (Chen et al, 2016;Qin & Bulbul, 2023). Borghini et al (2014) estimates mental workload using theta-to-alpha brain waves ratio from EEG data.…”

Section: Mental Workload Evaluation With Eegmentioning

confidence: 99%

“…Similarly, another study (Missonnier et al, 2006) indicated that using EEG signals, an increase in mental workload is noticed when there is a decrease in alpha brain waves (8-13Hz) and increase in the theta brain waves (4-8Hz) during the execution of specific tasks. In the construction industry context, EEG has been used in some studies (Chen et al, 2016;Chen et al, 2017;Qin & Bulbul, 2023;Yang et al, 2023) to estimate mental workload. For instance, EEG was used to estimate the mental workload of construction workers to on-site safety conditions (Chen et al, 2016).…”

Section: Mental Workload Evaluation With Eegmentioning

confidence: 99%

“…However, the use of supervised learning technique requires hand-crafting of features which could be labor-intensive and may be insufficient to support real-time monitoring of mental workload (Wang et al, 2023). Deep learning techniques, such as convolutional neural networks (CNN), have been used to extract intrinsic features from time-series data for recognizing occupational stress, fatigue and mental workload (Jebelli et al, 2019;Mehmood et al, 2023;Qin & Bulbul, 2023). Recurrent neural network, a class of CNN, is widely used for forecasting time-series data such as brain activity.…”

Section: Machine Learning For Mental Workload Predictionmentioning

confidence: 99%

“…Also, Liu et al (2022) used LSTM for detecting fatigue of drivers. In construction, Qin and Bulbul (2023) used LSTM for predicting the mental workload of workers while using augmented reality head-mounted display for construction assembly. Despite these possibilities, limited studies have harnessed LSTM for predicting mental workload associated with exoskeleton-use for construction work.…”

Section: Machine Learning For Mental Workload Predictionmentioning

confidence: 99%

See 4 more Smart Citations

Predicting Mental Workload of Using Exoskeletons for Construction Work: A Deep Learning Approach

Akanmu,

Afolabi,

Okunola

2023

CONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality

View full text Add to dashboard Cite

Exoskeletons are gaining attention as a potential solution for addressing low back injury in the construction industry. However, use of active back-support exoskeletons in construction can trigger unintended consequences which could increase mental workload of users while working with exoskeletons. Prolonged increase in mental workload could impact workers’ wellbeing and productivity. Prediction of mental workload during exoskeleton-use could inform strategies to mitigate the triggers. This study investigates a machine-learning framework for predicting mental workload of workers while using active back-support exoskeletons for construction work. Laboratory experiments were conducted wherein Electroencephalography (EEG) data were collected from participants wearing active back-support exoskeletons to perform flooring task. The EEG data underwent preprocessing, including band filtering, notch filtering, and independent component analysis, to remove artifacts and ensure data quality. A regression-based Long Short-Term Memory network was trained to forecast future time steps of the processed EEG data. The performance of the network was evaluated using root mean square error (RMSE) and r-squared (R2). A RMSE of 0.1527 and R2 of 0.9665 indicating good fit and strong correlation, respectively, were observed between the predicted and actual EEG data. Results of the comparison between the actual and predicted mental workload also show strong correction with an R2 of 0.8692. The findings motivate research directions into real-time monitoring of mental workload of workers during exoskeleton-use. The study has significant implications for stakeholders, enabling them to gain a deeper understanding of the impact of mental workload while using exoskeletons thereby providing opportunities for mitigation

show abstract

Section: Mental Workloadsupporting

confidence: 60%

Section: Mental Workload Evaluation With Eegmentioning

confidence: 99%

Section: Mental Workload Evaluation With Eegmentioning

confidence: 99%

Section: Machine Learning For Mental Workload Predictionmentioning

confidence: 99%

Section: Machine Learning For Mental Workload Predictionmentioning

confidence: 99%

See 3 more Smart Citations

Predicting Mental Workload of Using Exoskeletons for Construction Work: A Deep Learning Approach

Akanmu,

Afolabi,

Okunola

2023

CONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality

View full text Add to dashboard Cite

show abstract

Augmented reality technologies in construction project assembly phases

Alkan,

Basaga

2023

Automation in Construction

View full text Add to dashboard Cite

Systematic review of experimental paradigms and deep neural networks for electroencephalography-based cognitive workload detection

K N,

Gupta

2024

Prog. Biomed. Eng.

View full text Add to dashboard Cite

This article summarizes a systematic literature review of deep neural network-based cognitive workload (CWL) estimation from electroencephalographic (EEG) signals. The focus of this article can be delineated into two main elements: first is the identification of experimental paradigms prevalently employed for CWL induction, and second, is an inquiry about the data structure and input formulations commonly utilized in deep neural networks (DNN) – based CWL detection. The survey revealed several experimental paradigms that can reliably induce either graded levels of CWL or a desired cognitive state due to sustained induction of CWL.They have been characterized in this article with respect to the number of distinct CWL levels, cognitive states, experimental environment, and agents in focus. Further, this literature analysis found that DNNs can successfully detect distinct levels of CWL despite the inter-subject and inter-session variability typically observed in EEG signals. Several methodologies were found using EEG signals in its native representation of a two-dimensional matrix as input to the classification algorithm, bypassing traditional feature selection steps. More often than not, researchers used DNNs as black-box type models, and only a few studies employed interpretable or explainable DNNs for CWL detection. However, these algorithms were mostly post hoc data analysis and classification schemes, and only a few studies adopted real-time CWL estimation methodologies. Further, it has been suggested that using interpretable deep learning methodologies may shed light on EEG correlates of CWL, but this remains mostly an unexplored area. This systematic review suggests using networks that are sensitive to temporal dependencies along with appropriate input formulations for each type of DNN architecture to achieve robust classification performance. An additional suggestion is to utilize transfer learning methods to achieve high generalizability across tasks (task-independent classifiers), while simple cross-subject data pooling may achieve the same for subject-independent classifiers.

show abstract

Electroencephalogram-based mental workload prediction for using Augmented Reality head mounted display in construction assembly: A deep learning approach

Cited by 8 publications

References 42 publications

Predicting Mental Workload of Using Exoskeletons for Construction Work: A Deep Learning Approach

Predicting Mental Workload of Using Exoskeletons for Construction Work: A Deep Learning Approach

Augmented reality technologies in construction project assembly phases

Systematic review of experimental paradigms and deep neural networks for electroencephalography-based cognitive workload detection

Contact Info

Product

Resources

About