Classification of Cognitive Load and Expertise for Adaptive Simulation using Deep Multitask Learning

Sarkar, Payel; Ross, Kyle; Ruberto, Aaron J.; Rodenbura, Dirk; Hungler, Paul; Etemad, Ali

doi:10.1109/acii.2019.8925507

Cited by 33 publications

(26 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several studies have also considered the variations in drivers’ behavioural data obtained from vehicular signals for drivers’ cognitive load classification [ 15 , 16 ]. Machine learning methods that have been used for detecting the driver state, e.g., cognitive load include SVM [ 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 ], artificial neural network [ 2 , 26 , 27 , 28 , 29 ], random forest [ 30 , 31 ], deep learning [ 32 , 33 ], and case-based reasoning [ 34 , 35 , 36 , 37 ]. The performance of cognitive load classification is often poor when there are uncertainties—such as participants failing to perform some task or, in a real-time system.…”

Section: Introductionmentioning

confidence: 99%

Towards Intelligent Data Analytics: A Case Study in Driver Cognitive Load Classification

Ahmed

Begum

2020

Brain Sciences

View full text Add to dashboard Cite

One debatable issue in traffic safety research is that the cognitive load by secondary tasks reduces primary task performance, i.e., driving. In this paper, the study adopted a version of the n-back task as a cognitively loading secondary task on the primary task, i.e., driving; where drivers drove in three different simulated driving scenarios. This paper has taken a multimodal approach to perform ‘intelligent multivariate data analytics’ based on machine learning (ML). Here, the k-nearest neighbour (k-NN), support vector machine (SVM), and random forest (RF) are used for driver cognitive load classification. Moreover, physiological measures have proven to be sophisticated in cognitive load identification, yet it suffers from confounding factors and noise. Therefore, this work uses multi-component signals, i.e., physiological measures and vehicular features to overcome that problem. Both multiclass and binary classifications have been performed to distinguish normal driving from cognitive load tasks. To identify the optimal feature set, two feature selection algorithms, i.e., sequential forward floating selection (SFFS) and random forest have been applied where out of 323 features, a subset of 42 features has been selected as the best feature subset. For the classification, RF has shown better performance with F1-score of 0.75 and 0.80 than two other algorithms. Moreover, the result shows that using multicomponent features classifiers could classify better than using features from a single source.

show abstract

Section: Introductionmentioning

confidence: 99%

Towards Intelligent Data Analytics: A Case Study in Driver Cognitive Load Classification

Ahmed

Begum

2020

Brain Sciences

View full text Add to dashboard Cite

show abstract

“…Additionally, electroencephalogram (EEG) and electrooculogram (EOG) have been found to have a strong correlation with cognitive load [16]. These biological signals could also be indicative of level of expertise [17]. Machine learning classifiers using these bio-signals, could therefore, be used to facilitate dynamic classification of expertise for adaptive simulation.…”

Section: Related Workmentioning

confidence: 99%

Toward Dynamically Adaptive Simulation: Multimodal Classification of User Expertise Using Wearable Devices

Ross

Sarkar

Rodenburg

et al. 2019

Sensors

Self Cite

View full text Add to dashboard Cite

Simulation-based training has been proven to be a highly effective pedagogical strategy. However, misalignment between the participant’s level of expertise and the difficulty of the simulation has been shown to have significant negative impact on learning outcomes. To ensure that learning outcomes are achieved, we propose a novel framework for adaptive simulation with the goal of identifying the level of expertise of the learner, and dynamically modulating the simulation complexity to match the learner’s capability. To facilitate the development of this framework, we investigate the classification of expertise using biological signals monitored through wearable sensors. Trauma simulations were developed in which electrocardiogram (ECG) and galvanic skin response (GSR) signals of both novice and expert trauma responders were collected. These signals were then utilized to classify the responders’ expertise, successive to feature extraction and selection, using a number of machine learning methods. The results show the feasibility of utilizing these bio-signals for multimodal expertise classification to be used in adaptive simulation applications.

show abstract

“…Electrocardiogram (ECG) has been proven to be a reliable source of information for emotion recognition systems [1][2][3][4]. Automated ECG analysis can identify the affective states of users such as happiness, sadness, and stress, among others.…”

Section: Introductionmentioning

confidence: 99%

“…In [6], a dynamic difficulty adjustment mechanism for computer games was proposed to provide tailored gaming experience to individual users by analysing ECG and GSR. An ECG-based deep multitask learning framework was proposed in [3,7] for adaptive simulation. The aim was to provide personalised training experience to individual users based on their level of expertise and cognitive load.…”

Section: Introductionmentioning

confidence: 99%

Self-Supervised Learning for ECG-Based Emotion Recognition

Sarkar

Etemad

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

We present an electrocardiogram (ECG) -based emotion recognition system using self-supervised learning. Our proposed architecture consists of two main networks, a signal transformation recognition network and an emotion recognition network. First, unlabelled data are used to successfully train the former network to detect specific pre-determined signal transformations in the self-supervised learning step. Next, the weights of the convolutional layers of this network are transferred to the emotion recognition network, and two dense layers are trained in order to classify arousal and valence scores. We show that our self-supervised approach helps the model learn the ECG feature manifold required for emotion recognition, performing equal or better than the fully-supervised version of the model. Our proposed method outperforms the state-of-the-art in ECG-based emotion recognition with two publicly available datasets, SWELL and AMIGOS. Further analysis highlights the advantage of our self-supervised approach in requiring significantly less data to achieve acceptable results.

show abstract

Classification of Cognitive Load and Expertise for Adaptive Simulation using Deep Multitask Learning

Cited by 33 publications

References 40 publications

Towards Intelligent Data Analytics: A Case Study in Driver Cognitive Load Classification

Towards Intelligent Data Analytics: A Case Study in Driver Cognitive Load Classification

Toward Dynamically Adaptive Simulation: Multimodal Classification of User Expertise Using Wearable Devices

Self-Supervised Learning for ECG-Based Emotion Recognition

Contact Info

Product

Resources

About