Study on the Identification of Irritability Emotion Based on the Percentage Change in Pupil Size

Zhao, Minrui; Hongni, Gao; Wang, Wei; Qu, Jinping; Chen, Long

doi:10.1145/3388818.3389166

Cited by 3 publications

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To eliminate the subject-based variations in pupil diameter, we calculated the percentage change in pupil size (PCPS) utilizing the following equation [71]:…”

Section: Pupillometrymentioning

confidence: 99%

See 1 more Smart Citation

Assessing the Effects of Various Physiological Signal Modalities on Predicting Different Human Cognitive States

Aygun,

Nguyen,

Scheutz

2024

Preprint

View full text Add to dashboard Cite

Robust estimation of systemic human cognitive states is critical for a variety of applications, from simply detecting inefficiencies in task assignments, to the adaptation of artificial agents’ behaviors to improve team performance in mixed-initiative human-machine teams. This study showed that human eye gaze, in particular, the percentage change in pupil size (PCPS), is the most reliable biomarker for assessing three human cognitive states including workload, sense of urgency, and mind wandering compared to electroencephalogram (EEG), functional near-infrared spectroscopy (fNIRS), respiration, and skin conductance. We used comprehensive multi-modal driving dataset to examine the accuracy of signals to assess these cognitive states. We performed comprehensive statistical tests to validate the performance of several physiological signals to determine human cognitive states and demonstrated that PCPS shows noticeably superior performance. We also characterized the link between workload and sense of urgency with eye gaze and observed that consecutive occurrences of higher sense of urgency were prone to increase overall workload. Finally, we trained five machine learning (ML) models and showed that four of them had similar accuracy in cognitive state classification (with one, random forest, showing inferior performance). The results provided evidence that the PCPS is a reliable physiological marker for cognitive state estimation.

show abstract

“…To eliminate the subject-based variations in pupil diameter, we calculated the percentage change in pupil size (PCPS) utilizing the following equation [71]:…”

Section: Pupillometrymentioning

confidence: 99%

“…Here, BM is determined by calculating the mean of a 10-seconds signal before the stimulus. Additionally, we calculated the average PCPS (APCPS) as follows [71]:…”

Section: Pupillometrymentioning

confidence: 99%

Assessing the Effects of Various Physiological Signal Modalities on Predicting Different Human Cognitive States

Aygun,

Nguyen,

Scheutz

2024

Preprint

View full text Add to dashboard Cite

show abstract

Cognitive Workload Assessment via Eye Gaze and EEG in an Interactive Multi-Modal Driving Task

Aygun

Lyu

Nguyen

et al. 2022

Proceedings of the 2022 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Investigating Methods for Cognitive Workload Estimation for Assistive Robots

Aygun

Nguyen

Haga

et al. 2022

Sensors

View full text Add to dashboard Cite

Robots interacting with humans in assistive contexts have to be sensitive to human cognitive states to be able to provide help when it is needed and not overburden the human when the human is busy. Yet, it is currently still unclear which sensing modality might allow robots to derive the best evidence of human workload. In this work, we analyzed and modeled data from a multi-modal simulated driving study specifically designed to evaluate different levels of cognitive workload induced by various secondary tasks such as dialogue interactions and braking events in addition to the primary driving task. Specifically, we performed statistical analyses of various physiological signals including eye gaze, electroencephalography, and arterial blood pressure from the healthy volunteers and utilized several machine learning methodologies including k-nearest neighbor, naive Bayes, random forest, support-vector machines, and neural network-based models to infer human cognitive workload levels. Our analyses provide evidence for eye gaze being the best physiological indicator of human cognitive workload, even when multiple signals are combined. Specifically, the highest accuracy (in %) of binary workload classification based on eye gaze signals is 80.45 ∓ 3.15 achieved by using support-vector machines, while the highest accuracy combining eye gaze and electroencephalography is only 77.08 ∓ 3.22 achieved by a neural network-based model. Our findings are important for future efforts of real-time workload estimation in the multimodal human-robot interactive systems given that eye gaze is easy to collect and process and less susceptible to noise artifacts compared to other physiological signal modalities.

show abstract

Study on the Identification of Irritability Emotion Based on the Percentage Change in Pupil Size

Cited by 3 publications

References 7 publications

Assessing the Effects of Various Physiological Signal Modalities on Predicting Different Human Cognitive States

Assessing the Effects of Various Physiological Signal Modalities on Predicting Different Human Cognitive States

Cognitive Workload Assessment via Eye Gaze and EEG in an Interactive Multi-Modal Driving Task

Investigating Methods for Cognitive Workload Estimation for Assistive Robots

Contact Info

Product

Resources

About