Machine learning-based analysis of operator pupillary response to assess cognitive workload in clinical ultrasound imaging

Abstract

Introduction Pupillometry, the measurement of eye pupil diameter, is a well-established and objective modality correlated with cognitive workload. In this paper, we analyse the pupillary response of ultrasound imaging operators to assess their cognitive workload, captured while they undertake routine fetal ultrasound examinations. Our experiments and analysis are performed on real-world datasets obtained using remote eye-tracking under natural clinical environmental conditions. Methods Our analysis pipeline involves careful temporal sequence (time-series) extraction by retrospectively matching the pupil diameter data with tasks captured in the corresponding ultrasound scan video in a multi-modal data acquisition setup. This is followed by the pupil diameter pre-processing and the calculation of pupillary response sequences. Exploratory statistical analysis of the operator pupillary responses and comparisons of the distributions between ultrasonographic tasks (fetal heart versus fetal brain) and operator expertise (newly-qualified versus experienced operators) are performed. Machine learning is explored to automatically classify the temporal sequences into the corresponding ultrasonographic tasks and operator experience using temporal, spectral, and time-frequency features with classical (shallow) models, and convolutional neural networks as deep learning models. Results Preliminary statistical analysis of the extracted pupillary response shows a significant variation for different ultrasonographic tasks and operator expertise, suggesting different extents of cognitive workload in each case, as measured by pupillometry. The best-performing machine learning models achieve receiver operating characteristic (ROC) area under curve (AUC) values of 0.98 and 0.80, for ultrasonographic task classification and operator experience classification, respectively. Conclusion We conclude that we can successfully assess cognitive workload from pupil diameter changes measured while ultrasound operators perform routine scans. The machine learning allows the discrimination of the undertaken ultrasonographic tasks and scanning expertise using the pupillary response sequences as an index of the operators’ cognitive workload. A high cognitive workload can reduce operator efficiency and constrain their decision-making, hence, the ability to objectively assess cognitive workload is a first step towards understanding these effects on operator performance in biomedical applications such as medical imaging.

Pupil Dilation as an Index of Examinee's Cognitive Load in Answering a Mathematics Question: A Comparison Study of Different Approaches

“…Although this approach considers BPD correction, it also reveals that TEPRs can vary nonlinearly in a manner similar to autonomic functions such as heart rate (i.e., magnitude of evoked responses decreases with an increasing baseline; Reilly et al, 2019). Because the magnitude of evoked TEPRs is lower for those with a higher baseline value, the interindividual differences in variability in dilation must be considered (Sharma, Drukker, Papageorghiou, & Noble, 2021; Winn et al, 2018). Winn et al (2018) and Sharma et al (2021) also highlighted that pupil diameter has a limited value and biological traits and that dilation range may vary; thus, the proportional change from baseline or the dynamic range of an individual's pupil size can be used to address the biological traits of dilation variability such that the measurements from multiple participants can be combined.…”

Research on Community Risk Prediction Model and Management Based on Deep Learning

“…The larger the value of R m is, the higher the correlation degree is. When the correlation is less than 0.05, there is no significant correlation between variables [22]. 1.…”

Thelxinoë: Recognizing Human Emotions Using Pupillometry and Machine Learning

“…We removed the noise in the form of blinks. We also removed samples where one eye was closed, which usually either led up to or followed a blink event [19]. The other pre-processing that we carried us was replacing the date and time with a timestamp.…”