Multimodal Physiological Signals for Workload Prediction in Robot-assisted Surgery

Abstract

Monitoring surgeon workload during robot-assisted surgery can guide allocation of task demands, adapt system interfaces, and assess the robotic system's usability. Current practices for measuring cognitive load primarily rely on questionnaires that are subjective and disrupt surgical workflow. To address this limitation, a computational framework is demonstrated to predict user workload during telerobotic surgery. This framework leverages wireless sensors to monitor surgeons' cognitive load and predict their cognitive states. Continuous data across multiple physiological modalities (e.g., heart rate variability, electrodermal, and electroencephalogram activity) were simultaneously recorded for twelve surgeons performing surgical skills tasks on the validated da Vinci Skills Simulator. These surgical tasks varied in difficulty levels, e.g., requiring varying visual processing demand and degree of fine motor control. Collected multimodal physiological signals were fused using independent component analysis, and the predicted results were compared to the ground-truth workload level. Results compared performance of different classifiers, sensor fusion schemes, and physiological modality (i.e., prediction with single vs. multiple modalities). It was found that our multisensor approach outperformed individual signals and can correctly predict cognitive workload levels 83.2% of the time during basic and complex surgical skills tasks. CCS Concepts: • Human-centered computing → Human computer interaction (HCI); Interactive systems and tools; User interface management systems;

Multi-modal physiological sensing approach for distinguishing high workload events in remotely piloted aircraft simulation

“…Another method would be to examine moments where physiological activation increased and exploring RPA tasks contributors to the increase. Many metrics and methods exist for examining physiological metrics during training, and further work can explore additional metrics (e.g., eye-tracking, engagement index (Berka et al 2004), sleep onset, and data fusion (Zhou et al 2020)) and optimize time windows for workload prediction. For example, physiological response times or delays in response times can impact inferences drawn from these signals, especially for the short time windows of the studied RPA events.…”

Methods and Measures for Mental Stress Assessment in Surgery: A Systematic Review of 20 Years of Literature

“…While the majority (92.96%) of selected papers used statistical methods, only 14.08% applied advanced regression models, and only one reviewed study conducted feature selection and classification for stress assessment [62]. Applying modern ML techniques in this context demands further research since, while multi-modal datasets enable information fusion from different temporal and spatial resolutions for decision making, the intercorrelation of different modalities may decrease the accuracy of their predictions [158], [159].…”

Protocol for a scoping review on ‘surgical sabermetrics:’ technology-enhanced measurement of operative non-technical skills

“…Additionally, these ratings are influenced by subject variability. 7 A large proportion of tasks in surgery are related to cognition. A surgeon's NTSs consist of several cognitive processes, including situational awareness, decision making, teamwork and communication skills.…”