Wearable technology-based metrics for predicting operator performance during cardiac catheterisation

Abstract

IntroductionUnobtrusive metrics that can auto-assess performance during clinical procedures are of value. Three approaches to deriving wearable technology-based metrics are explored: (1) eye tracking, (2) psychophysiological measurements [e.g. electrodermal activity (EDA)] and (3) arm and hand movement via accelerometry. We also measure attentional capacity by tasking the operator with an additional task to track an unrelated object during the procedure.MethodsTwo aspects of performance are measured: (1) using eye gaze and psychophysiology metrics and (2) measuring attentional capacity via an additional unrelated task (to monitor a visual stimulus/playing cards). The aim was to identify metrics that can be used to automatically discriminate between levels of performance or at least between novices and experts. The study was conducted using two groups: (1) novice operators and (2) expert operators. Both groups made two attempts at a coronary angiography procedure using a full-physics virtual reality simulator. Participants wore eye tracking glasses and an E4 wearable wristband. Areas of interest were defined to track visual attention on display screens, including: (1) X-ray, (2) vital signs, (3) instruments and (4) the stimulus screen (for measuring attentional capacity).ResultsExperts provided greater dwell time (63% vs 42%, p = 0.03) and fixations (50% vs 34%, p = 0.04) on display screens. They also provided greater dwell time (11% vs 5%, p = 0.006) and fixations (9% vs 4%, p = 0.007) when selecting instruments. The experts’ performance for tracking the unrelated object during the visual stimulus task negatively correlated with total errors (r = − 0.95, p = 0.0009). Experts also had a higher standard deviation of EDA (2.52 µS vs 0.89 µS, p = 0.04).ConclusionsEye tracking metrics may help discriminate between a novice and expert operator, by showing that experts maintain greater visual attention on the display screens. In addition, the visual stimulus study shows that an unrelated task can measure attentional capacity.Trial registration This work is registered through clinicaltrials.gov, a service of the U.S. National Health Institute, and is identified by the trial reference: NCT02928796.

A Systematic Review of Physiological Measurements, Factors, Methods, and Applications in Virtual Reality

“…This indicates that they were better able to cope with the stress (Clifford et al, 2020). Currie et al (2019) worked with a similar approach. Their virtual training environment is focused on a high-fidelity surgical procedure.…”

European expert consensus on a structured approach to training robotic‐assisted low anterior resection using performance metrics

“…OSATS and GEARS) [26,28] and provide the trainee with objective, transparent and explicit feedback which is derived from a consensus of expert senior surgeons [11][12][13]15,17]. In contrast to hand/tool 'efficiency' measures these types of performance metrics will also underpin and bolster the development of effective virtual reality simulation and machine learning [20,[39][40][41].…”

Simulation Training in Neuroangiography—Validation and Effectiveness

“…Observations of gaze direction, blink frequency, pupil size and dwell time are recognized means of examining attention and cognitive stress [29]. Based on such data Richstone et al were able to distinguish unequivocally between beginners and experts in surgical laparoscopy [30], but for endovascular cardiac interventions, Currie et al found hardly any differences [31]. The assumption that experts would turn their gaze less often away from the X-ray screen could therefore not be confirmed by them.…”

“…Currie et al. also measured numerous parameters such as heart rate variability, electrodermal activity and skin temperature in physicians undertaking cardiac endovascular procedures but could not find differences between experience levels [ 31 ]. Hence, mean heart rate seems to be an unreliable predictor, matching the results of our study, where it was not suitable to distinguish between beginners and experts.…”