Ensemble deep learning for the prediction of proficiency at a virtual simulator for robot-assisted surgery

Abstract

Background Artificial intelligence (AI) has the potential to enhance patient safety in surgery, and all its aspects, including education and training, will derive considerable benefit from AI. In the present study, deep-learning models were used to predict the rates of proficiency acquisition in robot-assisted surgery (RAS), thereby providing surgical programs directors information on the levels of the innate ability of trainees to facilitate the implementation of flexible personalized training. Methods 176 medical students, without prior experience with surgical simulators, were trained to reach proficiency in five tasks on a virtual simulator for RAS. Ensemble deep neural networks (DNN) models were developed and compared with other ensemble AI algorithms, i.e., random forests and gradient boosted regression trees (GBRT). Results DNN models achieved a higher accuracy than random forests and GBRT in predicting time to proficiency, 0.84 vs. 0.70 and 0.77, respectively (Peg board 2), 0.83 vs. 0.79 and 0.78 (Ring walk 2), 0.81 vs 0.81 and 0.80 (Match board 1), 0.79 vs. 0.75 and 0.71 (Ring and rail 2), and 0.87 vs. 0.86 and 0.84 (Thread the rings 2). Ensemble DNN models outperformed random forests and GBRT in predicting number of attempts to proficiency, with an accuracy of 0.87 vs. 0.86 and 0.83, respectively (Peg board 2), 0.89 vs. 0.88 and 0.89 (Ring walk 2), 0.91 vs. 0.89 and 0.89 (Match board 1), 0.89 vs. 0.87 and 0.83 (Ring and rail 2), and 0.96 vs. 0.94 and 0.94 (Thread the rings 2). Conclusions Ensemble DNN models can identify at an early stage the acquisition rates of surgical technical proficiency of trainees and identify those struggling to reach the required expected proficiency level.

Clinical applications of artificial intelligence in robotic surgery

“…Moglia et al utilized ensemble deep learning neural network models to identify at an early stage the acquisition rates of surgical technical proficiency of trainees. [29] Hung et al [31] previously used robotic surgical APMs during RARP and clinicopathological data to accurately predict continence after RARP, achieving a C-index of 0.6 via a DL model (DeepSurv) [31]. The study demonstrated that surgeons with more efficient APMs achieved higher continence rates at 3 and 6 months post-RARP.…”

Evolution of a surgical system using deep learning in minimally invasive surgery (Review)

“…Moglia et al used deep learning to develop a model to predict the proficiency of medical students in a surgical simulator based on their training data. Subsequently, they used the model to predict proficiency from simulator data of untrained medical students with an accuracy rate of >80% ( 27 ). Excessive stress experienced by surgeons can negatively affect their surgical procedures, and Zheng et al developed a deep-learning model to detect, in real time, the movements of surgical procedures in which surgeons appear to be under stress.…”

“…Although it will still require a long time before the development of a fully autonomous surgical system, the development of a hybrid system may be realized in the near future. Therefore, it is necessary to discuss ethical issues related to autonomous surgical systems ( 27 ).…”

“…In a three-part sequence of operations, including input, internal state (deep learning algorithm), and output, full transparency is not ensured. This is especially true for the algorithm, owing to the ‘black box’ design inherent in deep-learning systems and the non-disclosure of source code for reasons of copyright law and protection of trade secrets ( 12 , 27 ). This prevents physicians and patients from trusting robots.…”

“…One proposed solution regarding liability is that, as in the case of autonomous vehicles, the surgeon stays in the same room as the surgical robot, which can be controlled by the physician at any time. Another option is to use a limited system that does not give full autonomy but assists the surgeon in routine surgery ( 12 , 27 ). Discussion of ethical and other regulations should be completed by the time an autonomous surgical robot is developed that is safe and has more consistent performance than a human surgeon.…”