Continuous monitoring of surgical bimanual expertise using deep neural networks in virtual reality simulation

Abstract

In procedural-based medicine, the technical ability can be a critical determinant of patient outcomes. Psychomotor performance occurs in real-time, hence a continuous assessment is necessary to provide action-oriented feedback and error avoidance guidance. We outline a deep learning application, the Intelligent Continuous Expertise Monitoring System (ICEMS), to assess surgical bimanual performance at 0.2-s intervals. A long-short term memory network was built using neurosurgeon and student performance in 156 virtually simulated tumor resection tasks. Algorithm predictive ability was tested separately on 144 procedures by scoring the performance of neurosurgical trainees who are at different training stages. The ICEMS successfully differentiated between neurosurgeons, senior trainees, junior trainees, and students. Trainee average performance score correlated with the year of training in neurosurgery. Furthermore, coaching and risk assessment for critical metrics were demonstrated. This work presents a comprehensive technical skill monitoring system with predictive validation throughout surgical residency training, with the ability to detect errors.

AI in Surgical Curriculum Design and Unintended Outcomes for Technical Competencies in Simulation Training

“…With advancements in live intraoperative data acquisition for quality improvement, AI systems are believed to play a key role in mitigating adverse events and suggesting instructions in the operating room . Rolling out intelligent systems for patient care or formal postgraduate training should occur in phases and requires the same rigor of scientific practice required for pharmacologic therapies or other medical devices.…”

“…However, most participants in the skilled group primarily practiced cranial neurosurgery, and subpial resection is a fundamental technique that is mastered throughout postgraduate training . Furthermore, previously validated models trained on this sample could not only distinguish expertise but project residents’ postgraduate training year in neurosurgery . Although best practice is to compare trainee performance with quantifiable criterion standards of experts, training medical students to achieve benchmarks closer to their competency level, for example that of a junior resident, may prove more practical .…”

“…In surgical education, technical competency is one of the key factors of efficacy in the curriculum, and it is an independent factor that is directly associated with postoperative patient outcomes . By gathering large amounts of data from surgical simulation training, machine learning and deep learning algorithms can be used to design intelligent tutoring systems to deliver a curriculum enhanced with artificial intelligence (AI) . An intelligent tutoring system is a pedagogical tool powered by an AI model that can provide learners with tailored performance assessment and feedback .…”

Investigating and acquiring motor expertise using virtual reality

“…The most established paradigm of this kind is surgical training [50,51]. Several studies have shown construct validity of virtual environments by differentiating among novice, intermediate, and experts based on their surgical performance in VR [52][53][54], and others have shown improvement in specific surgical skills via simulated practice [55,56]. However, these studies have not explicitly tested specific hypotheses about motor expertise.…”

Real-time simulation of viscoelastic tissue behavior with physics-guided deep learning

“…Additionally, the availability of high precision data streams from surgical simulation allows to automatically categorize the level of expertise of the trainee using machine learning algorithms [32]. Continuous expertise monitoring systems can further assess surgical bimanual performance in real-time, which could be leveraged to provide predictive validation during surgical residency training, allowing the early detection of errors [30] and more efficient training. Virtual reality simulators thus enable trainees to practice on a variety of educational scenarios as well as to enable the definition of new training metrics and applications (e.g.…”