Machine learning distinguishes neurosurgical skill levels in a virtual reality tumor resection task

Abstract

Background: Virtual reality simulators and machine learning have the potential to augment understanding, assessment and training of psychomotor performance in neurosurgery residents.Objective: This study outlines the first application of machine learning to distinguish "skilled" and "novice" psychomotor performance during a virtual reality neurosurgical task.Methods: Twenty-three neurosurgeons and senior neurosurgery residents comprising the "skilled" group and 92 junior neurosurgery residents and medical students the "novice" group. The task involved removing a series of virtual brain tumors without causing injury to surrounding tissue.Over 100 features were extracted and 68 selected using t-test analysis. These features were provided to 4 classifiers: K-Nearest Neighbors, Parzen Window, Support Vector Machine, and Fuzzy K-Nearest Neighbors. Equal Error Rate was used to assess classifier performance. Results:Ratios of train set size to test set size from 10% to 90% and 5 to 30 features, chosen by the forward feature selection algorithm, were employed. A working point of 50% train to test set size ratio and 15 features resulted in an equal error rates as low as 8.3% using the Fuzzy K-Nearest Neighbors classifier. Conclusion:Machine learning may be one component helping realign the traditional apprenticeship educational paradigm to a more objective model based on proven performance standards.

Effect of Artificial Intelligence Tutoring vs Expert Instruction on Learning Simulated Surgical Skills Among Medical Students

“…32 Because this simulator records timeseries data of users' interaction in the virtual space, 33 machine learning algorithms have been demonstrated to successfully differentiate surgical expertise based on validated performance metrics. [8][9][10]34 Virtual Reality Tumor Resection Procedures Subpial resection is a neurosurgical technique in oncologic and epilepsy surgery that requires coordinated bimanual psychomotor ability to resect pathologic tissue with preservation of surrounding brain and vessels. 35 The student's objective was to remove a simulated cortical tumor with minimal bleeding and damage to surrounding tissues using a simulated aspirator in the dominant hand and a simulated bipolar forceps in the nondominant hand (Video).…”

“…[5][6][7] Virtual reality simulation and machine learning algorithms can objectively quantify performance and improve the precision and granularity of bimanual technical skills classification. [8][9][10] These systems may enhance surgical educators' ability to develop more quantitative formative and summative assessment tools to manage future challenging pedagogic requirements. The COVID-19 pandemic has significantly altered surgical trainees' ability to obtain intraoperative instruction necessary for skill acquisition, 11 and innovative solutions, such as AI-powered tutoring systems, may help in addressing such disruptions.…”

Machine learning for technical skill assessment in surgery: a systematic review

“…Bad performance when imbalanced sampling datasets. • k-means clustering 14 , 17 , 77 , 80 , 86 , 87 , 89 , 94 – 96 , 104 Naïve Bayes Supervised classification algorithm based on Bayes Theorem. The simplified from of the Bayes Algorithm—Naïve Bayes is built with the assumption that features are conditionally independent.…”

Virtual Reality in the Neurosciences: Current Practice and Future Directions

“…The data may then be analyzed to quantify psychomotor skills in neurosurgical training (48-54). When paired with artificial intelligence (AI), links are created between medicine, computer science, and education that can collaboratively revolutionize surgical training (52). The utility of AI to assess psychomotor skills is still formative but has been shown to classify individuals into different expertise levels with an accuracy of over 90% (48)(49)(50)(51)(52).…”

“…When paired with artificial intelligence (AI), links are created between medicine, computer science, and education that can collaboratively revolutionize surgical training (52). The utility of AI to assess psychomotor skills is still formative but has been shown to classify individuals into different expertise levels with an accuracy of over 90% (48)(49)(50)(51)(52). By providing a new tool to classify surgical skills training, a shift in the longstanding paradigm of case volume being correlated with skill level could revolutionize the means in which residents are trained.…”