Assessing performance of augmented reality-based neurosurgical training

Abstract

This paper presents a novel augmented reality (AR)-based neurosurgical training simulator which provides a very natural way for surgeons to learn neurosurgical skills. Surgical simulation with bimanual haptic interaction is integrated in this work to provide a simulated environment for users to achieve holographic guidance for pre-operative training. To achieve the AR guidance, the simulator should precisely overlay the 3D anatomical information of the hidden target organs in the patients in real surgery. In this regard, the patient-specific anatomy structures are reconstructed from segmented brain magnetic resonance imaging. We propose a registration method for precise mapping of the virtual and real information. In addition, the simulator provides bimanual haptic interaction in a holographic environment to mimic real brain tumor resection. In this study, we conduct AR-based guidance validation and a user study on the developed simulator, which demonstrate the high accuracy of our AR-based neurosurgery simulator, as well as the AR guidance mode's potential to improve neurosurgery by simplifying the operation, reducing the difficulty of the operation, shortening the operation time, and increasing the precision of the operation.

Evaluation of the precision of operative augmented reality compared to standard neuronavigation using a 3D-printed skull

“…[20][21][22][23] Apprenticeship, surgical experience, and hands-on experience have traditionally been a challenge for young neurosurgical residents. 22,23 ,25-27 Si et al 23 have proposed in their works results from a test of a new AR-guided neurosurgical simulator with haptic answers. After producing a 3D-printed skull from a patient and registering it, 10 participants without neurosurgical experience "in real life" were enrolled.…”

“…[18][19] Together with virtual reality (or the simulation of a virtual environment with which the user can interact), AR offers great opportunities in the fields of education and neurosurgical training. [20][21][22][23] In particular during cranial surgeries, AR has gained popularity because it offers 3 main advantages that are not offered by standard NV: 1) it allows direct projection of preregistered overlaid images on the real-world view while using the surgical microscope; 2) the surgeon keeps his or her focus on the surgical field; 24 and 3) modern recalibration tools palliate inherent imprecision, such as brain shifts during resection. The surgeon is able to reregister the AR according to intracranial reference structures such as vessels or bones.…”

Augmented reality and physical hybrid model simulation for preoperative planning of metopic craniosynostosis surgery

“…The hybrid model we present here allowed a detailed discussion of the aspects of the surgical approach by presenting different consistencies and resistances of the lifelike material and enabling the prediction of bilateral orbitotomy measurements with millimetric accuracy. 12 We believe that hybrid models allow for enhanced insight into the underlying anatomy since they supply comprehensive 3D imagery and tactile feedback for each abstraction. Compared with traditional imaging and other methods of advanced imaging visualization such as 3D computer models, the 3D-printed models, as delineated by Wake et al, individually join multisensory inputs of touch and vision, leading to improved spatial conceptualization compared with simple 3D anatomical images viewed as a computer model.…”

Applications of Haptic Technology, Virtual Reality, and Artificial Intelligence in Medical Training During the COVID-19 Pandemic

“…In conjunction with VR, the AR technology has also been used in various medical fields for training such as neurosurgical training ( Si et al, 2019 ). Anatomical information and other sensory information can be visualized to the surgeons more properly, and therefore, more accurate decision can be made during a surgery.…”