Virtual dissection of the real brain: integration of photographic 3D models into virtual reality and its effect on neurosurgical resident education

Abstract

OBJECTIVE Virtual reality (VR) is increasingly being used for education and surgical simulation in neurosurgery. So far, the 3D sources for VR simulation have been derived from medical images, which lack real color. The authors made photographic 3D models from dissected cadavers and integrated them into the VR platform. This study aimed to introduce a method of developing a photograph-integrated VR and to evaluate the educational effect of these models. METHODS A silicone-injected cadaver head was prepared. A CT scan of the specimen was taken, and the soft tissue and skull were segmented to 3D objects. The cadaver was dissected layer by layer, and each layer was 3D scanned by a photogrammetric method. The objects were imported to a free VR application and layered. Using the head-mounted display and controllers, the various neurosurgical approaches were demonstrated to neurosurgical residents. After performing hands-on virtual surgery with photographic 3D models, a feedback survey was collected from 31 participants. RESULTS Photographic 3D models were seamlessly integrated into the VR platform. Various skull base approaches were successfully performed with photograph-integrated VR. During virtual dissection, the landmark anatomical structures were identified based on their color and shape. Respondents rated a higher score for photographic 3D models than for conventional 3D models (4.3 ± 0.8 vs 3.2 ± 1.1, respectively; p = 0.001). They responded that performing virtual surgery with photographic 3D models would help to improve their surgical skills and to develop and study new surgical approaches. CONCLUSIONS The authors introduced photographic 3D models to the virtual surgery platform for the first time. Integrating photographs with the 3D model and layering technique enhanced the educational effect of the 3D models. In the future, as computer technology advances, more realistic simulations will be possible.

Development and Validation of a Novel Methodological Pipeline to Integrate Neuroimaging and Photogrammetry for Immersive 3D Cadaveric Neurosurgical Simulation

“…Neurosurgical education and training have been gradually moving toward virtual reality ( 5 , 33 ). The COVID-19 pandemic further accelerated this trend.…”

“…The process of creating 3D virtual models in medicine requires the segmentation of CT or MRI, then volume rendering ( 35 ), which produces surface models with high spatial resolution. This technique is currently useful for identifying a pathologic location and surgical planning; however, the current 3D models lack color, texture, and fine anatomical details, such as fine blood vessels, cranial nerves, and arachnoid membranes, which cannot be reproduced ( 5 ). The next step in the evolution of 3D anatomical and virtual simulations is the development of models that are both spatially accurate and have fine anatomical detail and realistic textures.…”

“…A possible solution to this limitation is to incorporate photogrammetry into building the models. Photogrammetry creates 3D representations using multiple 2D photographs of an object and has long been used in anatomy to bring 3D perception to 2D photographs or to create virtual 3D representations of anatomical specimens ( 1 , 5 , 10 , 36 – 38 ). De Benedictis et al ( 36 ) reported quantitative validation of photogrammetry for the study of white matter connectivity of the human brain.…”

“…Although their study used a relatively sophisticated setup, our group recently showed the applicability of a freely available 360° photogrammetry tool for neuroanatomy studies ( 1 ). In addition, Roh et al ( 5 ) also incorporated photogrammetry into a virtual 3D environment to create realistic texture details of cadaveric brain specimens, but their study lacked quantitative validity. Nonetheless, these studies clearly show the potential of this rapidly developing technology in enhancing neurosurgical and neuroanatomy training.…”

Identification of cranial nerve ganglia using sectioned images and three-dimensional models of a cadaver

“…This is because of the long time required for the cadaver sectioning. Most of other cadaver sectioned images are inadequate for the observation of minute ganglia, owing to their low resolution or artifacts [ 39 ]. For investigating differences in the cranial nerve ganglia between individuals, future studies using histologic or plastination specimens can be conducted.…”