Development of an Intraoperative Pipeline for Holographic Mixed Reality Visualization During Spinal Fusion Surgery

Abstract

Objective. Holographic mixed reality (HMR) allows for the superimposition of computer-generated virtual objects onto the operator’s view of the world. Innovative solutions can be developed to enable the use of this technology during surgery. The authors developed and iteratively optimized a pipeline to construct, visualize, and register intraoperative holographic models of patient landmarks during spinal fusion surgery. Methods. The study was carried out in two phases. In phase 1, the custom intraoperative pipeline to generate patient-specific holographic models was developed over 7 patients. In phase 2, registration accuracy was optimized iteratively for 6 patients in a real-time operative setting. Results. In phase 1, an intraoperative pipeline was successfully employed to generate and deploy patient-specific holographic models. In phase 2, the registration error with the native hand-gesture registration was 20.2 ± 10.8 mm (n = 7 test points). Custom controller-based registration significantly reduced the mean registration error to 4.18 ± 2.83 mm (n = 24 test points, P < .01). Accuracy improved over time (B = −.69, P < .0001) with the final patient achieving a registration error of 2.30 ± .58 mm. Across both phases, the average model generation time was 18.0 ± 6.1 minutes (n = 6) for isolated spinal hardware and 33.8 ± 8.6 minutes (n = 6) for spinal anatomy. Conclusions. A custom pipeline is described for the generation of intraoperative 3D holographic models during spine surgery. Registration accuracy dramatically improved with iterative optimization of the pipeline and technique. While significant improvements and advancements need to be made to enable clinical utility, HMR demonstrates significant potential as the next frontier of intraoperative visualization.

Augmented Reality in Spinal Surgery: Highlights From Augmented Reality Lectures at the Emerging Technologies Annual Meetings

“…Twenty-one percent of the results were studies focusing on the proof of concept of AR. This included testing efficacy in cadaveric models [12][13][14] as well as comparing the efficacy of pedicle screw instrumentation in AR with the use of navigation assistance or freehand techniques [15,16]. Feasibility/usability studies were defined as investigations into the practicality, strengths and weaknesses of AR in the operating room (OR), this category accounting for 26% of the query.…”

Spine Surgery Assisted by Augmented Reality: Where Have We Been?

“…Second, the manual calibration of holograms and models is tedious due to inaccurate mobile hologram gestures or voice control. 10 65 Due to the aforementioned technique concerns, Buch, et al 66 introduced a generalizable pipeline for improving registration accuracy, and the results significantly reduced registration errors. Furthermore, the excessive hardware head weight caused operator discomfort after a long period of use.…”

Virtual, Augmented, and Mixed Reality Applications for Surgical Rehearsal, Operative Execution, and Patient Education in Spine Surgery: A Scoping Review

“…They demonstrated 96% accuracy in screw placements using VisAR microscopy [23]. A similar study by Buch et al used intraoperative holographic models to register landmarks during spinal fusion surgery as a proof of concept that must also be further explored [44]. Another study used intraoperative cone beam CT scan combined with Allura ARSN in a controlled study for spinal fixation surgeries aiming to reduce the need for a postoperative CT scan as well as the accurate placement of pedicle screws [77].…”

“…Additionally, Gu et al demonstrated a reduction in the risk, operative time, radiation exposure, and bleeding in patients who had undergone lumbar pedicle screw placement with HoloLens technology used with MR compared with patients who had undergone traditional C-arm fluoroscopy methods [42]. Buch et al further established the applicability of MR in spinal fusion by developing a method of creating and iteratively optimizing holographic model constructs from patient landmarks in a real-time, intraoperative setting [44]. While such initial results illustrate the potential benefits associated with MR technological applications for spinal surgeries, further research must be conducted on a larger scale to develop a solid pipeline for its implementation.…”