Augmented Reality with HoloLens® in Parotid Tumor Surgery: A Prospective Feasibility Study

Abstract

<b><i>Introduction:</i></b> Augmented reality can improve planning and execution of surgical procedures. Head-mounted devices such as the HoloLens® (Microsoft, Redmond, WA, USA) are particularly suitable to achieve these aims because they are controlled by hand gestures and enable contactless handling in a sterile environment. <b><i>Objectives:</i></b> So far, these systems have not yet found their way into the operating room for surgery of the parotid gland. This study explored the feasibility and accuracy of augmented reality-assisted parotid surgery. <b><i>Methods:</i></b> 2D MRI holographic images were created, and 3D holograms were reconstructed from MRI DICOM files and made visible via the HoloLens. 2D MRI slices were scrolled through, 3D images were rotated, and 3D structures were shown and hidden only using hand gestures. The 3D model and the patient were aligned manually. <b><i>Results:</i></b> The use of augmented reality with the HoloLens in parotic surgery was feasible. Gestures were recognized correctly. Mean accuracy of superimposition of the holographic model and patient’s anatomy was 1.3 cm. Highly significant differences were seen in position error of registration between central and peripheral structures (<i>p</i> = 0.0059), with a least deviation of 10.9 mm (centrally) and highest deviation for the peripheral parts (19.6-mm deviation). <b><i>Conclusion:</i></b> This pilot study offers a first proof of concept of the clinical feasibility of the HoloLens for parotid tumor surgery. Workflow is not affected, but additional information is provided. The surgical performance could become safer through the navigation-like application of reality-fused 3D holograms, and it improves ergonomics without compromising sterility. Superimposition of the 3D holograms with the surgical field was possible, but further invention is necessary to improve the accuracy.

Mixed Reality in Modern Surgical and Interventional Practice: Narrative Review of the Literature

“…In otolaryngology and maxillofacial surgery, studies have demonstrated improved understanding of tumor-related anatomy and planning of surgical approach when MR was used instead of conventional radiological imaging [ 8 , 9 ]. Mitani et al [ 8 ] found that MR facilitated recognition of dissection lines in parotid tumor surgery, where consideration of detailed maxillary anatomy is of critical importance.…”

“…In otolaryngology and maxillofacial surgery, evidence suggests that MR enables easier extracapsular dissection of parotid tumors, which is related to lower complication rates, improved preservation of glandular tissue, and decreased incidence of saliva fistulas [ 9 ]. Tian et al [ 35 ] reported satisfactory use of intraoperative MR in cochlear implant surgery due to improved positioning of the implant.…”

Augmented Reality-Based Surgery on the Human Cadaver Using a New Generation of Optical Head-Mounted Displays: Development and Feasibility Study

“…Manual registration, which showed a mean error of alignment of 12.4 mm, would be another option. After appropriate training or assistance by fiducial markers, the error was reduced to 10 mm [ 11 , 33 ]. Consequently, image-based registration with Vuforia, which is much more accurate, is still the method of choice for most applications [ 29 , 30 ].…”

“…Examples of applications in the operating theater are orthognathic surgery, oncology including parotid surgery, and traumatology. Anatomical and pathological structures, drilling and implant position, resection margins, and reconstructive planning are visualized using different AR technologies [ 10 , 11 ]. Similar examples can also be found on cadavers [ 12 - 14 ].…”

Key Ergonomics Requirements and Possible Mechanical Solutions for Augmented Reality Head-Mounted Displays in Surgery

“…General-purpose OST-HMDs, such as the Microsoft© HoloLens ™ 1 and 2 and Magic Leap ™ 1 devices, are increasingly proposed as promising tools in the context of computer-aided surgery. Their use has been demonstrated on phantoms [12][13][14][15], in cadaver studies [16,17], and/or in pre and intra-operative planning tasks [18][19][20][21][22]. More recently, one study demonstrated the use of Microsoft© HoloLens ™ 2 device in spine surgery on a real patient [23].…”