Value of the surgeon’s sightline on hologram registration and targeting in mixed reality

Abstract

Purpose Mixed reality (MR) is being evaluated as a visual tool for surgical navigation. Current literature presents unclear results on intraoperative accuracy using the Microsoft HoloLens 1®. This study aims to assess the impact of the surgeon’s sightline in an inside-out marker-based MR navigation system for open surgery. Methods Surgeons at Akershus University Hospital tested this system. A custom-made phantom was used, containing 18 wire target crosses within its inner walls. A CT scan was obtained in order to segment all wire targets into a single 3D-model (hologram). An in-house software application (CTrue), developed for the Microsoft HoloLens 1, uploaded 3D-models and automatically registered the 3D-model with the phantom. Based on the surgeon’s sightline while registering and targeting (free sightline /F/or a strictly perpendicular sightline /P/), 4 scenarios were developed (FF-PF-FP-PP). Target error distance (TED) was obtained in three different working axes-(XYZ). Results Six surgeons (5 males, age 29–62) were enrolled. A total of 864 measurements were collected in 4 scenarios, twice. Scenario PP showed the smallest TED in XYZ-axes mean = 2.98 mm ± SD 1.33; 2.28 mm ± SD 1.45; 2.78 mm ± SD 1.91, respectively. Scenario FF showed the largest TED in XYZ-axes with mean = 10.03 mm ± SD 3.19; 6.36 mm ± SD 3.36; 16.11 mm ± SD 8.91, respectively. Multiple comparison tests, grouped in scenarios and axes, showed that the majority of scenario comparisons had significantly different TED values (p < 0.05). Y-axis always presented the smallest TED regardless of scenario tested. Conclusion A strictly perpendicular working sightline in relation to the 3D-model achieves the best accuracy results. Shortcomings in this technology, as an intraoperative visual cue, can be overcome by sightline correction. Incidentally, this is the preferred working angle for open surgery.

Accuracy assessment of marker recognition using ultra wide angle camera

“…The main advantage of such markers is that camera pose can be obtained with only one marker usage [13]. Moreover, binary square fiducial markers are widely used in medical research and practice, as they are simple, highly reliable, and multifunctional solution [14][15][16].…”

Registration Techniques for Clinical Applications of Three-Dimensional Augmented Reality Devices

“…In addition to the method and quality of the registration, the user’s viewing angle and the quality of the AR visualization can affect AR-guided performance. The importance of viewing angle was demonstrated by a study that found that strictly perpendicular sightlines resulted in better accuracy than free sightlines when guiding a needle tip to a target with 3D-AR guidance [58] . Another important factor in 3D-AR visualization is the user’s inter-pupillary distance (IPD).…”

Mixed reality technology enhances teaching of spinal blockade procedures

“…Mixed reality (MR) technology is a new digital holographic imaging technology that emerged after virtual reality (VR) and augmented reality (AR) (Barteit et al, 2021), which combines a realistic physical model with a pre‐constructed virtual image for its display (Dennler et al, 2020; Wang et al, 2020). This allows the wearer to “see through the eyes” of a surgeon while feeling a physical model (Wang et al, 2019), thus assisting students to learn relevant procedures (Luzon et al, 2020). The purpose of this study is to investigate whether MR technology‐assisted teaching can improve the effectiveness of teaching posterior medial spinal nerve blocks and to investigate students' attitudes toward this new approach.…”

“…The high cost of MR devices is not conducive to the budget of some hospitals and schools. The navigation effect and accuracy of MR devices in clinical practice are influenced by the patient's breathing, body shape, and tissue deformation, which can easily lead to errors in punctures (Luzon et al, 2020).…”