Augmented reality visualization for aiding biopsy procedure according to computed tomography based virtual plan

Abstract

Purpose: The purpose of this study was to develop and verify an intraoperative module for supporting navigated biopsy procedures using optical see-through head-mounted display (HMD). Methods: Biopsy procedure including entry and endpoints of needle insertion was planned preoperatively having regard to the resection region segmentation and safety margin definition. Biopsy procedures were performed by two users using an intraoperative optical navigation module on a specially prepared brain phantom. Two visualization techniques were compared: an accurate augmented reality one, where a virtual plan is superimposed onto surgical field by using optical see-through HMD together with personalized calibration method and visualization on the external display. Results: Averaged errors from 24 trials using external display were 2.04 ± 0.83 mm for the first user and 2.69 ± 1.11 mm for the second one, while applying HMD 2.50 ± 0.93 mm (the first user) and 2.17 ± 0.82 mm (the second user), respectively. Conclusions: Proper usage of HMD visualization preceded by the personalized calibration allows the user to perform navigated biopsy procedure with comparable accuracy to its equivalent with the external display. Additionally, augmented reality visualization improves ergonomics and enables focusing on the surgical field without losing a direct line of sight with the field of view as it happens for external displays. However, ensuring high accuracy of augmented reality visualization still requires proper calibration and some user experience, which is challenging.

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

“…The most commonly reported OST-HMD-led tracking solution was surface tracking, involving the use of the RGB camera capabilities of the headset, and square marker fiducials, such as those used in the Vuforia [ 64 , 69 , 74 , 96 ] (PTC, Boston, MA, USA) or ArUco [ 31 , 46 , 77 , 91 ] libraries. There were several papers which reported the use of custom markers for surface-based tracking, though the performance did not significantly differ from ArUco or Vuforia tracking solutions [ 82 , 97 ]. In the surface-based tracking strategy, the square marker fiducials served to localize the patient and any tracked tools relative to the reference frame of the OST-HMD, allowing for the overlay of virtual content [ 33 , 46 , 88 , 91 , 92 ].…”

“…In the context of OST-HMD use in surgery, perception refers to the quantitative assessment of a user’s interpretation and understanding of virtually augmented content. Challenges in perception of virtually augmented elements can be attributed to a number of factors, including incorrect IPD estimation [ 40 ], incorrect or unaccounted for per-user display calibration [ 33 , 97 ], display limitations due to vergence-accommodation [ 10 ], misregistration of virtual content due to failed registration [ 106 ], and limitations of depth understanding of virtually augmented content [ 107 ]. Perceptual limitations contributed to additional errors in the stability of calibration during ultrasound image-guided robot cervical pedicle screw placement [ 93 ], with navigation during total shoulder arthroscopy [ 99 ], and with marker-less tracking for guidance in head and neck carcinoma imaging [ 75 ].…”

“…Attention shift describes the requirement for a surgeon to frequently gaze between the provided navigation information on an external display and the patient; with an OST-HMD, virtual guidance information can be brought directly into the FOV of the wearer to remove this requirement. Improved spatial awareness and mental mapping as compared with traditional guidance paradigms was reported by numerous articles across different surgical applications [ 33 , 46 , 55 , 75 , 97 ]. Reduced attention shift by using an OST-HMD for the display of guidance information was measured in fluoroscopic imaging for orthopedic surgery [ 81 ] and ultrasound imaging for ultrasound-guided procedures [ 92 ].…”

Augmented Reality in Stereotactic Neurosurgery: Current Status and Issues

“…The first research direction is the evaluation of the feasibility of ARN-guided frameless stereotaxy. [4][5][6] ARN-guided stereotaxy for cranial phantoms showed that this procedure has the advantage of allowing the surgeon to advance the needle in a natural posture. The second research direction is the application of ARN in improving the safety of frame-based stereotaxy.…”

“…Recent progress in the accuracy and reliability of AR systems has led to the increased use of AR in stereotactic neurosurgery over the past several years. [4][5][6][7] In this report, we review the feasibility of AR as a surgical support modality or as a distinct approach to stereotactic neurosurgery. Our previous study included in this report was approved by the Jichi Medical University Hospital clinical research ethics committee as a study on the "development and clinical application of AR neuronavigation" (ID B15-116 and 20-143).…”