A hand‐eye calibration method for augmented reality applied to computer‐assisted orthopedic surgery

Oliveira, Marcelo Elias de; Debarba, Henrique Galvan; Lädermann, Alexandre; Chagué, Sylvain; Charbonnier, Caecilia

doi:10.1002/rcs.1969

Cited by 29 publications

(14 citation statements)

References 34 publications

(29 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To our knowledge, there are no previous works in the literature emphasizing the importance of visual axes on TED in an inside-out, marker-based navigation system and demonstrating consistent variations in registration accuracy. What can be found in the literature are data on outside-in tracking systems and/or marker-less surface geometry tracking systems that corroborate with our results, although not showing accuracy in same axes [1,17,38]. Clinically, we can suggest that there is a greater value in using the HoloLens only in small time intervals.…”

Section: Discussionsupporting

confidence: 90%

“…4. In addition, this 2 mm error range was achieved without adding any commercial outside-in optical tracking systems, in comparison with other studies achieving similar results [1,17,28]. While different studies testing the accuracy of ArUco marker-based registration can be found in the literature [24,29], the units of accuracy are not usually given in mm, but rather in pixels and computing time because the camera used in the MR device has a major role in its registration accuracy.…”

Section: Discussionsupporting

confidence: 71%

“…For these reasons, it seems that a viable surgical navigation system could facilitate more extensive surgery without compromising safety [15], provided acceptable accuracy. From a technical perspective, we mainly find heterogenous results in the literature describing the accuracy of the HoloLens, using mathematical methods [16,17]. This is why we introduce in this study a perceptual variable: the surgeon's sightline.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Value of the surgeon’s sightline on hologram registration and targeting in mixed reality

et al. 2020

View full text Add to dashboard Cite

Purpose Mixed reality (MR) is being evaluated as a visual tool for surgical navigation. Current literature presents unclear results on intraoperative accuracy using the Microsoft HoloLens 1®. This study aims to assess the impact of the surgeon’s sightline in an inside-out marker-based MR navigation system for open surgery. Methods Surgeons at Akershus University Hospital tested this system. A custom-made phantom was used, containing 18 wire target crosses within its inner walls. A CT scan was obtained in order to segment all wire targets into a single 3D-model (hologram). An in-house software application (CTrue), developed for the Microsoft HoloLens 1, uploaded 3D-models and automatically registered the 3D-model with the phantom. Based on the surgeon’s sightline while registering and targeting (free sightline /F/or a strictly perpendicular sightline /P/), 4 scenarios were developed (FF-PF-FP-PP). Target error distance (TED) was obtained in three different working axes-(XYZ). Results Six surgeons (5 males, age 29–62) were enrolled. A total of 864 measurements were collected in 4 scenarios, twice. Scenario PP showed the smallest TED in XYZ-axes mean = 2.98 mm ± SD 1.33; 2.28 mm ± SD 1.45; 2.78 mm ± SD 1.91, respectively. Scenario FF showed the largest TED in XYZ-axes with mean = 10.03 mm ± SD 3.19; 6.36 mm ± SD 3.36; 16.11 mm ± SD 8.91, respectively. Multiple comparison tests, grouped in scenarios and axes, showed that the majority of scenario comparisons had significantly different TED values (p < 0.05). Y-axis always presented the smallest TED regardless of scenario tested. Conclusion A strictly perpendicular working sightline in relation to the 3D-model achieves the best accuracy results. Shortcomings in this technology, as an intraoperative visual cue, can be overcome by sightline correction. Incidentally, this is the preferred working angle for open surgery.

show abstract

Section: Discussionsupporting

confidence: 90%

Section: Discussionsupporting

confidence: 71%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Value of the surgeon’s sightline on hologram registration and targeting in mixed reality

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Using this approach, the registration between the 3D-AR headset and the external tracking system can be viewed as a hand-eye calibration. Originally developed for robotics applications [20] , [21] , this approach is described in detail and validated in a study of 3D-AR applied to orthopedic surgery [52] . This was also the approach used in the study of 3D-AR for guiding a flexible needle to a 2 cm phantom target [45] .…”

Section: External Tracking Hardwarementioning

confidence: 99%

Registration Techniques for Clinical Applications of Three-Dimensional Augmented Reality Devices

Andrews

Henry²,

Soriano³

et al. 2021

IEEE J. Transl. Eng. Health Med.

View full text Add to dashboard Cite

Many clinical procedures would benefit from direct and intuitive real-time visualization of anatomy, surgical plans, or other information crucial to the procedure. Three-dimensional augmented reality (3D-AR) is an emerging technology that has the potential to assist physicians with spatial reasoning during clinical interventions. The most intriguing applications of 3D-AR involve visualizations of anatomy or surgical plans that appear directly on the patient. However, commercially available 3D-AR devices have spatial localization errors that are too large for many clinical procedures. For this reason, a variety of approaches for improving 3D-AR registration accuracy have been explored. The focus of this review is on the methods, accuracy, and clinical applications of registering 3D-AR devices with the clinical environment. The works cited represent a variety of approaches for registering holograms to patients, including manual registration, computer vision-based registration, and registrations that incorporate external tracking systems. Evaluations of user accuracy when performing clinically relevant tasks suggest that accuracies of approximately 2 mm are feasible. 3D-AR device limitations due to the vergenceaccommodation conflict or other factors attributable to the headset hardware add on the order of 1.5 mm of error compared to conventional guidance. Continued improvements to 3D-AR hardware will decrease these sources of error.

show abstract

“…However, fully automated image-to-patient registration is most beneficial, as it eliminates the need for complicated calibration and adjustment procedures. Therefore, it is an active field of research 12 .…”

Section: Background and Summarymentioning

confidence: 99%

Facial model collection for medical augmented reality in oncologic cranio-maxillofacial surgery

et al. 2019

View full text Add to dashboard Cite

Medical augmented reality (AR) is an increasingly important topic in many medical fields. AR enables x-ray vision to see through real world objects. In medicine, this offers pre-, intra- or post-interventional visualization of “hidden” structures. In contrast to a classical monitor view, AR applications provide visualization not only on but also in relation to the patient. However, research and development of medical AR applications is challenging, because of unique patient-specific anatomies and pathologies. Working with several patients during the development for weeks or even months is not feasible. One alternative are commercial patient phantoms, which are very expensive. Hence, this data set provides a unique collection of head and neck cancer patient PET-CT scans with corresponding 3D models, provided as stereolitography (STL) files. The 3D models are optimized for effective 3D printing at low cost. This data can be used in the development and evaluation of AR applications for head and neck surgery.

show abstract

A hand‐eye calibration method for augmented reality applied to computer‐assisted orthopedic surgery

Cited by 29 publications

References 34 publications

Value of the surgeon’s sightline on hologram registration and targeting in mixed reality

Value of the surgeon’s sightline on hologram registration and targeting in mixed reality

Registration Techniques for Clinical Applications of Three-Dimensional Augmented Reality Devices

Facial model collection for medical augmented reality in oncologic cranio-maxillofacial surgery

Contact Info

Product

Resources

About