Registration Techniques for Clinical Applications of Three-Dimensional Augmented Reality Devices

Abstract

Many clinical procedures would benefit from direct and intuitive real-time visualization of anatomy, surgical plans, or other information crucial to the procedure. Three-dimensional augmented reality (3D-AR) is an emerging technology that has the potential to assist physicians with spatial reasoning during clinical interventions. The most intriguing applications of 3D-AR involve visualizations of anatomy or surgical plans that appear directly on the patient. However, commercially available 3D-AR devices have spatial localization errors that are too large for many clinical procedures. For this reason, a variety of approaches for improving 3D-AR registration accuracy have been explored. The focus of this review is on the methods, accuracy, and clinical applications of registering 3D-AR devices with the clinical environment. The works cited represent a variety of approaches for registering holograms to patients, including manual registration, computer vision-based registration, and registrations that incorporate external tracking systems. Evaluations of user accuracy when performing clinically relevant tasks suggest that accuracies of approximately 2 mm are feasible. 3D-AR device limitations due to the vergenceaccommodation conflict or other factors attributable to the headset hardware add on the order of 1.5 mm of error compared to conventional guidance. Continued improvements to 3D-AR hardware will decrease these sources of error.

Augmented Reality in Vascular and Endovascular Surgery: Scoping Review

“…This can be performed manually or using trackers. Marker-less tracking is now possible owing to technological advances that correlate patterns in real and virtual data in real time [ 46 , 47 ]. Integrated or external optical sensors (infrared or color) can be used to track markers or recognize patterns from a patient’s anatomy.…”

Evaluation of optical tracking and augmented reality for needle navigation in sacral nerve stimulation

“…Additionally, markerbased registration is still a nuisance, as the marker always needs to be visible, frustrating users when the detection fails. As an alternative, surface recognition could be a solution to this problem, but accuracy results are still far from those obtained with markerbased registration [38] .…”

Current trends in three-dimensional visualization and real-time navigation as well as robot-assisted technologies in hepatobiliary surgery

“…Upgrading the system to avoid eddy currents and performing a system calibration function can help to some extent with inherent system errors[ 79 , 88 ]. Static pre-calibration processes are cumbersome and ineffective for most dynamic clinical procedures, and often require too many EM sensors to compensate for field distortion in dynamic environments, making them inefficient.…”