Towards natural 3D interaction for laparoscopic augmented reality registration

Abstract

Augmented reality (AR) is a widely researched route for navigation support in laparoscopic surgery. Accurate registration is a crucial component for such AR systems. We introduce two methods for interactive registration that aim to be minimally invasive to the workflow and to mimic natural manipulation of 3D objects. The methods utilise spatially tracked laparoscopic tools to manipulate the virtual 3D content. We comparatively evaluated the methods against a reference, landmark-based registration method in a user study with 12 participants. We tested the methods for registration accuracy, time, and subjective usability perception. Our methods did not outperform the reference method on these parameters but showed promising results. The results indicate that our methods present no finalised solutions but that one of them is a promising approach for which we identified concrete improvement measures to be implemented in future research.

A comprehensive study to learn the impact of augmented reality and haptic interaction in ultrasound-guided percutaneous liver biopsy training and education

“…AR facilitates this process to make this task easier by reducing the stress level of the practitioners. A study has presented two techniques for interactive registration that strive to have minimum impact on laparoscopic surgical procedure while replicating the effortless control of 3D objects (Joeres et al 2021).The technique employs laparoscopic instruments that are tracked in place to edit the virtual 3D information.…”

“…For radiological image data rendering, AR is sufficient; it is crucial to ensure that the calibration, accuracy of the tracked objects, and the registrations are accomplished precisely to avoid any misalignment of the overlapped structures. Interactive registration techniques have been introduced which strive to have minimum impact on surgical procedures (Joeres et al 2021). Zhang et al (2019) introduce a markerless automated framework for deformable registration in laparoscopic partial nephrectomy video see through AR navigation.…”