Comparison of manual and semi-automatic registration in augmented reality image-guided liver surgery: a clinical feasibility study

Abstract

Background The laparoscopic approach to liver resection may reduce morbidity and hospital stay. However, uptake has been slow due to concerns about patient safety and oncological radicality. Image guidance systems may improve patient safety by enabling 3D visualisation of critical intra- and extrahepatic structures. Current systems suffer from non-intuitive visualisation and a complicated setup process. A novel image guidance system (SmartLiver), offering augmented reality visualisation and semi-automatic registration has been developed to address these issues. A clinical feasibility study evaluated the performance and usability of SmartLiver with either manual or semi-automatic registration. Methods Intraoperative image guidance data were recorded and analysed in patients undergoing laparoscopic liver resection or cancer staging. Stereoscopic surface reconstruction and iterative closest point matching facilitated semi-automatic registration. The primary endpoint was defined as successful registration as determined by the operating surgeon. Secondary endpoints were system usability as assessed by a surgeon questionnaire and comparison of manual vs. semi-automatic registration accuracy. Since SmartLiver is still in development no attempt was made to evaluate its impact on perioperative outcomes. Results The primary endpoint was achieved in 16 out of 18 patients. Initially semi-automatic registration failed because the IGS could not distinguish the liver surface from surrounding structures. Implementation of a deep learning algorithm enabled the IGS to overcome this issue and facilitate semi-automatic registration. Mean registration accuracy was 10.9 ± 4.2 mm (manual) vs. 13.9 ± 4.4 mm (semi-automatic) (Mean difference − 3 mm; p = 0.158). Surgeon feedback was positive about IGS handling and improved intraoperative orientation but also highlighted the need for a simpler setup process and better integration with laparoscopic ultrasound. Conclusion The technical feasibility of using SmartLiver intraoperatively has been demonstrated. With further improvements semi-automatic registration may enhance user friendliness and workflow of SmartLiver. Manual and semi-automatic registration accuracy were comparable but evaluation on a larger patient cohort is required to confirm these findings.

What is Artificial Intelligence Surgery?

“…These AR systems are essentially guidance systems that enable the surgeon to see the tumor and its relationship to major intra-parenchymal vascular structures in real time [27] . Although initially 2-D, images can now be created for open and laparoscopic liver resection using stereoscopic surface reconstruction and semi-automatic registration in combination with deep learning to create 3-D AR intra-operative images (SmartLiver) [28] . Other teams have confirmed this concept, however, AR still remains in the domain of feasibility as confirmation of the AR image still need to be done by the surgeon via standard intra-operative ultrasound and via visual confirmation using pre-operative cross-sectional imaging [29] .…”

CBCT‐based navigation system for open liver surgery: Accurate guidance toward mobile and deformable targets with a semi‐rigid organ approximation and electromagnetic tracking of the liver

“…27 Once the surgeon starts manipulating the organ, deformable biomechanical models are required to approximate the motion of both the underlying vasculature and the tumors. Although several groups are actively developing biomechanical liver models and have already illustrated very promising results, 23,28 the absence of intraoperative imaging in this group of navigation systems implies that the surgeon should fully rely on the accuracy of the guidance (e.g., projection of the underlying vasculature), resulting in slower widespread clinical acceptance of the approach.…”

“…In the first group (i), surface-based registration methods (e.g., based on an optical scan or landmark registration) are utilized to guarantee alignment between the surgical scene and the preoperative plan. [18][19][20][21][22] This approach is largely applied in laparoscopic and robotic surgery, and shows the biggest potential for future integration into the augmented reality (AR) setup. [23][24][25] However, because it does not allow for visualizing the liver's underlying anatomy, accurate registration is only feasible prior to the start of the dissection (e.g., when the complete surface is intact) and its accuracy assessment is primarily based on evaluating liver surface alignment.…”

“…In the first group (i), surface‐based registration methods (e.g., based on an optical scan or landmark registration) are utilized to guarantee alignment between the surgical scene and the preoperative plan 18–22 . This approach is largely applied in laparoscopic and robotic surgery, and shows the biggest potential for future integration into the augmented reality (AR) setup 23–25 .…”

Automatic, global registration in laparoscopic liver surgery

“…Multiple automatic solutions have been proposed for local alignment, assuming a good initialisation [ 2 , 18 , 20 ]. For liver surgery, global alignment is usually achieved manually [ 21 , 27 , 29 ] or in a semi-automatic way requiring annotations from the clinician during the intervention [ 16 , 26 ].…”