Image-guided simulation of heterogeneous tissue deformation for augmented reality during hepatic surgery

Abstract

Figure 1: A sequence of images showing the superimposition of the 3D real-time biomechanical model onto the human liver, undergoing deformation due to instrument interaction during minimally invasive hepatic surgery. The liver is represented in wireframe, the tumor in purple, the hepatic vein is shown in blue and the portal vein in green. ABSTRACTThis paper presents a method for real-time augmentation of vascular network and tumors during minimally invasive liver surgery. Internal structures computed from pre-operative CT scans can be overlaid onto the laparoscopic view for surgery guidance. Compared to state-of-the-art methods, our method uses a real-time biomechanical model to compute a volumetric displacement field from partial three-dimensional liver surface motion. This permits to properly handle the motion of internal structures even in the case of anisotropic or heterogeneous tissues, as it is the case for the liver and many anatomical structures. Real-time augmentation results are presented on in vivo and phantom data and illustrate the benefits of such an approach for minimally invasive surgery.

What makes Data Science different? A discussion involving Statistics2.0 and Computational Sciences

“…These "twins" require predictive, high-fidelity models to learn from real-time data acquired during the life of the system, accounting for "real" conditions during predictions. These Twins could enable to predict the motion of target areas during surgery with predefined accuracy [54][55][56] or fuel virtual reality engines [57] enabling the surgeons to "see through" the patient, investigate the potential response of a patient to a given treatment [58]. Digital Twins could also enable to transition from "factors of safety" and associated over-engineering to adaptive structures and systems which adapt to their environment [59][60][61][62][63][64].…”

Single view augmentation of 3D elastic objects

“…In our past works on liver augmentation [11], we mainly consider deformations of the liver that mainly occurs in a fronto-parallel plane but not very in depth. In addition, visual features were acquired in 3D thanks to a stereovision set-up.…”

“…Visualization of additional pre-operative information -such as tumors-in the field of view of the surgeon requires to track in real time possibly large elastic deformations of the liver. Existing techniques that take into account organ elasticity [11] [23] have been proposed. These methods rely on a combination of a stereo estimation of organ motion and on biomechanical models to characterize the elastic behaviour.…”

Surgical Augmented Reality with Topological Changes

“…Patient-specific biomechanical models have demonstrated their relevance for volume deformation, as they allow to account for anisotropic and elastic properties of the shape and to infer in-depth structure motion [2], [3]. In [4], a 4D scan of the heart is jointly used with a biomechanical model to couple the surface motion with external forces derived from camera data.…”