Improving Intraoperative Liver Registration in Image-Guided Surgery with Learning-Based Reconstruction

Abstract

Nonrigid image-to-physical registration is a crucial component in image-guided liver surgery. To overcome the problems caused by noisy, partial, and sparse intraoperative sampling, we propose a novel occupancy-learning-based mesh to point cloud registration and apply it to align the preoperative liver image to intraoperative samples. We train a point cloud deep network to reconstruct occupancy function from sparse points and use this reconstructed liver to guide the nonrigid registration. Experiments show this method reduces Target Registration Error (TRE) of rigid and nonrigid baselines by 21.5% and 11.8%. For training this occupancy network, a novel crossover method is proposed to synthesize deformed liver meshes and points. We demonstrate that the system is robust to the imperfectness of generated training data. This method might be useful in other areas that require soft-tissue registration, where only very sparse data is available during acquisition.

Organ registration from partial surface data in augmented surgery from an optimal control perspective

“…We obtain the second-best result among all submissions, close to the best-performing method [26], also based on an inverse biomechanical simulation. In the method from VEIDA [61], a neural network predicts a shape from the point cloud in the form level-set function, and then the liver deformation is estimated by fitting this shape. In the V2SNet method [62], a neural network is trained from synthetic simulations to predict a displacement field in one step from the point cloud.…”

The Image-to-Physical Liver Registration Sparse Data Challenge: comparison of state-of-the-art using a common dataset

“…8 indicates that deformable registration algorithms may express different levels of sensitivity to variations in the initial rigid alignment. It should be noted that three of the five deformable registration methods (Heiselman, 14 , 15 Jia, 13 and Ringel 17 ) take additional precautions to concurrently re-optimize rigid pose parameters during deformable registration. Considering best practices, it may be worthwhile to operate under an assumption that surface-based rigid alignments are fundamentally unreliable in the presence of soft tissue deformation.…”

Comparison study of sparse data-driven soft tissue registration: preliminary results from the image-to-physical liver registration sparse data challenge

“…The fifth registration strategy is a data-driven nonrigid approach [7] based on a learned occupancy map to predict the likelihood that a particular volumetric shape takes a certain configuration through a point convolutional neural network (PCNN). The authors propose a deep neural network to model a differentiable occupancy map 𝑔(𝑥 𝑖 , 𝑃) ∈ [0, 1] ∶ ℝ → ℝ 3 for the probability a shape occupies position 𝑥 𝑖 given a point cloud 𝑃 that describes a sparse representation of the shape surface.…”

“…The penalty term 𝛼𝐸(𝜙) represents regularization by the strain energy 𝐸(𝜙) in the manner of Rucker et al [12]. More information and details about this method are provided in [7]. Compared to conventional simulation approaches that assume a model-data correspondence function and minimize error between the set of observed data points and their corresponding locations on the deforming organ model, this technique explores an interesting angle towards learned objective functions that encode correspondence through probabilistic model occupancy, which may offer new approaches to offset deleterious effects of uncertainty in registrations to sparse point cloud data.…”

“…(5) A deep learning method based on probabilistic shape occupancy maps (Jia 2021) [7]. (6) A biomechanical finite element method based on linearized iterative boundary condition reconstruction (Heiselman 2020) [8,9] (7) A biomechanical finite element method based on adjoint boundary condition reconstruction (Mestdagh 2022) [10].…”