Stochastic Planner-Actor-Critic for Unsupervised Deformable Image Registration

Abstract

Large deformations of organs, caused by diverse shapes and nonlinear shape changes, pose a significant challenge for medical image registration. Traditional registration methods need to iteratively optimize an objective function via a specific deformation model along with meticulous parameter tuning, but which have limited capabilities in registering images with large deformations. While deep learning-based methods can learn the complex mapping from input images to their respective deformation field, it is regression-based and is prone to be stuck at local minima, particularly when large deformations are involved. To this end, we present Stochastic Planner-Actor-Critic (spac), a novel reinforcement learning-based framework that performs step-wise registration. The key notion is warping a moving image successively by each time step to finally align to a fixed image. Considering that it is challenging to handle high dimensional continuous action and state spaces in the conventional reinforcement learning (RL) framework, we introduce a new concept `Plan' to the standard Actor-Critic model, which is of low dimension and can facilitate the actor to generate a tractable high dimensional action. The entire framework is based on unsupervised training and operates in an end-to-end manner. We evaluate our method on several 2D and 3D medical image datasets, some of which contain large deformations. Our empirical results highlight that our work achieves consistent, significant gains and outperforms state-of-the-art methods.

Predicting Potential Risk: Cerebral Stroke via Regret Minimization

“…Recent work [8,9] suggests that reinforcement learning has a wide range of applications in medical information processing [10]. By selecting features from medical test reports, extracting feature combinations and learning strategies are two important tasks in this process for diferent prediction task scenarios.…”

“…Integrated reinforcement learning with MR image manipulation can reconstruct damaged images [21]. Reference [8] proposed and optimized the Stochastic Planner-Actor-Critic (SPAC) method for medical image alignment. Nonindependent and homogeneously distributed (non-iid) data in medical images remain a prominent challenge in real practice.…”

Distributional stochastic Planner‐Actor‐Critic for deformable image registration

“…It was only with the contributions of Krebs and others that reinforcement learning was extended to deformable registration tasks 16 . For large deformation registration tasks, Ziwei Luo, Jing Hu, and others first introduced the reinforcement learning‐based large deformation Stochastic Planner‐Actor‐Critic (SPAC) registration model in 2022 17 . The complexity of large deformation registration tasks is evident not only in the significant increase in computational requirements but also in the challenging aspect of finding correspondences between feature points with limited convolutional kernel receptive fields.…”

“…16 For large deformation registration tasks, Ziwei Luo, Jing Hu, and others first introduced the reinforcement learning-based large deformation Stochastic Planner-Actor-Critic (SPAC) registration model in 2022. 17 The complexity of large deformation registration tasks is evident not only in the significant increase in computational requirements but also in the challenging aspect of finding correspondences between feature points with limited convolutional kernel receptive fields. And SPAC adopts a recursive cascading approach, learning a series of states to gradually transform the moving image into the warped image for large deformation registration.…”

Attention Guided Policy Optimization for 3D Medical Image Registration

“…Many of these approaches are regression-based and generally require multi-layered feed-forward neural networks which take unaligned image pairs as input and generate registration parameters. Several other approaches [11]- [13] adopt radically different patterns by considering the registration task as a temporal decision-making issue. These methods explicitly imitate the way that human experts perform image registration via temporally action selection.…”