Unsupervised Deformable Registration for Multi-modal Images via Disentangled Representations

Chen, Qin; Shi, Bibo; Liao, Rui; Mansi, Tommaso; Rueckert, Daniel; Kamen, Ali

doi:10.1007/978-3-030-20351-1_19

Cited by 104 publications

(82 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several groups have achieved comparable or even better results in terms of TRE on DIRLAB 3D-CT lung DIR. However, most of the methods in this category focused [115] Lung, Brain 2D-2D No T1-T2, CT-MR Deformable on unimodality registration. There has been a lack of investigation in multi-modality image registration using unsupervised methods.…”

Section: Assessmentmentioning

confidence: 99%

Deep learning in medical image registration: a review

et al. 2020

View full text Add to dashboard Cite

This paper presents a review of deep learning (DL) based medical image registration methods. We summarized the latest developments and applications of DL-based registration methods in the medical field. These methods were classified into seven categories according to their methods, functions and popularity. A detailed review of each category was presented, highlighting important contributions and identifying specific challenges. A short assessment was presented following the detailed review of each category to summarize its achievements and future potentials. We provided a comprehensive comparison among DL-based methods for lung and brain registration using benchmark datasets. Lastly, we analyzed the statistics of all the cited works from various aspects, revealing the popularity and future trend of DL-based medical image registration. 1 arXiv:1912.12318v1 [eess.IV] 27 Dec 2019 1. Summarize the latest developments in DL-based medical image registration. 2. Highlight contributions, identify challenges and outline future trends. 3. Provide detailed statistics on recent publications from different perspectives. 2 Deep Learning 2 Convolutional Neural NetworkConvolutional neural network (CNN) is a class of deep neural networks with regularized multilayer perceptron. CNN uses convolution operation in place of general matrix multiplication in simple neural networks. The convolutional filters and operations in CNN make it suitable for visual imagery signal processing. Because of its excellent feature extraction ability, CNN is one of the most successful models for image analysis. Since the breakthrough of AlexNet [79], many variants of CNN have been proposed and have achieved the-state-of-art performances in various image processing tasks. A typical CNN usually consists of multiple convolutional layers, max pooling layers, batch normalization layers, dropout layers, a sigmoid or softmax layer. In each convolutional layer, multiple channels of feature maps were extracted by sliding trainable convolutional kernels across the input feature maps. Hierarchical features with high-level abstraction are extracted using multiple convolutional layers. These feature maps usually go through multiple fully connected layer before reaching the final decision layer. Max pooling layers are often used to reduce the image sizes and to promote spatial invariance of the network. Batch normalization is used to reduce internal covariate shift among the training samples. Weight regularization and dropout layers are used to alleviate data overfitting. The loss function is defined as the difference between the predicted and the target output. CNN is usually trained by minimizing the loss via gradient back propagation using optimization methods. Many different types of network architectures have been proposed to improve the performance of CNN [93]. U-Net proposed by Ronneberger et al. is among one of the most used network architectures [120]. U-Net was originally used to perform neuronal structures segmentation. U-Net adopts symmetrical contractiv...

show abstract

Section: Assessmentmentioning

confidence: 99%

Deep learning in medical image registration: a review

et al. 2020

View full text Add to dashboard Cite

show abstract

“…47 Qin et al employed 2D image-to-image translation to disentangle image into domain-invariant latent space prior to registration. 48 A discriminator was trained to distinguish whether an image patch pair was well-aligned or not. Mahapatra et al combined a conditional GAN and a cyclic GAN for multimodal image registration.…”

Section: Related Workmentioning

confidence: 99%

“…Qin et al . employed 2D image‐to‐image translation to disentangle image into domain‐invariant latent space prior to registration . A discriminator was trained to distinguish whether an image patch pair was well‐aligned or not.…”

Section: Related Workmentioning

confidence: 99%

LungRegNet: An unsupervised deformable image registration method for 4D‐CT lung

Lei

Wang

et al. 2020

Medical Physics

View full text Add to dashboard Cite

Purpose: To develop an accurate and fast deformable image registration (DIR) method for fourdimensional computed tomography (4D-CT) lung images. Deep learning-based methods have the potential to quickly predict the deformation vector field (DVF) in a few forward predictions. We have developed an unsupervised deep learning method for 4D-CT lung DIR with excellent performances in terms of registration accuracies, robustness, and computational speed. Methods: A fast and accurate 4D-CT lung DIR method, namely LungRegNet, was proposed using deep learning. LungRegNet consists of two subnetworks which are CoarseNet and FineNet. As the name suggests, CoarseNet predicts large lung motion on a coarse scale image while FineNet predicts local lung motion on a fine scale image. Both the CoarseNet and FineNet include a generator and a discriminator. The generator was trained to directly predict the DVF to deform the moving image. The discriminator was trained to distinguish the deformed images from the original images. Coarse-Net was first trained to deform the moving images. The deformed images were then used by the Fine-Net for FineNet training. To increase the registration accuracy of the LungRegNet, we generated vessel-enhanced images by generating pulmonary vasculature probability maps prior to the network prediction. Results: We performed fivefold cross validation on ten 4D-CT datasets from our department. To compare with other methods, we also tested our method using separate 10 DIRLAB datasets that provide 300 manual landmark pairs per case for target registration error (TRE) calculation. Our results suggested that LungRegNet has achieved better registration accuracy in terms of TRE than other deep learning-based methods available in the literature on DIRLAB datasets. Compared to conventional DIR methods, LungRegNet could generate comparable registration accuracy with TRE smaller than 2 mm. The integration of both the discriminator and pulmonary vessel enhancements into the network was crucial to obtain high registration accuracy for 4D-CT lung DIR. The mean and standard deviation of TRE were 1.00 AE 0.53 mm and 1.59 AE 1.58 mm on our datasets and DIRLAB datasets respectively. Conclusions: An unsupervised deep learning-based method has been developed to rapidly and accurately register 4D-CT lung images. LungRegNet has outperformed its deep-learning-based peers and achieved excellent registration accuracy in terms of TRE.

show abstract

“…The method of Hu YP et al is regarded as a segmentation-based registration method [49]. In addition to the approaches that are discussed above, without requiring any intensity similarity, generative adversarial networks (GANs) were used to assess the quality of image alignment [16], [22], [30]- [32]. The segmentation task and the registration task can be improved via competition with each other.…”

Section: Introductionmentioning

confidence: 99%

Brain Deformable Registration Using Global and Local Label-Driven Deep Regression Learning in the First Year of Life

Zhang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Accurate medical image registration is highly important for the quantitative analysis of infant brain dynamic development in the first year of life. However, the deformable registration of infant brain magnetic resonance (MR) images is highly challenging for the following two reasons: First, there are very large anatomical and appearance variations in these longitudinal images; Second, there is a one-tomany correspondence in appearance between global anatomical tissues and the small local tissues therein. In this paper, we use a CNN (convolution neural network)-based global-and-local-label-driven deformable registration scheme. Two to-be-registered image patches are input into the UNet-style regression network. Then, a dense displacement field (DDF) between them is obtained by optimizing the total loss function between two corresponding label patches. Global and local label patches are used only during training. During inference, two new MR images are divided into many patch pairs and fed into the trained network. By averaging the deformation of the patches at the same location, the final 3D DDF between the two whole images is obtained. The highlight is that the global (white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF)) and local tissues can be registered simultaneously without any prior ground-truth deformation. Especially for the local hippocampal tissues, the Dice ratios are substantially improved after registration via our method. Experimental results are presented on the intrasubject and intersubject registration of infant brain MR images between different time points, and the intersubject registration of brain T1-weighted MR images on the OASIS-1 dataset, according to which the proposed method realizes higher accuracy on both global and local tissues compared with state-of-the-art registration methods. INDEX TERMS Infant brain MR images, deformable registration, label-driven learning.

show abstract

Unsupervised Deformable Registration for Multi-modal Images via Disentangled Representations

Cited by 104 publications

References 20 publications

Deep learning in medical image registration: a review

Deep learning in medical image registration: a review

LungRegNet: An unsupervised deformable image registration method for 4D‐CT lung

Brain Deformable Registration Using Global and Local Label-Driven Deep Regression Learning in the First Year of Life

Contact Info

Product

Resources

About