SymReg-GAN: Symmetric Image Registration with Generative Adversarial Networks

Abstract

Symmetric image registration estimates bi-directional spatial transformations between images while enforcing an inverse-consistency. Its capability of eliminating bias introduced inevitably by generic single-directional image registration allows more precise analysis in different interdisciplinary applications of image registration, e.g. computational anatomy and shape analysis. However, most existing symmetric registration techniques especially for multimodal images are limited by low speed from the commonly-used iterative optimization, hardship in exploring inter-modality relations or high labor cost for labeling data. We propose SymReg-GAN to shatter these limits, which is a novel generative adversarial networks (GAN) based approach to symmetric image registration. We formulate symmetric registration of unimodal/multimodal images as a conditional GAN and train it with a semi-supervised strategy. The registration symmetry is realized by introducing a loss for encouraging that the cycle composed of the geometric transformation from one image to another and its reverse should bring an image back. The semi-supervised learning enables both the precious labeled data and large amounts of unlabeled data to be fully exploited. Experimental results from 6 public brain magnetic resonance imaging (MRI) datasets and 1 our own computed tomography (CT) & MRI dataset demonstrate the superiority of SymReg-GAN to several existing state-of-the-art methods.

Review of Generative Adversarial Networks in mono- and cross-modal biomedical image registration

“…SymReg-GAN (Zheng et al, 2021 ) proposes a GAN-based symmetric registration to resolve the inverse-consistent translation between cross-modal images. A generator performs the modality translation, consisting of an affine-translation regressor and a non-linear-deformation regressor.…”

“…where A (ϕ 1 ) denotes a warped image produced by the module R1, A (ϕ 1 , ϕ 2 ) denotes a warped image produced by the module R2, and L reg (ϕ 1 ) is the smooth regularization L reg . SymReg-GAN (Zheng et al, 2021) proposes a GANbased symmetric registration to resolve the inverse-consistent translation between cross-modal images. A generator performs the modality translation, consisting of an affine-translation regressor and a non-linear-deformation regressor.…”

The Successive Next Network as Augmented Regularization for Deformable Brain MR Image Registration

“…Simultaneously, GANs have been applied to single- and multi-mode image registration. Zheng et al [ 23 ] used a GAN network to realize symmetric image registration and then transformed the symmetric registration formula of single- and multi-mode images into a conditional GAN. To align a pair of single-mode images, the registration method constitutes a cyclical process of transformation from one image to another and its inverse transformation.…”

A Transformer-Based Coarse-to-Fine Wide-Swath SAR Image Registration Method under Weak Texture Conditions

“…With strong abilities of feature extraction and characterization, deep learning is in wide usage across remote sensing scenarios, including classification [15], detection [16], image registration [17], and change detection [18]. More and more methods [19,20] use learning-based methods in the registration of the CV field. He et al [19] proposed a Siamese CNN (Convolutional Neural Networks) to evaluate the similarity of patch pairs.…”

“…He et al [19] proposed a Siamese CNN (Convolutional Neural Networks) to evaluate the similarity of patch pairs. Zheng et al [20] proposed SymReg-GAN, which achieves good results in medical image registration by using a generator that predicts the geometric transformation between images, and a discriminator that distinguishes the transformed images from the real images. Specific to the SAR image (remote sensing image) registration field, Li et al [21] proposed a RotNET to predict the rotation relationship between two images.…”