Learning a Metric for Multimodal Medical Image Registration without Supervision Based on Cycle Constraints

Abstract

Deep learning based medical image registration remains very difficult and often fails to improve over its classical counterparts where comprehensive supervision is not available, in particular for large transformations—including rigid alignment. The use of unsupervised, metric-based registration networks has become popular, but so far no universally applicable similarity metric is available for multimodal medical registration, requiring a trade-off between local contrast-invariant edge features or more global statistical metrics. In this work, we aim to improve over the use of handcrafted metric-based losses. We propose to use synthetic three-way (triangular) cycles that for each pair of images comprise two multimodal transformations to be estimated and one known synthetic monomodal transform. Additionally, we present a robust method for estimating large rigid transformations that is differentiable in end-to-end learning. By minimising the cycle discrepancy and adapting the synthetic transformation to be close to the real geometric difference of the image pairs during training, we successfully tackle intra-patient abdominal CT-MRI registration and reach performance on par with state-of-the-art metric-supervision and classic methods. Cyclic constraints enable the learning of cross-modality features that excel at accurate anatomical alignment of abdominal CT and MRI scans.

Chronological Registration of OCT and Autofluorescence Findings in CSCR: Two Distinct Patterns in Disease Course

“…Each block is composed of two convolutional layers with filter size , batch normalization and ReLU activation. To allow for different processing of the different input modalities, we do not pass B-scan and retina segmentation as a two-channel input but use two separate convolutional blocks at the first level of the network [ 33 ]. The resulting feature maps are concatenated after the first maximum pooling.…”

Cardiovascular/Stroke Risk Stratification in Diabetic Foot Infection Patients Using Deep Learning-Based Artificial Intelligence: An Investigative Study

“…The main procedures that are used at this time for the examination of diabetesrelated foot problems include traditional radiography, computerized tomography, nuclear medicine scintigraphy, magnetic resonance imaging, ultrasound, and positron emission tomography [184,185]. On the other hand, each one of these modalities cannot provide enough information by itself; therefore, a multimodal approach is required to arrive at an accurate diagnosis [186].…”

“…It is possible to observe that the database contains patient characteristics that are particular to a given region. Because of this, the model can produce false positive or negative results for other places, which would make the algorithm biased [185,202].…”

Robust Unsupervised Image to Template Registration Without Image Similarity Loss

“…As far as we know, only in [12] is addressed the problem of unsupervised registration without image dissimilarity losses: the fixed and moving image are encoded with a separate encoder, a correlation matrix between local features is computed providing a rough displacement likelihood map for each cell and a robust fit finally outputs the estimated affine transformation matrix. Only a cycle consistency loss is used for the training.…”