Weakly-supervised learning of multi-modal features for regularised iterative descent in 3D image registration

Blendowski, Max; Hansen, Lasse; Heinrich, Mattias P.

doi:10.1016/j.media.2020.101822

Cited by 29 publications

(23 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, instead of image registration, they were used either for image restoration and reconstruction or for video understanding. Recently, Blendowski et al [48] proposed a supervised iterative descent algorithm (SUITS) for multi-modal image registration, which has similar ingredients to our method. SUITS uses a CNN to extract image features, which are then plugged into the Horn and Schunck (HS) model [49] to compute displacements.…”

Section: ω = 1 Lossmentioning

confidence: 96%

See 1 more Smart Citation

Learning a Model-Driven Variational Network for Deformable Image Registration

Jia

Thorley

Chen

et al. 2022

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Data-driven deep learning approaches to image registration can be less accurate than conventional iterative approaches, especially when training data is limited. To address this issue and meanwhile retain the fast inference speed of deep learning, we propose VR-Net, a novel cascaded variational network for unsupervised deformable image registration. Using a variable splitting optimization scheme, we first convert the image registration problem, established in a generic variational framework, into two sub-problems, one with a point-wise, closed-form solution and the other one being a denoising problem. We then propose two neural layers (i.e. warping layer and intensity consistency layer) to model the analytical solution and a residual U-Net (termed generalized denoising layer) to formulate the denoising problem. Finally, we cascade the three neural layers multiple times to form our VR-Net. Extensive experiments on three (two 2D and one 3D) cardiac magnetic resonance imaging datasets show that VR-Net outperforms state-of-the-art deep learning methods on registration accuracy, whilst maintaining the fast inference speed of deep learning and the data-efficiency of variational models.

show abstract

Section: ω = 1 Lossmentioning

confidence: 96%

“…Instead, we use the iterative process for optimization only to guide the design of network architecture. Moreover, unlike [48] which uses an algebraic multigrid solver (AMG) to solve the linear system of equations, all subproblems (network layers) in our method have closed-form, point-wise solutions.…”

Section: ω = 1 Lossmentioning

confidence: 99%

Learning a Model-Driven Variational Network for Deformable Image Registration

Jia

Thorley

Chen

et al. 2022

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

show abstract

“…However, for medical applications, synthetic deformations have been deployed for monomodal image registration [ 5 , 6 , 7 ]. Alternatively, label supervision that primarily maximises the alignment of known structures with expert annotations could be employed [ 2 , 8 , 9 ]. This leads to improved registration of anatomies that are well represented, but can introduce a bias and deteriorating performance for unseen labels.…”

Section: Introductionmentioning

confidence: 99%

Learning a Metric for Multimodal Medical Image Registration without Supervision Based on Cycle Constraints

Siebert

Hansen

Heinrich

2022

Sensors

Self Cite

View full text Add to dashboard Cite

Deep learning based medical image registration remains very difficult and often fails to improve over its classical counterparts where comprehensive supervision is not available, in particular for large transformations—including rigid alignment. The use of unsupervised, metric-based registration networks has become popular, but so far no universally applicable similarity metric is available for multimodal medical registration, requiring a trade-off between local contrast-invariant edge features or more global statistical metrics. In this work, we aim to improve over the use of handcrafted metric-based losses. We propose to use synthetic three-way (triangular) cycles that for each pair of images comprise two multimodal transformations to be estimated and one known synthetic monomodal transform. Additionally, we present a robust method for estimating large rigid transformations that is differentiable in end-to-end learning. By minimising the cycle discrepancy and adapting the synthetic transformation to be close to the real geometric difference of the image pairs during training, we successfully tackle intra-patient abdominal CT-MRI registration and reach performance on par with state-of-the-art metric-supervision and classic methods. Cyclic constraints enable the learning of cross-modality features that excel at accurate anatomical alignment of abdominal CT and MRI scans.

show abstract

“…This architecture performs comparibly to the aforementioned traditional techniques while improving runtime significantly [14,17]. There has however, been minimal investigation of VoxelMorph, and unsupervised registration networks in general, for multi-modal registration problems [8,18].…”

Section: Introductionmentioning

confidence: 99%

Combined Mass Spectrometry and Histopathology Imaging for Perioperative Tissue Assessment in Cancer Surgery

et al. 2021

View full text Add to dashboard Cite

Mass spectrometry is an effective imaging tool for evaluating biological tissue to detect cancer. With the assistance of deep learning, this technology can be used as a perioperative tissue assessment tool that will facilitate informed surgical decisions. To achieve such a system requires the development of a database of mass spectrometry signals and their corresponding pathology labels. Assigning correct labels, in turn, necessitates precise spatial registration of histopathology and mass spectrometry data. This is a challenging task due to the domain differences and noisy nature of images. In this study, we create a registration framework for mass spectrometry and pathology images as a contribution to the development of perioperative tissue assessment. In doing so, we explore two opportunities in deep learning for medical image registration, namely, unsupervised, multi-modal deformable image registration and evaluation of the registration. We test this system on prostate needle biopsy cores that were imaged with desorption electrospray ionization mass spectrometry (DESI) and show that we can successfully register DESI and histology images to achieve accurate alignment and, consequently, labelling for future training. This automation is expected to improve the efficiency and development of a deep learning architecture that will benefit the use of mass spectrometry imaging for cancer diagnosis.

show abstract

Weakly-supervised learning of multi-modal features for regularised iterative descent in 3D image registration

Cited by 29 publications

References 13 publications

Learning a Model-Driven Variational Network for Deformable Image Registration

Learning a Model-Driven Variational Network for Deformable Image Registration

Learning a Metric for Multimodal Medical Image Registration without Supervision Based on Cycle Constraints

Combined Mass Spectrometry and Histopathology Imaging for Perioperative Tissue Assessment in Cancer Surgery

Contact Info

Product

Resources

About