Deep learning in medical image registration

Abstract

Image registration is a fundamental task in multiple medical image analysis applications. With the advent of deep learning, there have been significant advances in algorithmic performance for various computer vision tasks in recent years, including medical image registration. The last couple of years have seen a dramatic increase in the development of deep learning-based medical image registration algorithms. Consequently, a comprehensive review of the current state-of-the-art algorithms in the field is timely, and necessary. This review is aimed at understanding the clinical applications and challenges that drove this innovation, analysing the functionality and limitations of existing approaches, and at providing insights to open challenges and as yet unmet clinical needs that could shape future research directions. To this end, the main contributions of this paper are: (a) discussion of all deep learning-based medical image registration papers published since 2013 with significant methodological and/or functional contributions to the field; (b) analysis of the development and evolution of deep learning-based image registration methods, summarising the current trends and challenges in the domain; and (c) overview of unmet clinical needs and potential directions for future research in deep learning-based medical image registration.

The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective

“…Inspiration can be drawn from the applications of 3D/3D and 2D/2D deformable image registration ( Chen et al, 2020 ; Fu et al, 2020 ; Haskins et al, 2020 ), where machine learning techniques have dominated most existing research into registration uncertainty. Some of these approaches report interval estimates, either by reformulating the registration objective function as a probability distribution and drawing samples ( Risholm et al, 2013 ; Le Folgoc et al, 2016 ; Schultz et al, 2018 ), approximating the sampling process using test-time NN drop-out ( Yang et al, 2017 ) or sampling using the test-time deformation covariance matrices embedded within a variational autoencoder ( Dalca et al, 2019 ).…”

Deep Learning Based Joint PET Image Reconstruction and Motion Estimation

“…Recently, deep learning (DL) techniques have provided new approaches to image registration [14]- [19]. There are three widely used components of DL-based image registration [20], an encoder-decoder based architecture incorporating several hierarchical convolution layers for multi-scale feature extraction [16], [19], [21], a spatial transformer network (STN) [22] for spatial transformation, and a generative adversarial network (GAN) [23]- [25] where a generator predicts the deformation field and warps the moving image and the warped moving image is evaluated by a discriminator. One advantage of DL methods is that a pretrained network can incorporate prior knowledge of motion fields from training data into the motion estimation process.…”

Combined Mass Spectrometry and Histopathology Imaging for Perioperative Tissue Assessment in Cancer Surgery

“…Recent literature have shown that DL can improve computation cost by using a learned optimization strategy in place of iteration [8][9][10][11]. The DL approaches that are used for deformable registration are either supervised or unsupervised, indicating that the models are either trained with or without ground truth registrations [8].…”