U-ReSNet: Ultimate Coupling of Registration and Segmentation with Deep Nets

Abstract

In this study, we propose a 3D deep neural network called U-ReSNet, a joint framework that can accurately register and segment medical volumes. The proposed network learns to automatically generate linear and elastic deformation models, trained by minimizing the mean square error and the local cross correlation similarity metrics. In parallel, a coupled architecture is integrated, seeking to provide segmentation maps for anatomies or tissue patterns using an additional decoder part trained with the dice coefficient metric. U-ReSNet is trained in an end to end fashion, while due to this joint optimization the generated network features are more informative leading to promising results compared to other deep learning-based methods existing in the literature. We evaluated the proposed architecture using the publicly available OASIS 3 dataset, measuring the dice coefficient metric for both registration and segmentation tasks. Our promising results indicate the potentials of our method which is composed from a convolutional architecture that is extremely simple and light in terms of parameters. Our code is publicly available https://github.com/TheoEst/coupling_registration_ segmentation.

Learning deformable registration of medical images with anatomical constraints

“…In particular, deep convolutional neural networks (CNN) have proved to outperform all existent strategies in other fundamental tasks of computer vision, like image segmentation [27] and classification [23]. During the last years, we have witnessed the advent of deep learning-based image registration methods [25,49,35,39,46,45,1,6,11], which achieve state-of-the-art performance, and drastically reduce the required computational time. These works have made a fundamental contribution by setting novel architectures for CNN-based deformable image registration (following supervised, unsupervised and semi-supervised training approaches).…”

Learning a Deformable Registration Pyramid

“…Our resulting loss function can be written as We use the (soft) Dice coefficient (DCS) [ 6 ] for structural similarity and the normalized cross-correlation (NCC) [ 7 ] for image similarity. To ensure smooth displacements we regularize the affine displacement field with the L2-loss between the estimated value and an identity displacement field ( ) and the deformable field with the spatial gradient of the displacement field [ 8 ], where and represent downsampled versions of the moving and fixed images at each level and and indicate the estimated affine and deformable registrations (for each level). The hyperparameters and determine the importance of the corresponding terms.…”

“…We use the (soft) Dice coefficient (DCS) [6] for structural similarity and the normalized cross-correlation (NCC) [7] for image similarity. To ensure smooth displacements we regularize the affine displacement field with the L2-loss between the estimated value and an identity displacement field (φ (l) 0 ) and the deformable field with the spatial gradient of the displacement field [8],…”

Unsupervised Multistep Deformable Registration of Remote Sensing Imagery Based on Deep Learning

“…Least square matching is then employed to refine these initial points and finally obtain the transformation coefficients. In recent years, the advances of deep learning have led the computer vision community to registration solutions that are more related to neural networks, especially in the medical field [12][13][14][15]. Despite this progress, little effort has been made to adjust these frameworks for image registration in remote sensing, with semiautomated algorithms still being widely employed [1,16,17].…”