Multi-Modal Medical Image Registration with Full or Partial Data: A Manifold Learning Approach

Bashiri, Fereshteh S.; Baghaie, Ahmadreza; Rostami, Reihaneh; Yu, Zeyun; D’Souza, Roshan

doi:10.3390/jimaging5010005

Cited by 36 publications

(23 citation statements)

References 75 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They demonstrated that the network could rapidly and accurately quantify registration performance. Multi-grid Inference [163] Brain 3D-3D MR-US Rigid LSTM [8] Brain 2D-2D CT, T1, T2, PD Rigid Manifold Learning [165] Brain, Abdomen 2D-3D CT-PET, CT-MRI Deformable CAE, DSCNN [178] Spine 3D-2D 3DCT-Xray Rigid FasterRCNN [54] Brain 2D-2D T1-T2, T1-PD Deformable FCN [183] Spine 3D-2D 3DCT-Xray Rigid Domain adaptation…”

Section: Registration Validation Using Deep Learningmentioning

confidence: 99%

Deep learning in medical image registration: a review

et al. 2020

View full text Add to dashboard Cite

This paper presents a review of deep learning (DL) based medical image registration methods. We summarized the latest developments and applications of DL-based registration methods in the medical field. These methods were classified into seven categories according to their methods, functions and popularity. A detailed review of each category was presented, highlighting important contributions and identifying specific challenges. A short assessment was presented following the detailed review of each category to summarize its achievements and future potentials. We provided a comprehensive comparison among DL-based methods for lung and brain registration using benchmark datasets. Lastly, we analyzed the statistics of all the cited works from various aspects, revealing the popularity and future trend of DL-based medical image registration. 1 arXiv:1912.12318v1 [eess.IV] 27 Dec 2019 1. Summarize the latest developments in DL-based medical image registration. 2. Highlight contributions, identify challenges and outline future trends. 3. Provide detailed statistics on recent publications from different perspectives. 2 Deep Learning 2 Convolutional Neural NetworkConvolutional neural network (CNN) is a class of deep neural networks with regularized multilayer perceptron. CNN uses convolution operation in place of general matrix multiplication in simple neural networks. The convolutional filters and operations in CNN make it suitable for visual imagery signal processing. Because of its excellent feature extraction ability, CNN is one of the most successful models for image analysis. Since the breakthrough of AlexNet [79], many variants of CNN have been proposed and have achieved the-state-of-art performances in various image processing tasks. A typical CNN usually consists of multiple convolutional layers, max pooling layers, batch normalization layers, dropout layers, a sigmoid or softmax layer. In each convolutional layer, multiple channels of feature maps were extracted by sliding trainable convolutional kernels across the input feature maps. Hierarchical features with high-level abstraction are extracted using multiple convolutional layers. These feature maps usually go through multiple fully connected layer before reaching the final decision layer. Max pooling layers are often used to reduce the image sizes and to promote spatial invariance of the network. Batch normalization is used to reduce internal covariate shift among the training samples. Weight regularization and dropout layers are used to alleviate data overfitting. The loss function is defined as the difference between the predicted and the target output. CNN is usually trained by minimizing the loss via gradient back propagation using optimization methods. Many different types of network architectures have been proposed to improve the performance of CNN [93]. U-Net proposed by Ronneberger et al. is among one of the most used network architectures [120]. U-Net was originally used to perform neuronal structures segmentation. U-Net adopts symmetrical contractiv...

show abstract

Section: Registration Validation Using Deep Learningmentioning

confidence: 99%

Deep learning in medical image registration: a review

et al. 2020

View full text Add to dashboard Cite

show abstract

“…This idea has been applied with high quality data 24,36 , but the multi modality registration problem is more challenging when extra signals are present. Related work 37,38 has approached this challenge, but have been restricted to simple deformations. The method we develop jointly addresses each of these challenges: estimating arbitrary 3D and 2D deformations, predicting locations of extra signal by treating them as missing data in an Expectation Maximization setting as in [38][39][40] , and modeling different contrasts by synthesizing them in a generative framework.…”

Section: Discussionmentioning

confidence: 99%

Solving thewhereproblem and quantifying geometric variation in neuroanatomy using generative diffeomorphic mapping

Tward

Huo

et al. 2020

Preprint

View full text Add to dashboard Cite

Mapping information from different brains gathered using different modalities into a common coordinate space corresponding to a reference brain is an aspirational goal in modern neuroscience, analogous in importance to mapping genomic data to a reference genome. While brain-atlas mapping workflows exist for single-modality data (3D MRI or STPT image volumes), generally speaking data sets need to be combined across modalities with different contrast mechanisms and scale, in the presence of missing data as well as signals not present in the reference. This has so far been an unsolved problem. We have solved this problem in its full generality by developing and implementing a rigorous, non-parametric generative framework, that learns unknown mappings between contrast mechanisms from data and infers missing data. Our methodology permits rigorous quantification of the local sale changes between different individual brains, which has so far been neglected. We are also able to quantitatively characterize the individual variation in shape. Our work establishes a quantitative, scalable and streamlined workflow for unifying a broad spectrum of multi-modal whole-brain light microscopic data volumes into a coordinate-based atlas framework, a step that is a prerequisite for large scale integration of whole brain data sets in modern neuroscience. Figure 1. Challenge: Integrating neuroimaging data to reference coordinates. (a-d) Diversity of neuroanatomical data types for mapping to a common atlas coordinate framework (data pictured from a) Slide-seq 1 , b,d) brainachitecrture.org, c) neuromorpho.org). e) A common reference brain together with a coordinate system to which the diverse data types need to be mapped. Technical challenges include accommodating f) 3D and 2D imaging, g) shape variability due to specimen preparation or individual variation, h) multiple contrasts associated with different imaging technologies, i) and signals that are not present in reference images such as labels, missing tissue, or artifacts. brain, across a large population set. We note that the magnitude of individual variation is often greater than differences between different sample preparation techniques. Our work establishes a quantitative, scalable and streamlined workflow for unifying a broad spectrum of multi-modal whole-brain light microscopic data volumes into a coordinate-based atlas framework, a step that is a prerequisite for large scale integration of whole brain data sets in modern neuroscience. 2/133/13 4/13

show abstract

“…As slice spacing decreases, it is hard to distinguish adjacent slices, which results in the deviation of many multimodal similarity algorithms. To verify our method's validation, we performed modality-group similarity experiment with 4 different methods: (1) the proposed method in [8] using entropy (M1 function) images, (2) the method using Laplacian method in manifold learning [14], (3) multimodal registration with mutual information (MI) [4], and (4) traditional method with mean absolute differences (MAD). e above result of the experiment is illustrated on Tables 2-4.…”

Section: Modality-group Similarity Experiments On Rigidmentioning

confidence: 99%

“…However, it focuses on space location relationship and loses sight of potential richness. Most recently, in 2019, Bashiri et al [14] expressed the descriptor set in high dimensional space, studying potential structures of an image through Laplacian eigenmap. Nonlinear dimensionality reduction from manifold space will result in the loss of original potential information.…”

Section: Introductionmentioning

confidence: 99%

Logarithmic Fuzzy Entropy Function for Similarity Measurement in Multimodal Medical Images Registration

Gao

Zhang

et al. 2020

Computational and Mathematical Methods in Medicine

View full text Add to dashboard Cite

Multimodal medical images are useful for observing tissue structure clearly in clinical practice. To integrate multimodal information, multimodal registration is significant. The entropy-based registration applies a structure descriptor set to replace the original multimodal image and compute similarity to express the correlation of images. The accuracy and converging rate of the registration depend on this set. We propose a new method, logarithmic fuzzy entropy function, to compute the descriptor set. It is obvious that the proposed method can increase the upper bound value from log(r) to log(r) + ∆(r) so that a more representative structural descriptor set is formed. The experiment results show that our method has faster converging rate and wider quantified range in multimodal medical images registration.

show abstract

Multi-Modal Medical Image Registration with Full or Partial Data: A Manifold Learning Approach

Cited by 36 publications

References 75 publications

Deep learning in medical image registration: a review

Deep learning in medical image registration: a review

Solving thewhereproblem and quantifying geometric variation in neuroanatomy using generative diffeomorphic mapping

Logarithmic Fuzzy Entropy Function for Similarity Measurement in Multimodal Medical Images Registration

Contact Info

Product

Resources

About