Automatic large quantity landmark pairs detection in 4DCT lung images

Fu, Yabo; Wu, Xue; Thomas, A.; Li, Harold H.; Yang, Deshan

doi:10.1002/mp.13726

Cited by 16 publications

(24 citation statements)

References 58 publications

(95 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ROI Dimension Modality End point [112] HN 3D CT TRE prediction [34] Lung 3D CT Registration error [31] Brain 3D MRI DSC score [47] Lung 3D CT Landmark Pairs [49] Lung 3D CT Registration error [138] Lung 3D CT Registration error…”

Section: Referencesmentioning

confidence: 99%

Deep learning in medical image registration: a review

et al. 2020

Self Cite

View full text Add to dashboard Cite

This paper presents a review of deep learning (DL) based medical image registration methods. We summarized the latest developments and applications of DL-based registration methods in the medical field. These methods were classified into seven categories according to their methods, functions and popularity. A detailed review of each category was presented, highlighting important contributions and identifying specific challenges. A short assessment was presented following the detailed review of each category to summarize its achievements and future potentials. We provided a comprehensive comparison among DL-based methods for lung and brain registration using benchmark datasets. Lastly, we analyzed the statistics of all the cited works from various aspects, revealing the popularity and future trend of DL-based medical image registration. 1 arXiv:1912.12318v1 [eess.IV] 27 Dec 2019 1. Summarize the latest developments in DL-based medical image registration. 2. Highlight contributions, identify challenges and outline future trends. 3. Provide detailed statistics on recent publications from different perspectives. 2 Deep Learning 2 Convolutional Neural NetworkConvolutional neural network (CNN) is a class of deep neural networks with regularized multilayer perceptron. CNN uses convolution operation in place of general matrix multiplication in simple neural networks. The convolutional filters and operations in CNN make it suitable for visual imagery signal processing. Because of its excellent feature extraction ability, CNN is one of the most successful models for image analysis. Since the breakthrough of AlexNet [79], many variants of CNN have been proposed and have achieved the-state-of-art performances in various image processing tasks. A typical CNN usually consists of multiple convolutional layers, max pooling layers, batch normalization layers, dropout layers, a sigmoid or softmax layer. In each convolutional layer, multiple channels of feature maps were extracted by sliding trainable convolutional kernels across the input feature maps. Hierarchical features with high-level abstraction are extracted using multiple convolutional layers. These feature maps usually go through multiple fully connected layer before reaching the final decision layer. Max pooling layers are often used to reduce the image sizes and to promote spatial invariance of the network. Batch normalization is used to reduce internal covariate shift among the training samples. Weight regularization and dropout layers are used to alleviate data overfitting. The loss function is defined as the difference between the predicted and the target output. CNN is usually trained by minimizing the loss via gradient back propagation using optimization methods. Many different types of network architectures have been proposed to improve the performance of CNN [93]. U-Net proposed by Ronneberger et al. is among one of the most used network architectures [120]. U-Net was originally used to perform neuronal structures segmentation. U-Net adopts symmetrical contractiv...

show abstract

Section: Referencesmentioning

confidence: 99%

Deep learning in medical image registration: a review

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…One of the advantages of Siamese networks is that they can facilitate one-shot learning which requires minimal training data. This application is similar to the study presented by Fu et al who suggested a Siamese network architecture for automatic landmark pair detection in 4DCT lung images [20]. However, while their approach is dependent on a large dataset of manually, annotated images, we make use of data augmentation and simulation instead.…”

Section: Networkmentioning

confidence: 78%

“…Traditional approaches to calculating these metrics face the same challenges as manual registration (exhaustive operator intervention, time intensive, high cost, etc.). Because of this, DL methods have also been proposed for performing these evaluations [20][21][22][23]. However, DL-based evaluation strategies are all supervised and heavily reliant on access to large manually annotated datasets which makes it difficult to apply them to emerging imaging techniques [24].…”

Section: Introductionmentioning

confidence: 99%

Combined Mass Spectrometry and Histopathology Imaging for Perioperative Tissue Assessment in Cancer Surgery

et al. 2021

View full text Add to dashboard Cite

Mass spectrometry is an effective imaging tool for evaluating biological tissue to detect cancer. With the assistance of deep learning, this technology can be used as a perioperative tissue assessment tool that will facilitate informed surgical decisions. To achieve such a system requires the development of a database of mass spectrometry signals and their corresponding pathology labels. Assigning correct labels, in turn, necessitates precise spatial registration of histopathology and mass spectrometry data. This is a challenging task due to the domain differences and noisy nature of images. In this study, we create a registration framework for mass spectrometry and pathology images as a contribution to the development of perioperative tissue assessment. In doing so, we explore two opportunities in deep learning for medical image registration, namely, unsupervised, multi-modal deformable image registration and evaluation of the registration. We test this system on prostate needle biopsy cores that were imaged with desorption electrospray ionization mass spectrometry (DESI) and show that we can successfully register DESI and histology images to achieve accurate alignment and, consequently, labelling for future training. This automation is expected to improve the efficiency and development of a deep learning architecture that will benefit the use of mass spectrometry imaging for cancer diagnosis.

show abstract

“…Deep learning-based DIR methods have been proposed for MRI brain, 21 CT head/neck, 22 CT chest, 23 MR/US prostate, 24 4D-CT lung [25][26][27][28] and so on. 29 Eppenhof et al proposed a supervised convolutional neural network (CNN) using U-Net architecture.…”

Section: Related Workmentioning

confidence: 99%

“…Deep learning‐based DIR methods have been proposed for MRI brain, CT head/neck, CT chest, MR/US prostate, 4D‐CT lung and so on . Eppenhof et al .…”

Section: Related Workmentioning

confidence: 99%

LungRegNet: An unsupervised deformable image registration method for 4D‐CT lung

Lei

Wang

et al. 2020

Medical Physics

Self Cite

View full text Add to dashboard Cite

Purpose: To develop an accurate and fast deformable image registration (DIR) method for fourdimensional computed tomography (4D-CT) lung images. Deep learning-based methods have the potential to quickly predict the deformation vector field (DVF) in a few forward predictions. We have developed an unsupervised deep learning method for 4D-CT lung DIR with excellent performances in terms of registration accuracies, robustness, and computational speed. Methods: A fast and accurate 4D-CT lung DIR method, namely LungRegNet, was proposed using deep learning. LungRegNet consists of two subnetworks which are CoarseNet and FineNet. As the name suggests, CoarseNet predicts large lung motion on a coarse scale image while FineNet predicts local lung motion on a fine scale image. Both the CoarseNet and FineNet include a generator and a discriminator. The generator was trained to directly predict the DVF to deform the moving image. The discriminator was trained to distinguish the deformed images from the original images. Coarse-Net was first trained to deform the moving images. The deformed images were then used by the Fine-Net for FineNet training. To increase the registration accuracy of the LungRegNet, we generated vessel-enhanced images by generating pulmonary vasculature probability maps prior to the network prediction. Results: We performed fivefold cross validation on ten 4D-CT datasets from our department. To compare with other methods, we also tested our method using separate 10 DIRLAB datasets that provide 300 manual landmark pairs per case for target registration error (TRE) calculation. Our results suggested that LungRegNet has achieved better registration accuracy in terms of TRE than other deep learning-based methods available in the literature on DIRLAB datasets. Compared to conventional DIR methods, LungRegNet could generate comparable registration accuracy with TRE smaller than 2 mm. The integration of both the discriminator and pulmonary vessel enhancements into the network was crucial to obtain high registration accuracy for 4D-CT lung DIR. The mean and standard deviation of TRE were 1.00 AE 0.53 mm and 1.59 AE 1.58 mm on our datasets and DIRLAB datasets respectively. Conclusions: An unsupervised deep learning-based method has been developed to rapidly and accurately register 4D-CT lung images. LungRegNet has outperformed its deep-learning-based peers and achieved excellent registration accuracy in terms of TRE.

show abstract

Automatic large quantity landmark pairs detection in 4DCT lung images

Cited by 16 publications

References 58 publications

Deep learning in medical image registration: a review

Deep learning in medical image registration: a review

Combined Mass Spectrometry and Histopathology Imaging for Perioperative Tissue Assessment in Cancer Surgery

LungRegNet: An unsupervised deformable image registration method for 4D‐CT lung

Contact Info

Product

Resources

About