Synthetic MRI-aided multi-organ segmentation on male pelvic CT using cycle consistent deep attention network

Dong, Xue; Lei, Yang; Tian, Sibo; Wang, Tonghe; Patel, Pretesh; Curran, Walter J.; Jani, Ashesh B.; Liu, Tian; Yang, Xiaofeng

doi:10.1016/j.radonc.2019.09.028

Cited by 118 publications

(130 citation statements)

References 26 publications

Supporting

Mentioning

129

Contrasting

Order By: Relevance

“…Multiple variants of GAN include conditional GAN (cGan) [109], InfoGan [16], , CycleGAN [184], StarGan [19] and so on. In medical imaging, GAN has been used to perform image synthesis for inter-or intra-modality, such as MR to synthetic CT [84,89], CT to synthetic MR [27,83], CBCT to synthetic CT [58], non-attenuation correction (non-AC) PET to CT [26], low-dose PET to synthetic full-dose PET [88], non-AC PET to AC PET [28], low-dose CT to full-dose CT [159] and so on. In medical image registration, GAN is usually used to either provide additional regularization or translate multi-modal registration to unimodal registration.…”

Section: Generative Adversarial Networkmentioning

confidence: 99%

“…The field of medical image registration has been evolving rapidly with hundreds of papers published each year. Recently, DL-based methods have changed the landscape of medical image processing research and achieved the-state-of-art performances in many applications [25,27,45,58,84,85,86,88,89,97,98,156,157,158,160,161]. However, deep learning in medical image registration has not been extensively studied until the past three to four years.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep learning in medical image registration: a review

et al. 2020

Self Cite

View full text Add to dashboard Cite

This paper presents a review of deep learning (DL) based medical image registration methods. We summarized the latest developments and applications of DL-based registration methods in the medical field. These methods were classified into seven categories according to their methods, functions and popularity. A detailed review of each category was presented, highlighting important contributions and identifying specific challenges. A short assessment was presented following the detailed review of each category to summarize its achievements and future potentials. We provided a comprehensive comparison among DL-based methods for lung and brain registration using benchmark datasets. Lastly, we analyzed the statistics of all the cited works from various aspects, revealing the popularity and future trend of DL-based medical image registration. 1 arXiv:1912.12318v1 [eess.IV] 27 Dec 2019 1. Summarize the latest developments in DL-based medical image registration. 2. Highlight contributions, identify challenges and outline future trends. 3. Provide detailed statistics on recent publications from different perspectives. 2 Deep Learning 2 Convolutional Neural NetworkConvolutional neural network (CNN) is a class of deep neural networks with regularized multilayer perceptron. CNN uses convolution operation in place of general matrix multiplication in simple neural networks. The convolutional filters and operations in CNN make it suitable for visual imagery signal processing. Because of its excellent feature extraction ability, CNN is one of the most successful models for image analysis. Since the breakthrough of AlexNet [79], many variants of CNN have been proposed and have achieved the-state-of-art performances in various image processing tasks. A typical CNN usually consists of multiple convolutional layers, max pooling layers, batch normalization layers, dropout layers, a sigmoid or softmax layer. In each convolutional layer, multiple channels of feature maps were extracted by sliding trainable convolutional kernels across the input feature maps. Hierarchical features with high-level abstraction are extracted using multiple convolutional layers. These feature maps usually go through multiple fully connected layer before reaching the final decision layer. Max pooling layers are often used to reduce the image sizes and to promote spatial invariance of the network. Batch normalization is used to reduce internal covariate shift among the training samples. Weight regularization and dropout layers are used to alleviate data overfitting. The loss function is defined as the difference between the predicted and the target output. CNN is usually trained by minimizing the loss via gradient back propagation using optimization methods. Many different types of network architectures have been proposed to improve the performance of CNN [93]. U-Net proposed by Ronneberger et al. is among one of the most used network architectures [120]. U-Net was originally used to perform neuronal structures segmentation. U-Net adopts symmetrical contractiv...

show abstract

Section: Generative Adversarial Networkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Deep learning in medical image registration: a review

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Given the recent advances in machine learning techniques, automatic segmentation methods have been developed based on machine learning algorithms, especially, deep convolutional neural networks. [24][25][26][27][28][29][30][31][32][33][34] Deep learning-based catheter segmentation algorithms have been recently reported in CT, 35 MRI, 17,18 and US [36][37][38] for biopsy and brachytherapy. While these methods are inspiring, reliable catheter detection approaches have to be further developed specifically for digitizing the catheters in MRI-guided HDR prostate brachytherapy.…”

Section: Introductionmentioning

confidence: 99%

Automatic multi‐catheter detection using deeply supervised convolutional neural network in MRI‐guided HDR prostate brachytherapy

et al. 2020

Self Cite

View full text Add to dashboard Cite

High-dose-rate (HDR) brachytherapy is an established technique to be used as monotherapy option or focal boost in conjunction with external beam radiation therapy (EBRT) for treating prostate cancer. Radiation source path reconstruction is a critical procedure in HDR treatment planning. Manually identifying the source path is labor intensive and time inefficient. In recent years, magnetic resonance imaging (MRI) has become a valuable imaging modality for image-guided HDR prostate brachytherapy due to its superb soft-tissue contrast for target delineation and normal tissue contouring. The purpose of this study is to investigate a deep-learning-based method to automatically reconstruct multiple catheters in MRI for prostate cancer HDR brachytherapy treatment planning. Methods: Attention gated U-Net incorporated with total variation (TV) regularization model was developed for multi-catheter segmentation in MRI. The attention gates were used to improve the accuracy of identifying small catheter points, while TV regularization was adopted to encode the natural spatial continuity of catheters into the model. The model was trained using the binary catheter annotation images offered by experienced physicists as ground truth paired with original MRI images. After the network was trained, MR images of a new prostate cancer patient receiving HDR brachytherapy were fed into the model to predict the locations and shapes of all the catheters. Quantitative assessments of our proposed method were based on catheter shaft and tip errors compared to the ground truth. Results: Our method detected 299 catheters from 20 patients receiving HDR prostate brachytherapy with a catheter tip error of 0.37 AE 1.68 mm and a catheter shaft error of 0.93 AE 0.50 mm. For detection of catheter tips, our method resulted in 87% of the catheter tips within an error of less than AE 2.0 mm, and more than 71% of the tips can be localized within an absolute error of no >1.0 mm. For catheter shaft localization, 97% of catheters were detected with an error of <2.0 mm, while 63% were within 1.0 mm. Conclusions: In this study, we proposed a novel multi-catheter detection method to precisely localize the tips and shafts of catheters in three-dimensional MRI images of HDR prostate brachytherapy. It paves the way for elevating the quality and outcome of MRI-guided HDR prostate brachytherapy.

show abstract

“…As is shown in Fig.2, the long skip connection was implemented by concatenating these feature maps extracted from the layer of encoding path with same sized feature maps extracted from the layer of decoding path. Attention gate (AG) could capture the most relevant semantic (segment) information without enlarging the receptive field [3]. Since in this work, the target RSP image is close to a semantic image, we propose to integrate AG into the long skip connection to highlight the semantic features from feature maps extracted from previous layer of encoding path.…”

Section: Introductionmentioning

confidence: 99%

“…Since in this work, the target RSP image is close to a semantic image, we propose to integrate AG into the long skip connection to highlight the semantic features from feature maps extracted from previous layer of encoding path. The details of the implementation of attention gate can be found in our previous work [3].…”

Section: Introductionmentioning

confidence: 99%

Deep learning-based relative stopping power mapping generation with cone-beam CT in proton radiation therapy

Wang

Harms

Lei

et al. 2020

Medical Imaging 2020: Physics of Medical Imaging

Self Cite

View full text Add to dashboard Cite

Purpose:Dual-energy CT (DECT) has been used to derive relative stopping power (RSP) map by obtaining the energy dependence of photon interactions. The DECT-derived RSP maps could potentially be compromised by image noise levels and the severity of artifacts when using physics-based mapping techniques, which would affect subsequent clinical applications. This work presents a noise-robust learning-based method to predict RSP maps from DECT for proton radiation therapy.Method:The proposed method uses a residual attention cycle-consistent generative adversarial network. Residual blocks with attention gates were used to force the model focus on the difference between RSP maps and DECT images. Cycle consistent generative adversarial networks were used to let the DECT-to-RSP mapping be close to a one-to-one mapping by introducing an inverse RSP-to-DECT mapping. To evaluate the accuracy of the method, we retrospectively investigated 20 head-and-neck cancer patients with DECT scans in proton radiation therapy simulation. Ground truth RSP values were assigned by calculation based on chemical compositions. These ground truth RSP maps acted as learning targets in the training process for DECT datasets, and were evaluated against results from the proposed method using a leave-one-out cross-validation strategy.Result:The predicted RSP maps showed an average normalized mean square error (NMSE) of 2.83% across the whole body volume, and average mean error (ME) less than 3% in all volumes of interest (VOIs). With additional simulated noise added in DECT datasets, the proposed method still maintained a comparable performance, while the physics-based stoichiometric method suffered degraded inaccuracy from increased noise level. Based on the statistical analysis of comparative dose volume histogram (DVH) metrics of dose maps calculated on ground truth and predicted RSP maps among 19 pencil beam scanning proton treatment plans, the average differences in DVH metrics for clinical target volumes (CTVs) were less than 0.2 Gy for D95% and Dmax with no statistical significance (average prescription dose = 60 Gy). Maximum difference in DVH metrics of organs-at-risk (OARs) was around 1 Gy on average. Conclusion:These results strongly indicate the high accuracy of RSP maps predicted by our machinelearning-based method and show its potential feasibility for proton treatment planning and dose calculation.

show abstract

Synthetic MRI-aided multi-organ segmentation on male pelvic CT using cycle consistent deep attention network

Cited by 118 publications

References 26 publications

Deep learning in medical image registration: a review

Deep learning in medical image registration: a review

Automatic multi‐catheter detection using deeply supervised convolutional neural network in MRI‐guided HDR prostate brachytherapy

Deep learning-based relative stopping power mapping generation with cone-beam CT in proton radiation therapy

Contact Info

Product

Resources

About