Ensembles of Multiple Models and Architectures for Robust Brain Tumour Segmentation

Kamnitsas, Konstantinos; Bai, Wenjia; Ferrante, Enzo; McDonagh, Steven; Sinclair, Matthew; Pawlowski, Nick; Rajchl, Martin; Lee, Matthew C. H.; Kainz, Bernhard; Rueckert, Daniel; Glocker, Ben

doi:10.1007/978-3-319-75238-9_38

Cited by 335 publications

(241 citation statements)

References 29 publications

Supporting

Mentioning

230

Contrasting

Order By: Relevance

“…Rather than truly correcting for scatter or other physical effects, a machine learning‐based method will learn to map voxels from a source distribution (CBCT) to a target distribution (planning CT) . A variety of machine learning‐based methods have been used in other applications of radiation therapy, including conversion of MR images to CT images, prediction of radiation toxicities, dose calculation, and automatic segmentation . Although machine learning‐based methods can be computationally expensive to train, once a well‐trained model is developed, the image correction can be applied in seconds, making this approach ideal for online ART.…”

Section: Introductionmentioning

confidence: 99%

Paired cycle‐GAN‐based image correction for quantitative cone‐beam computed tomography

et al. 2019

View full text Add to dashboard Cite

Purpose The incorporation of cone‐beam computed tomography (CBCT) has allowed for enhanced image‐guided radiation therapy. While CBCT allows for daily 3D imaging, images suffer from severe artifacts, limiting the clinical potential of CBCT. In this work, a deep learning‐based method for generating high quality corrected CBCT (CCBCT) images is proposed. Methods The proposed method integrates a residual block concept into a cycle‐consistent adversarial network (cycle‐GAN) framework, called res‐cycle GAN, to learn a mapping between CBCT images and paired planning CT images. Compared with a GAN, a cycle‐GAN includes an inverse transformation from CBCT to CT images, which constrains the model by forcing calculation of both a CCBCT and a synthetic CBCT. A fully convolution neural network with residual blocks is used in the generator to enable end‐to‐end CBCT‐to‐CT transformations. The proposed algorithm was evaluated using 24 sets of patient data in the brain and 20 sets of patient data in the pelvis. The mean absolute error (MAE), peak signal‐to‐noise ratio (PSNR), normalized cross‐correlation (NCC) indices, and spatial non‐uniformity (SNU) were used to quantify the correction accuracy of the proposed algorithm. The proposed method is compared to both a conventional scatter correction and another machine learning‐based CBCT correction method. Results Overall, the MAE, PSNR, NCC, and SNU were 13.0 HU, 37.5 dB, 0.99, and 0.05 in the brain, 16.1 HU, 30.7 dB, 0.98, and 0.09 in the pelvis for the proposed method, improvements of 45%, 16%, 1%, and 93% in the brain, and 71%, 38%, 2%, and 65% in the pelvis, over the CBCT image. The proposed method showed superior image quality as compared to the scatter correction method, reducing noise and artifact severity. The proposed method produced images with less noise and artifacts than the comparison machine learning‐based method. Conclusions The authors have developed a novel deep learning‐based method to generate high‐quality corrected CBCT images. The proposed method increases onboard CBCT image quality, making it comparable to that of the planning CT. With further evaluation and clinical implementation, this method could lead to quantitative adaptive radiation therapy.

show abstract

Section: Introductionmentioning

confidence: 99%

Paired cycle‐GAN‐based image correction for quantitative cone‐beam computed tomography

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Different feature fusion approaches using deep neural networks are discussed: The first and most basic method is to combine the input images/features and process them jointly in a single UNET. This describes most methods in the literature for handling multi‐image modality datasets such as the fusion of multiparametric brain MR images of T1‐ and T2‐ weighted MRI for brain tumor segmentation The second approach is to conduct feature fusion on images of different resolutions through two steps of extracting different sizes of input patches in the input images and giving them as inputs of different networks to obtain the different feature levels to conduct the feature fusion. The last method is to conduct feature fusion based on deep convolutional and recurrent neural networks (RNN), where the RNNs are responsible for exploiting the intraslice and interslice contexts respectively …”

Section: Discussionmentioning

confidence: 99%

“…In terms of the network architecture design, our DFCN‐CoSeg network was inspired by the encoder‐decoder based 3D fully convolutional networks (3D‐FCN)and the 3D‐UNets . As a natural extension of the well‐known 2D FCN proposed by Long et al ,.…”

Section: Discussionmentioning

confidence: 99%

“…In this work, we attempt to address these challenges and seek data‐driven deep learning solutions for automatically delineating features directly from PET‐CT scans to develop a computer‐aided automatic processing tool for tumor segmentation. Deep learning is able to outperform most conventional approaches and is able to manage many medical tasks . Several publications detail the potential power of this approach .…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Simultaneous cosegmentation of tumors in PET‐CT images using deep fully convolutional networks

et al. 2019

View full text Add to dashboard Cite

Purpose To investigate the use and efficiency of 3‐D deep learning, fully convolutional networks (DFCN) for simultaneous tumor cosegmentation on dual‐modality nonsmall cell lung cancer (NSCLC) and positron emission tomography (PET)‐computed tomography (CT) images. Methods We used DFCN cosegmentation for NSCLC tumors in PET‐CT images, considering both the CT and PET information. The proposed DFCN‐based cosegmentation method consists of two coupled three‐dimensional (3D)‐UNets with an encoder‐decoder architecture, which can communicate with the other in order to share complementary information between PET and CT. The weighted average sensitivity and positive predictive values denoted as Scores, dice similarity coefficients (DSCs), and the average symmetric surface distances were used to assess the performance of the proposed approach on 60 pairs of PET/CTs. A Simultaneous Truth and Performance Level Estimation Algorithm (STAPLE) of 3 expert physicians’ delineations were used as a reference. The proposed DFCN framework was compared to 3 graph‐based cosegmentation methods. Results Strong agreement was observed when using the STAPLE references for the proposed DFCN cosegmentation on the PET‐CT images. The average DSCs on CT and PET are 0.861 ± 0.037 and 0.828 ± 0.087, respectively, using DFCN, compared to 0.638 ± 0.165 and 0.643 ± 0.141, respectively, when using the graph‐based cosegmentation method. The proposed DFCN cosegmentation using both PET and CT also outperforms the deep learning method using either PET or CT alone. Conclusions The proposed DFCN cosegmentation is able to outperform existing graph‐based segmentation methods. The proposed DFCN cosegmentation shows promise for further integration with quantitative multimodality imaging tools in clinical trials.

show abstract

“…This network performed on par with the proposed DLM for all metrics except the HD95 metric, where it had a significant lower performance ( P < 0.01) (Data ). In future work, we may explore other neural network architectures, such as those with multiscale patch‐based networks or ensembles of different architectures for improved results.…”

Section: Discussionmentioning

confidence: 99%

Knowledge‐based and deep learning‐based automated chest wall segmentation in magnetic resonance images of extremely dense breasts

et al. 2019

View full text Add to dashboard Cite

Purpose Segmentation of the chest wall, is an important component of methods for automated analysis of breast magnetic resonance imaging (MRI). Methods reported to date show promising results but have difficulties delineating the muscle border correctly in breasts with a large proportion of fibroglandular tissue (i.e., dense breasts). Knowledge‐based methods (KBMs) as well as methods based on deep learning have been proposed, but a systematic comparison of these approaches within one cohort of images is currently lacking. Therefore, we developed a KBM and a deep learning method for segmentation of the chest wall in MRI of dense breasts and compared their performances. Methods Two automated methods were developed, an optimized KBM incorporating heuristics aimed at shape, location, and gradient features, and a deep learning‐based method (DLM) using a dilated convolution neural network. A data set of 115 T1‐weighted MR images was randomly selected from MR images of women with extremely dense breasts (ACR BI‐RADS category 4) participating in a screening trial of women (mean age 56.6 yr, range 49.5–75.2 yr) with dense breasts. Manual segmentations of the chest wall, acquired under supervision of an experienced breast radiologist, were available for all data sets. Both methods were optimized using the same randomly selected 36 MRI data sets from a total of 115 data sets. Each MR data set consisted of 179 transversal images with voxel size 0.64 mm3 × 0.64 mm3 × 1.00 mm3. In the remaining 79 data sets, the results of both segmentation methods were qualitatively evaluated. A radiologist reviewed the segmentation results of both methods in all transversal images (n = 14 141) and determined whether the result would impact the ability to accurately determine the volume of fibroglandular and fatty tissue and whether segmentations masked breast regions that might harbor lesions. When no relevant deviation was detected, the result was considered successful. In addition, all segmentations were quantitatively assessed using the Dice similarity coefficient (DSC) and Hausdorff distance (HD), 95th percentile of the Hausdorff distance (HD95), false positive fraction (FPF), and false negative fraction (FNF) metrics. Results According to the radiologist's evaluation, the DLM had a significantly higher success rate than the KBM (81.6% vs 78.4%, P < 0.01). The success rate was further improved to 92.1% by combining both methods. Similarly, the DLM had significantly lower values for FNF (0.003 ± 0.003 vs 0.009 ± 0.011, P < 0.01) and HD95 (2.58 ± 1.78 mm vs 3.37 ± 2.11, P < 0.01). However, the KBM resulted in a significantly lower FPF than the DLM (0.018 ± 0.009 vs 0.030 ± 0.009, P < 0.01).There was no significant difference between the KBM and DLM in terms of DSC (0.982 ± 0.006 vs 0.984 ± 0.008, P = 0.08) or HD (24.14 ± 20.69 mm vs 12.81 ± 27.28 mm, P = 0.05). Conclusion Both optimized knowledge‐based and DLM showed good results to segment the pectoral muscle in women with dense breasts. Qualitatively assessed, the DLM was the most robust...

show abstract

Ensembles of Multiple Models and Architectures for Robust Brain Tumour Segmentation

Cited by 335 publications

References 29 publications

Paired cycle‐GAN‐based image correction for quantitative cone‐beam computed tomography

Paired cycle‐GAN‐based image correction for quantitative cone‐beam computed tomography

Simultaneous cosegmentation of tumors in PET‐CT images using deep fully convolutional networks

Knowledge‐based and deep learning‐based automated chest wall segmentation in magnetic resonance images of extremely dense breasts

Contact Info

Product

Resources

About