A deep learning model integrating FCNNs and CRFs for brain tumor segmentation

Abstract

Accurate and reliable brain tumor segmentation is a critical component in cancer diagnosis, treatment planning, and treatment outcome evaluation. Build upon successful deep learning techniques, a novel brain tumor segmentation method is developed by integrating fully convolutional neural networks (FCNNs) and Conditional Random Fields (CRFs) in a unified framework to obtain segmentation results with appearance and spatial consistency. We train a deep learning based segmentation model using 2D image patches and image slices in following steps: 1) training FCNNs using image patches; 2) training CRFs as Recurrent Neural Networks (CRF-RNN) using image slices with parameters of FCNNs fixed; and 3) fine-tuning the FCNNs and the CRF-RNN using image slices. Particularly, we train 3 segmentation models using 2D image patches and slices obtained in axial, coronal and sagittal views respectively, and combine them to segment brain tumors using a voting based fusion strategy. Our method could segment brain images slice-by-slice, much faster than those based on image patches. We have evaluated our method based on imaging data provided by the Multimodal Brain Tumor Image Segmentation Challenge (BRATS) 2013, BRATS 2015 and BRATS 2016. The experimental results have demonstrated that our method could build a segmentation model with Flair, T1c, and T2 scans and achieve competitive performance as those built with Flair, T1, T1c, and T2 scans.

Cardiac substructure segmentation with deep learning for improved cardiac sparing

“…The improvement in segmentation agreement observed, along with the use of a 3D‐CRF to remove spurious isolated regions, parallels other emerging uses of 3D‐CRF post‐processing in medical imaging . Although this study implemented CRFs as a post‐processing step, some current studies have integrated CRFs into the utilized neural network and have seen improved segmentation performance and can be explored in future work for possible coronary artery segmentation improvement.…”

Multiclassifier for severity‐level categorization of glioma tumors using multimodal magnetic resonance imaging brain images

“…Thus, a deep learning model by Zhao et al 26 solved the issues in Reference 25, but the segmentation performance was affected due to the difference between the number of pixels and classes. Finally, the challenges of the existing methods lay a platform for proposing an automatic method.…”

“…The significance of the fractional concept is that the candidate solutions of the previous iterations are included while determining the best candidate solution as per the modified equation (26). The fractional constant is referred to as α , and the value varies in [0, 1].…”

Prospects of deep learning for medical imaging

“…In medical image analysis, spatial information is exploited using CNN, and temporal information is often exploited using RNN. Studies using dynamic imaging adopted a combination of CNN and RNN so that joint modeling of spatial and temporal information was possible references [72][73][74][75][76][77].…”

“…Image segmentation: Zhao et al [77] proposed a tumor segmentation method based on DL using multimodal brain MRI. They used a network composed of the combination of CNN and RNN architectures in which CNNs assigned the segmentation label to each pixel, and then RNNs optimized the segmentation result using both assigned labels and input images.…”