Fusion of encoder-decoder deep networks improves delineation of multiple nuclear phenotypes

Abstract

BackgroundNuclear segmentation is an important step for profiling aberrant regions of histology sections. If nuclear segmentation can be resolved, then new biomarkers of nuclear phenotypes and their organization can be predicted for the application of precision medicine. However, segmentation is a complex problem as a result of variations in nuclear geometry (e.g., size, shape), nuclear type (e.g., epithelial, fibroblast), nuclear phenotypes (e.g., vesicular, aneuploidy), and overlapping nuclei. The problem is further complicated as a result of variations in sample preparation (e.g., fixation, staining). Our hypothesis is that (i) deep learning techniques can learn complex phenotypic signatures that rise in tumor sections, and (ii) fusion of different representations (e.g., regions, boundaries) contributes to improved nuclear segmentation.ResultsWe have demonstrated that training of deep encoder-decoder convolutional networks overcomes complexities associated with multiple nuclear phenotypes, where we evaluate alternative architecture of deep learning for an improved performance against the simplicity of the design. In addition, improved nuclear segmentation is achieved by color decomposition and combining region- and boundary-based features through a fusion network. The trained models have been evaluated against approximately 19,000 manually annotated nuclei, and object-level Precision, Recall, F1-score and Standard Error are reported with the best F1-score being 0.91. Raw training images, annotated images, processed images, and source codes are released as a part of the Additional file 1.ConclusionsThere are two intrinsic barriers in nuclear segmentation to H&E stained images, which correspond to the diversity of nuclear phenotypes and perceptual boundaries between adjacent cells. We demonstrate that (i) the encoder-decoder architecture can learn complex phenotypes that include the vesicular type; (ii) delineation of overlapping nuclei is enhanced by fusion of region- and edge-based networks; (iii) fusion of ENets produces an improved result over the fusion of UNets; and (iv) fusion of networks is better than multitask learning. We suggest that our protocol enables processing a large cohort of whole slide images for applications in precision medicine.Electronic supplementary materialThe online version of this article (10.1186/s12859-018-2285-0) contains supplementary material, which is available to authorized users.

Accurate Neuronal Soma Segmentation Using 3D Multi-Task Learning U-Shaped Fully Convolutional Neural Networks

“…These methods assume that the neuronal soma is ball-like or ellipsoid-like, which may not suit irregular-shaped neuronal somata. The FCNs show better performance in both localization and segmentation, which is most likely due to their encoder-decoder architecture (Khoshdeli et al, 2018 ). Compared with the CPC random walker and rayburst sampling algorithms, the FCNs were able to learn effective feature representation from raw images and make accurate predictions.…”

“…Importantly, the predicted contour is helpful for splitting touching objects. Recently, encoder-decoder FCNs (Khoshdeli et al, 2018 ) were used in nuclei segmentation and performed well for varying nuclear phenotypes. It should be noted that a weak supervised 3D neuronal network has been applied to neuronal soma segmentation (Dong et al, 2018 ).…”

Biomarkers of Tumor Heterogeneity in Glioblastoma Multiforme Cohort of TCGA

“…Deep learning is now the preferred method for image-based analysis and representation [ 4 ]. For instance, cytological analyses use nuclear segmentation that extends U-Net [ 5 , 6 ] coupled with adversarial training [ 7 ], which is highly effective, particularly in identifying vesicular nuclear phenotypes [ 8 ] that traditional methods [ 9 ] could not detect. Although a thorough review of nuclear segmentation and feature-based representation for computational histopathology is beyond the scope of this manuscript, this article provides a summary of several studies focused on the analysis of low-grade glioma and GBM.…”

Number of necessary training examples for Neural Networks with different number of trainable parameters

“… 8 Another exciting application concerns the use of instance-based segmentation in pathology images. 25 , 26 , 29 , 39 There interactions between tumor cells and immune cells were investigated employing spatial proteomics methods in a formalin-fixed and paraffin-embedded substrate. 26 …”

“… 25 , 26 , 29 , 39 There interactions between tumor cells and immune cells were investigated employing spatial proteomics methods in a formalin-fixed and paraffin-embedded substrate. 26 …”