Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images

Abstract

Copies of full items can be used for personal research or study, educational, or not-for profit purposes without prior permission or charge. Provided that the authors, title and full bibliographic details are credited, a hyperlink and/or URL is given for the original metadata page and the content is not changed in any way.Publisher's statement: "© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works." A note on versions:The version presented here may differ from the published version or, version of record, if you wish to cite this item you are advised to consult the publisher's version. Please see the 'permanent WRAP url' above for details on accessing the published version and note that access may require a subscription. Abstract-Detection and classification of cell nuclei in histopathology images of cancerous tissue stained with the standard hematoxylin and eosin stain is a challenging task due to cellular heterogeneity. Deep learning approaches have been shown to produce encouraging results on histopathology images in various studies. In this paper, we propose a Spatially Constrained Convolutional Neural Network (SC-CNN) to perform nucleus detection. SC-CNN regresses the likelihood of a pixel being the center of a nucleus, where high probability values are spatially constrained to locate in the vicinity of the center of nuclei. For classification of nuclei, we propose a novel Neighboring Ensemble Predictor (NEP) coupled with CNN to more accurately predict the class label of detected cell nuclei. The proposed approaches for detection and classification do not require segmentation of nuclei. We have evaluated them on a large dataset of colorectal adenocarcinoma images, consisting of more than 20,000 annotated nuclei belonging to four different classes. Our results show that the joint detection and classification of the proposed SC-CNN and NEP produces the highest average F1 score as compared to other recently published approaches. Prospectively, the proposed methods could offer benefit to pathology practice in terms of quantitative analysis of tissue constituents in whole-slide images, and could potentially lead to a better understanding of cancer.

A survey on deep learning in medical image analysis

“…Many works add features derived from deep networks to existing feature sets or compare (2013) Detection of basal cell carcinoma H&E Convolutional auto-encoder neural network Malon and Cosatto (2013) Mitosis detection H&E Combines shapebased features with CNN Wang et al (2014) Mitosis detection H&E Cascaded ensemble of CNN and handcrafted features Ferrari et al (2015) Bacterial colony counting Culture plate CNN-based patch classifier Ronneberger et al (2015) Cell segmentation EM U-Net with deformation augmentation Shkolyar et al (2015) Mitosis detection Live-imaging CNN-based patch classifier Song et al (2015) Segmentation of cytoplasm and nuclei H&E Multi-scale CNN and graph-partitioning-based method Xie et al (2015a) Nucleus detection Ki-67 CNN model that learns the voting offset vectors and voting confidence Xie et al (2015b) Nucleus detection H&E, Ki-67 CNN-based structured regression model for cell detection Akram et al (2016) Cell segmentation FL, PC, H&E fCNN for cell bounding box proposal and CNN for segmentation Albarqouni et al (2016) Mitosis detection H&E Incorporated 'crowd sourcing' layer into the CNN framework Bauer et al (2016) Nucleus classification IHC CNN-based patch classifier Chen et al (2016b) Mitosis detection H&E Deep regression network (DRN) Gao et al (2016e) Nucleus classification IFL Classification of Hep2-cells with CNN Han et al (2016) Nucleus classification IFL Classification of Hep2-cells with CNN Janowczyk et al (2016b) Nucleus segmentation H&E Resolution adaptive deep hierarchical learning scheme Kashif et al (2016) Nucleus detection H&E Combination of CNN and hand-crafted features Mao and Yin (2016) Mitosis detection PC Hierarchical CNNs for patch sequence classification Mishra et al (2016) Classification of mitochondria EM CNN-based patch classifier Phan et al (2016) Nucleus classification FL Classification of Hep2-cells using transfer learning (pre-trained CNN) Romo-Bucheli et al (2016) Tubule nuclei detection H&E CNN-based classification of pre-selected candidate nuclei Sirinukunwattana et al (2016) Nucleus detection and classification H&E CNN with spatially constrained regression Song et al (2017) Cell segmentation H&E Multi-scale C...…”

Nucleus Classification in Colon Cancer H&E Images using Deep Learning

“…Weighted F1 Score Our CNN 0.775 Softmax CNN+SSPP [5] 0.748 Superpixel Descriptor [6] 0.687 CRImage [7] 0.488 Table 1. Comparative results for nucleus classification.…”

“…The weighted average F1 score for our network is 0.7749 which is higher than the previous published works as shown in Table 1. Our model converges to minimum error in 30 epochs while the network in [5] converges in 120 epochs. Our weighted precision and recall are 0.7739 and 0.7759, respectively.…”

“…To implement the nucleus classification task, we designed a Convolution Neural Network (CNN) [4]. The ability of CNN in learning high-dimensional and complex classes of objects makes it a very useful tool for a variety of classification tasks in biomedical imaging.Our network was trained and evaluated by using a large colon cancer nuclei images dataset reported in [5] with 22,444 nuclei divided into four classes (Epithelial, Inflammatory, Fibroblast, and Miscellaneous) with the following number of samples (7,722,6,971,5,712, and 2,039), respectively. Our model consists of six layers (2 convolution layers, 2 subsampling layers, 1 fully connected layer, and output layer).…”

Discriminative bag-of-cells for imaging-genomics

“…Most studies have relied upon popular microscopy image analysis software, such as CellProfiler, 12 to perform image feature extraction, 13 but the commonly used available algorithms fail to generalize adequately to the significant variation of most large-scale histopathological image data sets. Recently, several methods for nuclear detection and segmentation have been developed using deep learning and convolutional neural networks (CNNs), [14][15][16][17] which have shown improvements over traditional methods. For our framework, we developed our own patch-based CNN for nuclear segmentation, which we optimized for computational efficiency and trained using additional image data from TCGA.…”