AggNet: Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images

Abstract

The lack of publicly available ground-truth data has been identified as the major challenge for transferring recent developments in deep learning to the biomedical imaging domain. Though crowdsourcing has enabled annotation of large scale databases for real world images, its application for biomedical purposes requires a deeper understanding and hence, more precise definition of the actual annotation task. The fact that expert tasks are being outsourced to non-expert users may lead to noisy annotations introducing disagreement between users. Despite being a valuable resource for learning annotation models from crowdsourcing, conventional machine-learning methods may have difficulties dealing with noisy annotations during training. In this manuscript, we present a new concept for learning from crowds that handle data aggregation directly as part of the learning process of the convolutional neural network (CNN) via additional crowdsourcing layer (AggNet). Besides, we present an experimental study on learning from crowds designed to answer the following questions. 1) Can deep CNN be trained with data collected from crowdsourcing? 2) How to adapt the CNN to train on multiple types of annotation datasets (ground truth and crowd-based)? 3) How does the choice of annotation and aggregation affect the accuracy? Our experimental setup involved Annot8, a self-implemented web-platform based on Crowdflower API realizing image annotation tasks for a publicly available biomedical image database. Our results give valuable insights into the functionality of deep CNN learning from crowd annotations and prove the necessity of data aggregation integration.

A survey on deep learning in medical image analysis

“…Many works add features derived from deep networks to existing feature sets or compare (2013) Detection of basal cell carcinoma H&E Convolutional auto-encoder neural network Malon and Cosatto (2013) Mitosis detection H&E Combines shapebased features with CNN Wang et al (2014) Mitosis detection H&E Cascaded ensemble of CNN and handcrafted features Ferrari et al (2015) Bacterial colony counting Culture plate CNN-based patch classifier Ronneberger et al (2015) Cell segmentation EM U-Net with deformation augmentation Shkolyar et al (2015) Mitosis detection Live-imaging CNN-based patch classifier Song et al (2015) Segmentation of cytoplasm and nuclei H&E Multi-scale CNN and graph-partitioning-based method Xie et al (2015a) Nucleus detection Ki-67 CNN model that learns the voting offset vectors and voting confidence Xie et al (2015b) Nucleus detection H&E, Ki-67 CNN-based structured regression model for cell detection Akram et al (2016) Cell segmentation FL, PC, H&E fCNN for cell bounding box proposal and CNN for segmentation Albarqouni et al (2016) Mitosis detection H&E Incorporated 'crowd sourcing' layer into the CNN framework Bauer et al (2016) Nucleus classification IHC CNN-based patch classifier Chen et al (2016b) Mitosis detection H&E Deep regression network (DRN) Gao et al (2016e) Nucleus classification IFL Classification of Hep2-cells with CNN Han et al (2016) Nucleus classification IFL Classification of Hep2-cells with CNN Janowczyk et al (2016b) Nucleus segmentation H&E Resolution adaptive deep hierarchical learning scheme Kashif et al (2016) Nucleus detection H&E Combination of CNN and hand-crafted features Mao and Yin (2016) Mitosis detection PC Hierarchical CNNs for patch sequence classification Mishra et al (2016) Classification of mitochondria EM CNN-based patch classifier Phan et al (2016) Nucleus classification FL Classification of Hep2-cells using transfer learning (pre-trained CNN) Romo-Bucheli et al (2016) Tubule nuclei detection H&E CNN-based classification of pre-selected candidate nuclei Sirinukunwattana et al (2016) Nucleus detection and classification H&E CNN with spatially constrained regression Song et al (2017) Cell segmentation H&E Multi-scale C...…”

Learning laparoscopic video shot classification for gynecological surgery

“…In total, they achieve best performance regarding recall, precision and f-score compared against logistic regression, SVMs, and CNNs. The importance of CNNs in medical applications is also apparent from their use within other applications such as nucleus segmentation [28], polyp detection in colonoscopy videos [15], microcalcification detection in digital breast tomosynthesis [22], mitosis detection in breast cancer histology [1], and short-term breast cancer risk prediction [19]. Our work is delimited to the aforementioned research as in contrast to the classification of a state (e.g., healthy or consolidation, type of tissue), we aim at classifying both, anatomical structures and surgical actions.…”

Crowd-Algorithm Collaboration for Large-Scale Endoscopic Image Annotation with Confidence

“…It has been shown that anonymous untrained individuals from an online community are able to generate training data of expert quality. Similar achievements were made in other biomedical imaging fields such as histopathological image analysis [2]. A remaining problem, however, is that object segmentation from scratch is relatively time-consuming and thus expensive compared to other tasks outsourced to the crowd.…”