Multiple Instance Learning for Heterogeneous Images: Training a CNN for Histopathology

Couture, Heather D.; Marron, J. S.; Perou, Charles M.; Troester, Melissa A.; Niethammer, Marc

doi:10.1007/978-3-030-00934-2_29

Cited by 49 publications

(32 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our approaches still work because classifiers can tolerate some error in the training data 52 . In the machine learning literature, this corresponds to the general problem of multi-label, multi-instance supervised learning with imbalanced data, an active area of research including for medical image data 28,[53][54][55] .…”

Section: Discussionmentioning

confidence: 99%

Deep learning-based cross-classifications reveal conserved spatial behaviors within tumor histological images

et al. 2020

View full text Add to dashboard Cite

Histopathological images are a rich but incompletely explored data type for studying cancer. Manual inspection is time consuming, making it challenging to use for image data mining. Here we show that convolutional neural networks (CNNs) can be systematically applied across cancer types, enabling comparisons to reveal shared spatial behaviors. We develop CNN architectures to analyze 27,815 hematoxylin and eosin scanned images from The Cancer Genome Atlas for tumor/normal, cancer subtype, and mutation classification. Our CNNs are able to classify TCGA pathologist-annotated tumor/normal status of whole slide images (WSIs) in 19 cancer types with consistently high AUCs (0.995 ± 0.008), as well as subtypes with lower but significant accuracy (AUC 0.87 ± 0.1). Remarkably, tumor/normal CNNs trained on one tissue are effective in others (AUC 0.88 ± 0.11), with classifier relationships also recapitulating known adenocarcinoma, carcinoma, and developmental biology. Moreover, classifier comparisons reveal intra-slide spatial similarities, with an average tile-level correlation of 0.45 ± 0.16 between classifier pairs. Breast cancers, bladder cancers, and uterine cancers have spatial patterns that are particularly easy to detect, suggesting these cancers can be canonical types for image analysis. Patterns for TP53 mutations can also be detected, with WSI self- and cross-tissue AUCs ranging from 0.65-0.80. Finally, we comparatively evaluate CNNs on 170 breast and colon cancer images with pathologist-annotated nuclei, finding that both cellular and intercellular regions contribute to CNN accuracy. These results demonstrate the power of CNNs not only for histopathological classification, but also for cross-comparisons to reveal conserved spatial behaviors across tumors.

show abstract

Section: Discussionmentioning

confidence: 99%

Deep learning-based cross-classifications reveal conserved spatial behaviors within tumor histological images

et al. 2020

View full text Add to dashboard Cite

show abstract

“…For NLP or sentiment analysis, saliency map can also take the form of "heat" scores over words in texts, as demonstrated by Arras et al [62] using LRP and by Karpathy et al [63]. In the medical field (see later section), Irvin et al [6], Zhao et al [44], Paschali et al [64], Couture et al [65], Li et al [66], Qin et al [67], Tang et al [68], Papanastasopoulos et al [69], and Lee et al [70] have studied methods employing saliency and visual explanations. It is noted that we also subcategorize LIME as a method that uses optimization and sensitivity as its underlying mechanisms, and many researches on interpretability span more than one subcategories.…”

Section: A Perceptive Interpretabilitymentioning

confidence: 99%

“…Multilayer CAM (MLCAM) is introduced in [91] for glioma (a type of brain tumor) localization. Multiinstance (MI) aggregation method is used with CNN to classify breast tumor tissue microarray (TMA) image's for five different tasks [65], for example the classification of the histologic subtype. Super-pixel maps indicate the region in each TMA image where the tumor cells are; each label corresponds to a class of tumor.…”

Section: A Perceptive Interpretabilitymentioning

confidence: 99%

A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI

Tjoa

Guan

2021

IEEE Trans. Neural Netw. Learning Syst.

1,079

547

View full text Add to dashboard Cite

Recently, artificial intelligence and machine learning in general have demonstrated remarkable performances in many tasks, from image processing to natural language processing, especially with the advent of deep learning (DL). Along with research progress, they have encroached upon many different fields and disciplines. Some of them require high level of accountability and thus transparency, for example, the medical sector. Explanations for machine decisions and predictions are thus needed to justify their reliability. This requires greater interpretability, which often means we need to understand the mechanism underlying the algorithms. Unfortunately, the blackbox nature of the DL is still unresolved, and many machine decisions are still poorly understood. We provide a review on interpretabilities suggested by different research works and categorize them. The different categories show different dimensions in interpretability research, from approaches that provide "obviously" interpretable information to the studies of complex patterns. By applying the same categorization to interpretability in medical research, it is hoped that: 1) clinicians and practitioners can subsequently approach these methods with caution; 2) insight into interpretability will be born with more considerations for medical practices; and 3) initiatives to push forward data-based, mathematically grounded, and technically grounded medical education are encouraged.

show abstract

“…When only weak annotations are available for images, such as in heterogeneous images, it is often useful to turn to multiple instance learning (MIL). Courture et al [237] described a CNN using quantile function for the classification of 5 types of breast tumor histology. They finetuned AlexNet [23].…”

Section: Breastmentioning

confidence: 99%

Going Deep in Medical Image Analysis: Concepts, Methods, Challenges, and Future Directions

et al. 2019

View full text Add to dashboard Cite

Medical Image Analysis is currently experiencing a paradigm shift due to Deep Learning. This technology has recently attracted so much interest of the Medical Imaging community that it led to a specialized conference in 'Medical Imaging with Deep Learning' in the year 2018. This article surveys the recent developments in this direction, and provides a critical review of the related major aspects. We organize the reviewed literature according to the underlying Pattern Recognition tasks, and further sub-categorize it following a taxonomy based on human anatomy. This article does not assume prior knowledge of Deep Learning and makes a significant contribution in explaining the core Deep Learning concepts to the non-experts in the Medical community. Unique to this study is the Computer Vision/Machine Learning perspective taken on the advances of Deep Learning in Medical Imaging. This enables us to single out 'lack of appropriately annotated large-scale datasets' as the core challenge (among other challenges) in this research direction. We draw on the insights from the sister research fields of Computer Vision, Pattern Recognition and Machine Learning etc.; where the techniques of dealing with such challenges have already matured, to provide promising directions for the Medical Imaging community to fully harness Deep Learning in the future.

show abstract

Multiple Instance Learning for Heterogeneous Images: Training a CNN for Histopathology

Cited by 49 publications

References 14 publications

Deep learning-based cross-classifications reveal conserved spatial behaviors within tumor histological images

Deep learning-based cross-classifications reveal conserved spatial behaviors within tumor histological images

A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI

Going Deep in Medical Image Analysis: Concepts, Methods, Challenges, and Future Directions

Contact Info

Product

Resources

About