SUMMARY Beyond sample curation and basic pathologic characterization, the digitized H&E-stained images of TCGA samples remain underutilized. To highlight this resource, we present mappings of tumor-infiltrating lymphocytes (TILs) based on H&E images from 13 TCGA tumor types. These TIL maps are derived through computational staining using a convolutional neural network trained to classify patches of images. Affinity propagation revealed local spatial structure in TIL patterns and correlation with overall survival. TIL map structural patterns were grouped using standard histopathological parameters. These patterns are enriched in particular T cell subpopulations derived from molecular measures. TIL densities and spatial structure were differentially enriched among tumor types, immune subtypes, and tumor molecular subtypes, implying that spatial infiltrate state could reflect particular tumor cell aberration states. Obtaining spatial lymphocytic patterns linked to the rich genomic characterization of TCGA samples demonstrates one use for the TCGA image archives with insights into the tumor-immune microenvironment.

Convolutional Neural Networks (CNN) are state-of-the-art models for many image classification tasks. However, to recognize cancer subtypes automatically, training a CNN on gigapixel resolution Whole Slide Tissue Images (WSI) is currently computationally impossible. The differentiation of cancer subtypes is based on cellular-level visual features observed on image patch scale. Therefore, we argue that in this situation, training a patch-level classifier on image patches will perform better than or similar to an image-level classifier. The challenge becomes how to intelligently combine patch-level classification results and model the fact that not all patches will be discriminative. We propose to train a decision fusion model to aggregate patch-level predictions given by patch-level CNNs, which to the best of our knowledge has not been shown before. Furthermore, we formulate a novel Expectation-Maximization (EM) based method that automatically locates discriminative patches robustly by utilizing the spatial relationships of patches. We apply our method to the classification of glioma and non-small-cell lung carcinoma cases into subtypes. The classification accuracy of our method is similar to the inter-observer agreement between pathologists. Although it is impossible to train CNNs on WSIs, we experimentally demonstrate using a comparable non-cancer dataset of smaller images that a patch-based CNN can outperform an image-based CNN.

The Cancer Genome Atlas (TCGA) project has generated gene expression data that divides glioblastoma (GBM) into four transcriptional classes: proneural, neural, classical, and mesenchymal. Because transcriptional class is only partially explained by underlying genomic alterations, we hypothesize that the tumor microenvironment may also have an impact. In this study, we focused on necrosis and angiogenesis because their presence is both prognostically and biologically significant. These features were quantified in digitized histological images of TCGA GBM frozen section slides that were immediately adjacent to samples used for molecular analysis. Correlating these features with transcriptional data, we found that the mesenchymal transcriptional class was significantly enriched with GBM samples that contained a high degree of necrosis. Furthermore, among 2422 genes that correlated with the degree of necrosis in GBMs, transcription factors known to drive the mesenchymal expression class were most closely related, including C/EBP-β, C/EBP-δ, STAT3, FOSL2, bHLHE40, and RUNX1. Non-mesenchymal GBMs in the TCGA data set were found to become more transcriptionally similar to the mesenchymal class with increasing levels of necrosis. In addition, high expression levels of the master mesenchymal factors C/EBP-β, C/EBP-δ, and STAT3 were associated with a poor prognosis. Strong, specific expression of C/EBP-β and C/EBP-δ by hypoxic, perinecrotic cells in GBM likely account for their tight association with necrosis and may be related to their poor prognosis.

High-resolution microscopy images of tissue specimens provide detailed information about the morphology of normal and diseased tissue. Image analysis of tissue morphology can help cancer researchers develop a better understanding of cancer biology. Segmentation of nuclei and classification of tissue images are two common tasks in tissue image analysis. Development of accurate and efficient algorithms for these tasks is a challenging problem because of the complexity of tissue morphology and tumor heterogeneity. In this paper we present two computer algorithms; one designed for segmentation of nuclei and the other for classification of whole slide tissue images. The segmentation algorithm implements a multiscale deep residual aggregation network to accurately segment nuclear material and then separate clumped nuclei into individual nuclei. The classification algorithm initially carries out patch-level classification via a deep learning method, then patch-level statistical and morphological features are used as input to a random forest regression model for whole slide image classification. The segmentation and classification algorithms were evaluated in the MICCAI 2017 Digital Pathology challenge. The segmentation algorithm achieved an accuracy score of 0.78. The classification algorithm achieved an accuracy score of 0.81. These scores were the highest in the challenge.

No abstract

Almost 20 years have passed since the commercial introduction of whole-slide imaging (WSI) scanners. During this time, the creation of various WSI devices with the ability to digitize an entire glass slide has transformed the field of pathology. Parallel advances in computational technology and storage have permitted rapid processing of large-scale WSI datasets. This article provides an overview of important past and present efforts related to WSI. An account of how the virtual microscope evolved from the need to visualize and manage satellite data for earth science applications is provided. The article also discusses important milestones beginning from the first WSI scanner designed by Bacus to the Food and Drug Administration approval of the first digital pathology system for primary diagnosis in surgical pathology. As pathology laboratories commit to going fully digitalize, the need has emerged to include WSIs into an enterprise-level vendor-neutral archive (VNA). The different types of VNAs available are reviewed as well as how best to implement them and how pathology can benefit from participating in this effort. Differences between traditional image algorithms that extract pixel-, object-, and semantic-level features versus deep learning methods are highlighted. The need for large-scale data management, analysis, and visualization in computational pathology is also addressed.

The Cancer Genome Atlas Project (TCGA) has produced an extensive collection of ‘-omic’ data on glioblastoma (GBM), resulting in several key insights on expression signatures. Despite the richness of TCGA GBM data, the absence of lower grade gliomas in this data set prevents analysis genes related to progression and the uncovering of predictive signatures. A complementary dataset exists in the form of the NCI Repository for Molecular Brain Neoplasia Data (Rembrandt), which contains molecular and clinical data for diffuse gliomas across the full spectrum of histologic class and grade. Here we present an investigation of the significance of the TCGA consortium's expression classification when applied to Rembrandt gliomas. We demonstrate that the proneural signature predicts improved clinical outcome among 176 Rembrandt gliomas that includes all histologies and grades, including GBMs (log rank test p = 1.16e-6), but also among 75 grade II and grade III samples (p = 2.65e-4). This gene expression signature was enriched in tumors with oligodendroglioma histology and also predicted improved survival in this tumor type (n = 43, p = 1.25e-4). Thus, expression signatures identified in the TCGA analysis of GBMs also have intrinsic prognostic value for lower grade oligodendrogliomas, and likely represent important differences in tumor biology with implications for treatment and therapy. Integrated DNA and RNA analysis of low-grade and high-grade proneural gliomas identified increased expression and gene amplification of several genes including GLIS3, TGFB2, TNC, AURKA, and VEGFA in proneural GBMs, with corresponding loss of DLL3 and HEY2. Pathway analysis highlights the importance of the Notch and Hedgehog pathways in the proneural subtype. This demonstrates that the expression signatures identified in the TCGA analysis of GBMs also have intrinsic prognostic value for low-grade oligodendrogliomas, and likely represent important differences in tumor biology with implications for treatment and therapy.

With the increasing availability of whole-slide scanners, analysis of digitized pathology images will become increasingly important in linking morphologic observations with genomic and clinical endpoints.