BackgroundFor virtually every patient with colorectal cancer (CRC), hematoxylin–eosin (HE)–stained tissue slides are available. These images contain quantitative information, which is not routinely used to objectively extract prognostic biomarkers. In the present study, we investigated whether deep convolutional neural networks (CNNs) can extract prognosticators directly from these widely available images.Methods and findingsWe hand-delineated single-tissue regions in 86 CRC tissue slides, yielding more than 100,000 HE image patches, and used these to train a CNN by transfer learning, reaching a nine-class accuracy of >94% in an independent data set of 7,180 images from 25 CRC patients. With this tool, we performed automated tissue decomposition of representative multitissue HE images from 862 HE slides in 500 stage I–IV CRC patients in the The Cancer Genome Atlas (TCGA) cohort, a large international multicenter collection of CRC tissue. Based on the output neuron activations in the CNN, we calculated a “deep stroma score,” which was an independent prognostic factor for overall survival (OS) in a multivariable Cox proportional hazard model (hazard ratio [HR] with 95% confidence interval [CI]: 1.99 [1.27–3.12], p = 0.0028), while in the same cohort, manual quantification of stromal areas and a gene expression signature of cancer-associated fibroblasts (CAFs) were only prognostic in specific tumor stages. We validated these findings in an independent cohort of 409 stage I–IV CRC patients from the “Darmkrebs: Chancen der Verhütung durch Screening” (DACHS) study who were recruited between 2003 and 2007 in multiple institutions in Germany. Again, the score was an independent prognostic factor for OS (HR 1.63 [1.14–2.33], p = 0.008), CRC-specific OS (HR 2.29 [1.5–3.48], p = 0.0004), and relapse-free survival (RFS; HR 1.92 [1.34–2.76], p = 0.0004). A prospective validation is required before this biomarker can be implemented in clinical workflows.ConclusionsIn our retrospective study, we show that a CNN can assess the human tumor microenvironment and predict prognosis directly from histopathological images.

This study highlights the importance of pathological reporting and suggests that tumor growth patterns and thorough examination but not surgical technique determine R1 resection rates in PDAC.

Analysis of tumor-infiltrating lymphocytes (TIL) in primary human colorectal cancer (CRC) by in situ immunohistochemical staining supports the hypothesis that the adaptive immune response influences the course of human CRC. Specifically, high densities of TILs in the primary tumor are associated with good prognosis independent of other prognostic markers. However, the prognostic role of TILs in metastatic CRC lesions is unknown, as is their role in response or resistance to conventional chemotherapy. We analyzed the association of TIL densities at the invasive margin of CRC liver metastases with response to chemotherapy and progression-free survival in a set of 101 large section samples. High-resolution automated microscopy on complete tissue sections was used to objectively generate cell densities for CD3, CD8, granzyme B, or FOXP3 positive immune cells. A predictive scoring system using TIL densities was developed in a training set and tested successfully in an independent validation set. TIL densities at the invasive margin of liver metastases allowed the prediction of response to chemotherapy with a sensitivity of 79% and specificity of 100%. The association of high density values with longer progression-free survival under chemotherapy was statistically significant. Overall, these findings extend the impact of the local immune response on the clinical course from the primary tumor also to metastatic lesions. Because detailed quantification of TILs in metastatic lesions revealed a strong association with chemotherapy efficacy and prognosis, we suggest that the developed scoring system may be used as a predictive tool for response to chemotherapy in metastatic CRC. Cancer Res; 71(17); 5670-7. Ó2011 AACR.