Deep learning based tissue analysis predicts outcome in colorectal cancer

Abstract

Image-based machine learning and deep learning in particular has recently shown expert-level accuracy in medical image classification. In this study, we combine convolutional and recurrent architectures to train a deep network to predict colorectal cancer outcome based on images of tumour tissue samples. The novelty of our approach is that we directly predict patient outcome, without any intermediate tissue classification. We evaluate a set of digitized haematoxylin-eosin-stained tumour tissue microarray (TMA) samples from 420 colorectal cancer patients with clinicopathological and outcome data available. The results show that deep learning-based outcome prediction with only small tissue areas as input outperforms (hazard ratio 2.3; CI 95% 1.79–3.03; AUC 0.69) visual histological assessment performed by human experts on both TMA spot (HR 1.67; CI 95% 1.28–2.19; AUC 0.58) and whole-slide level (HR 1.65; CI 95% 1.30–2.15; AUC 0.57) in the stratification into low- and high-risk patients. Our results suggest that state-of-the-art deep learning techniques can extract more prognostic information from the tissue morphology of colorectal cancer than an experienced human observer.

Artificial intelligence in digital pathology: a roadmap to routine use in clinical practice

“…The adoption of digital pathology (DP) technologies to replace microscopy has been slow and adoption of the use of image analysis/AI tools to augment the workflow or solve capacity issues is limited. Algorithms have the potential to either perform routine tasks which are currently undertaken by pathologists or provide new insights into disease, which are not possible by a human observer .…”

The Nordic EMBL Partnership – a paragon of collaboration

“…The Digital Diagnostics for Precision Medicine Grand Challenge focuses on developing devices and applications that enable artificial intelligence‐supported automated diagnostics. The paradigm shift from human expert‐based interpretations to computerized readouts has vast implications in research . In the future, pathology will become a more quantitative science, with an expert's decisions supported by an array of readouts performed by computer vision.…”

The use of digital pathology and image analysis in clinical trials

“…By training complex computation models directly from data it is often possible to build algorithms that surpass the capabilities of traditional image analysis methods. Examples include the scoring of PD‐L1 , quantification of immune infiltrates to predict outcomes in testicular tumours , detecting sentinel lymph node metastases and superior prediction of colorectal cancer outcome compared to standard morphological assessment . The interpretation of human versus deep learning studies in pathology can be affected by methodological considerations.…”