Image-based consensus molecular subtype (imCMS) classification of colorectal cancer using deep learning

Abstract

ObjectiveComplex phenotypes captured on histological slides represent the biological processes at play in individual cancers, but the link to underlying molecular classification has not been clarified or systematised. In colorectal cancer (CRC), histological grading is a poor predictor of disease progression, and consensus molecular subtypes (CMSs) cannot be distinguished without gene expression profiling. We hypothesise that image analysis is a cost-effective tool to associate complex features of tissue organisation with molecular and outcome data and to resolve unclassifiable or heterogeneous cases. In this study, we present an image-based approach to predict CRC CMS from standard H&E sections using deep learning.DesignTraining and evaluation of a neural network were performed using a total of n=1206 tissue sections with comprehensive multi-omic data from three independent datasets (training on FOCUS trial, n=278 patients; test on rectal cancer biopsies, GRAMPIAN cohort, n=144 patients; and The Cancer Genome Atlas (TCGA), n=430 patients). Ground truth CMS calls were ascertained by matching random forest and single sample predictions from CMS classifier.ResultsImage-based CMS (imCMS) accurately classified slides in unseen datasets from TCGA (n=431 slides, AUC)=0.84) and rectal cancer biopsies (n=265 slides, AUC=0.85). imCMS spatially resolved intratumoural heterogeneity and provided secondary calls correlating with bioinformatic prediction from molecular data. imCMS classified samples previously unclassifiable by RNA expression profiling, reproduced the expected correlations with genomic and epigenetic alterations and showed similar prognostic associations as transcriptomic CMS.ConclusionThis study shows that a prediction of RNA expression classifiers can be made from H&E images, opening the door to simple, cheap and reliable biological stratification within routine workflows.

State of machine and deep learning in histopathological applications in digestive diseases

“…The consensus molecular subtypes (CMS) transcriptionally distinguish four groups of colorectal cancer with different clinical behaviors and biology, so Sirinukunwattana et al [ 76 ] trained a model to designate image-based CMS (imCMS) classes to HE-stained slides. The authors used two resection cohorts (TCGA, FOCUS) and one biopsy cohort (GRAMPIAN), and patches were extracted from WSI regions annotated by pathologists to be tumor.…”

“…This study by Sirinukunwattana et al [ 76 ] offered two additional novelties in CMS classification. First, patches with high prediction confidence for imCMS subtypes could be extracted to examine histological patterns.…”

Status quo and future prospects of artificial neural network from the perspective of gastroenterologists

“…CNN algorithms were used to predict RNA expression classifiers from H&E images. This work lays the foundation for a comprehensive integration of morphology and molecular features[ 32 ]. Immunohistochemical staining and fluorescence microscopy belong to members of pathological examinations, and ANNs distinctly improved their diagnostic performance in GI tumors[ 33 - 35 ].…”

Requirements for implementation of artificial intelligence in the practice of gastrointestinal pathology

“…Specimens of various GI cancers can be processed to identify molecular markers that may predict responses to targeted therapies. Research has shown that certain clinically relevant molecular alterations in GI cancers are associated with specific histopathological features detected on hematoxylin-eosin (HE) slides; there have been some successful attempts to adopt AI applications for HE sections as surrogate markers for these alterations[ 31 - 34 ].…”

“…The DL algorithm demonstrated an AUC of 0.96 in the multi-institutional validation cohort. Furthermore, the consensus molecular subtype of colorectal cancer could be predicted from the images of colorectal surgical specimens using a CNN-based model[ 31 ]. Although prediction of molecular alterations by AI application might seem attractive, as clinically relevant biomarkers cannot be identified using HE stained slides and conventional PCR assay are both expensive and time-consuming, AI can neither achieve complete concordance with the gold standard test nor replace it.…”