Accurate diagnosis of colorectal cancer based on histopathology images using artificial intelligence

Abstract

Background Accurate and robust pathological image analysis for colorectal cancer (CRC) diagnosis is time-consuming and knowledge-intensive, but is essential for CRC patients’ treatment. The current heavy workload of pathologists in clinics/hospitals may easily lead to unconscious misdiagnosis of CRC based on daily image analyses. Methods Based on a state-of-the-art transfer-learned deep convolutional neural network in artificial intelligence (AI), we proposed a novel patch aggregation strategy for clinic CRC diagnosis using weakly labeled pathological whole-slide image (WSI) patches. This approach was trained and validated using an unprecedented and enormously large number of 170,099 patches, > 14,680 WSIs, from > 9631 subjects that covered diverse and representative clinical cases from multi-independent-sources across China, the USA, and Germany. Results Our innovative AI tool consistently and nearly perfectly agreed with (average Kappa statistic 0.896) and even often better than most of the experienced expert pathologists when tested in diagnosing CRC WSIs from multicenters. The average area under the receiver operating characteristics curve (AUC) of AI was greater than that of the pathologists (0.988 vs 0.970) and achieved the best performance among the application of other AI methods to CRC diagnosis. Our AI-generated heatmap highlights the image regions of cancer tissue/cells. Conclusions This first-ever generalizable AI system can handle large amounts of WSIs consistently and robustly without potential bias due to fatigue commonly experienced by clinical pathologists. It will drastically alleviate the heavy clinical burden of daily pathology diagnosis and improve the treatment for CRC patients. This tool is generalizable to other cancer diagnosis based on image recognition.

Colon Tissues Classification and Localization in Whole Slide Images Using Deep Learning

“…On the other hand, for colonic adenoma and adenocarcinoma, AUC 0.99 and AUC 0.96 were achieved, respectively. Considering 170,099 patches obtained from around 14,680 WSIs of more than 9631 subjects, the first-ever huge generalizable AI system was developed in [44]. The system used a novel patch aggregation strategy for the CRC diagnosis using weakly labeled WSI, wherein the Inception-v3 was used as the architecture with weights initialized from the transfer learning.…”

“…As presented in Table 5, a recent study [12] used TL with EfficientNet for the classification of breast cancer images and achieved an accuracy of 98.33%. Furthermore, another study [44] used TL with Inception-v3 and achieved AUC 0.988 for CRC classification. The achievements of both works were comparable to the performance of the IR-v2 Type 5 model.…”

Status quo and future prospects of artificial neural network from the perspective of gastroenterologists

“…Most studies concerning colorectal cancer (CRC) have also focused on distinguishing between benign and malignant tissues. Based on existing results, the accuracies or area under the curve (AUC) values of ANN-based models all exceed 80%[ 19 - 25 ]. Colon glands are important structures and indicators for pathological assessment[ 26 ].…”

A systematic review on application of deep learning in digestive system image processing

“…There have been retrospective studies on DL in the pathological diagnosis and prognosis analysis of Helicobacter pylori gastritis [ 78 ], rectal cancer [ 79 ], pancreatic tumors [ 80 ], gastrointestinal, and endocrine tumors [ 81 ]. Prospective, multi-center, and large-scale trials have also begun to verify these algorithms’ usability [ 82 ]. However, these studies generally have the problem of low interpretative ability for the results of CAD.…”

“…There have been retrospective studies on DL in the pathological diagnosis and prognosis analysis of Helicobacter pylori gastritis [78], rectal cancer [79], pancreatic tumors [80], gastrointestinal, and endocrine tumors [81]. Prospective, multi-center, and large-scale trials have also begun to verify these algorithms' usability [82]. However, these studies generally have the problem of low interpretative ability for the results of The number of metastatic lymph nodes is an essential determinant of the TNM staging of gastrointestinal malignant tumors and is also one of the most critical factors in evaluating gastric cancer prognosis.…”