Automatic Recognition of Colon and Esophagogastric Cancer with Machine Learning and Hyperspectral Imaging

Abstract

There are approximately 1.8 million diagnoses of colorectal cancer, 1 million diagnoses of stomach cancer, and 0.6 million diagnoses of esophageal cancer each year globally. An automatic computer-assisted diagnostic (CAD) tool to rapidly detect colorectal and esophagogastric cancer tissue in optical images would be hugely valuable to a surgeon during an intervention. Based on a colon dataset with 12 patients and an esophagogastric dataset of 10 patients, several state-of-the-art machine learning methods have been trained to detect cancer tissue using hyperspectral imaging (HSI), including Support Vector Machines (SVM) with radial basis function kernels, Multi-Layer Perceptrons (MLP) and 3D Convolutional Neural Networks (3DCNN). A leave-one-patient-out cross-validation (LOPOCV) with and without combining these sets was performed. The ROC-AUC score of the 3DCNN was slightly higher than the MLP and SVM with a difference of 0.04 AUC. The best performance was achieved with the 3DCNN for colon cancer and esophagogastric cancer detection with a high ROC-AUC of 0.93. The 3DCNN also achieved the best DICE scores of 0.49 and 0.41 on the colon and esophagogastric datasets, respectively. These scores were significantly improved using a patient-specific decision threshold to 0.58 and 0.51, respectively. This indicates that, in practical use, an HSI-based CAD system using an interactive decision threshold is likely to be valuable. Experiments were also performed to measure the benefits of combining the colorectal and esophagogastric datasets (22 patients), and this yielded significantly better results with the MLP and SVM models.

In vivo multi spectral colonoscopy in mice

“…This pinpoints that the usage of spatial features is a very important step in future for multispectral endoscopy. Furthermore, the MCC in this study is nearly identical to the ex vivo results from Collins et al 27 in which they used a 3d convolutional neural network (3DCNN). Finally, the multispectral data set with spatial features will likely have a similar data dimensionality as hyperspectral data sets.…”

New Intraoperative Imaging Tools and Image-Guided Surgery in Gastric Cancer Surgery

“…In addition, the combination of deep learning models and micro-spectral analyses has opened new perspectives for tissue characterization when compared to medical pathology [ 27 , 51 ]. Gastric tumor-classification based on micro-hyperspectral technology was performed by Hu et al [ 52 ].…”

Intraoperative Guidance Using Hyperspectral Imaging: A Review for Surgeons

“…Recently, a French-German collaborative group using fresh specimens of 12 patients with colorectal cancer and 10 with esophageal cancer, were able to achieve an acceptable automating recognition accuracy by using several machine learning supervised methods (Receiver Operator Curve Area-Under-Curve: ROC-AUC of 0.92 for colorectal and of 0.93 for esophagogastric cancer, respectively) [28]. Interestingly, the authors noticed a significant improvement in the automatic classification by combining both cancer types within the learning datasets.…”