Convolutional Neural Network-Based Clinical Predictors of Oral Dysplasia: Class Activation Map Analysis of Deep Learning Results

Abstract

Oral cancer/oral squamous cell carcinoma is among the top ten most common cancers globally, with over 500,000 new cases and 350,000 associated deaths every year worldwide. There is a critical need for objective, novel technologies that facilitate early, accurate diagnosis. For this purpose, we have developed a method to classify images as “suspicious” and “normal” by performing transfer learning on Inception-ResNet-V2 and generated automated heat maps to highlight the region of the images most likely to be involved in decision making. We have tested the developed method’s feasibility on two independent datasets of clinical photographic images of 30 and 24 patients from the UK and Brazil, respectively. Both 10-fold cross-validation and leave-one-patient-out validation methods were performed to test the system, achieving accuracies of 73.6% (±19%) and 90.9% (±12%), F1-scores of 97.9% and 87.2%, and precision values of 95.4% and 99.3% at recall values of 100.0% and 81.1% on these two respective cohorts. This study presents several novel findings and approaches, namely the development and validation of our methods on two datasets collected in different countries showing that using patches instead of the whole lesion image leads to better performance and analyzing which regions of the images are predictive of the classes using class activation map analysis.

Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methods

“…This interpretability comes at a cost of performance as the proposed network cannot incorporate fully connected layers. Camalan et al [16] used CAMs for interpreting the results of the classification model for oral cancer. Multi-Layer Class Activation Maps (ML-CAM) is an extension of CAM that can be incorporated at different CNN layers [55].…”

“…Method Reference Application Layerwise Relevance Propagation (LRP) [14] Alzheimer's disease classification [33] Multiple Sclerosis diagnosis Class Activation Maps (CAM) [16] Oral cancer classification Gradient-Class Activation Maps (Grad-CAM) [102] Automatic brain tumor grading [99] Detection of COVID-19 from Chest X-ray and CT scans Integrated Gradient (IG) [135] Diabetic Retinopathy (DR) prediction [148] Multiple Sclerosis classification Occlusion [62] Diagnosis of age-related macular degeneration and diabetic macular edema in OCT images [137] Diagnosis of Alzheimer's disease Local Interpretable Model-agnostic Explanations (LIME) [91] Parkinson's disease detection [120] Congestive heart failure prediction kernel SHAP (Linear LIME + Shapley values) [159] Skin cancer detection [169] Lung nodule classification…”

Automatic detection of oral cancer in smartphone-based images using deep learning for early diagnosis

“…With the rapid development of both imaging and sensing technologies in camera systems, the ubiquity of smartphones is equipped with higher quality, low-noise, and faster camera modules. Smartphone-based white light inspection methods 23 – 25 are good solutions for acquiring oral images. Camalan et al.…”

“…Camalan et al. 23 used a CNN-based network for classifying white light images as normal or suspicious; however, the patient sample is very limited: with only 54 cases. To build a reliable system, Welikala et al.…”

Lesion Segmentation Framework Based on Convolutional Neural Networks with Dual Attention Mechanism

“…At present, there are few related works to oral lesion segmentation. Camalan et al [15] developed a image classification method to identify the "suspicious" oral dysplasia or "normal" oral images through transfer learning on Inception-ResNet-V2. Jubair et al [16] proposed a method to predict oral cancer from oral images using a lightweight transfer learning model.…”