Interpretable deep learning approach for oral cancer classification using guided attention inference network

Abstract

. Significance: Convolutional neural networks (CNNs) show the potential for automated classification of different cancer lesions. However, their lack of interpretability and explainability makes CNNs less than understandable. Furthermore, CNNs may incorrectly concentrate on other areas surrounding the salient object, rather than the network’s attention focusing directly on the object to be recognized, as the network has no incentive to focus solely on the correct subjects to be detected. This inhibits the reliability of CNNs, especially for biomedical applications. Aim: Develop a deep learning training approach that could provide understandability to its predictions and directly guide the network to concentrate its attention and accurately delineate cancerous regions of the image. Approach: We utilized Selvaraju et al.’s gradient-weighted class activation mapping to inject interpretability and explainability into CNNs. We adopted a two-stage training process with data augmentation techniques and Li et al.’s guided attention inference network (GAIN) to train images captured using our customized mobile oral screening devices. The GAIN architecture consists of three streams of network training: classification stream, attention mining stream, and bounding box stream. By adopting the GAIN training architecture, we jointly optimized the classification and segmentation accuracy of our CNN by treating these attention maps as reliable priors to develop attention maps with more complete and accurate segmentation. Results: The network’s attention map will help us to actively understand what the network is focusing on and looking at during its decision-making process. The results also show that the proposed method could guide the trained neural network to highlight and focus its attention on the correct lesion areas in the images when making a decision, rather than focusing its attention on relevant yet incorrect regions. Conclusions: We demonstrate the effectiveness of our approach for more interpretable and reliable oral potentially malignant lesion and malignant lesion classification.

Machine learning in point-of-care automated classification of oral potentially malignant and malignant disorders: a systematic review and meta-analysis

“…Specific imaging method not disclosed NN based on DenseNet121 architecture, pre-trained on ImageNet Sensitivity Specificity Precision AUROC F 1 Grad-CAM Sensitivity 98.75 Specificity 100 Precision 100 AUROC 0.99 F 1 0.99 Lin et al 24 Heterogenous dataset from 4 different smartphones NN based on HRNet-W18 architecture, pre-trained on ImageNet Sensitivity Specificity Precision AUROC F 1 Grad-CAM Sensitivity 83.0 Specificity 96.6 Precision 0.84 AUROC 0.946 F 1 0.9 Welikala et al 25 Smartphone images of oral lesions as part of the MeMoSA initiative Multiple pre-trained NNs. Best performing algorithm based on VGG19 architecture Sensitivity Specificity Precision Accuracy F 1 Grad-CAM Sensitivity 85.7 Specificity 76.4 Precision 0.77 Accuracy 80.9 F 1 0.81 Figueroa et al 26 Clinical photographs. Specific imaging method not disclosed NN based on VGG19 architecture, pre-trained on ImageNet Sensitivity Specificity Accuracy Grad-CAM Sensitivity 74.4 Specificity 89.1 Accuracy 83.8 Warin et al 27 SLR camera NN based on ResNet architecture, pre-trained on ImageNet Sensitivity Specificity Precision AUROC Sensitivity 98.4 Specificity 91.7 Precision 92.0 AUROC 0.950 Tanriver et al 28 Clinical photographs taken in clinical department, supplemented by images from various search engines Multiple pre-trained NNs; best performance using EfficientNet-b4 architecture Sensitivity Precision F 1 Sensitivity 89.3 Precision 86.2 F 1 85.7 Jeyaraj et al 29 …”

“…S1 . Eight studies were found to have a high risk of bias across any of the 7 domains 2 , 16 , 21 , 22 , 26 , 28 , 30 , 35 . Within domain 1, 11% of studies were found to have high risk of bias, 26% low risk of bias, and 63% unclear risk of bias.…”

“…Just 8 studies were published before 2015 15 , 34 , 37 , 38 , 44 , 48 – 50 . The majority of studies provided data on classification of OSCC vs healthy (n = 13) 16 , 18 , 19 , 23 , 31 , 33 , 38 , 42 , 43 , 46 – 49 , 8 studies provided data on OSCC/OPMD vs healthy 25 , 26 , 28 , 30 , 37 , 39 – 41 , 6 on OSCC/OPMD vs benign lesions 15 , 17 , 21 , 35 , 36 , 50 , 3 on OSCC vs benign 29 , 34 , 44 , 2 on OSCC vs other (healthy, benign and OPMD) 2 , 45 , 1 on OSCC/OPMD vs benign/healthy 20 , 1 on OPMD vs healthy 27 , and 1 on OPMD vs benign 22 .…”

Intelligent Deep Learning Enabled Oral Squamous Cell Carcinoma Detection and Classification Using Biomedical Images

“…Figueroa et al [ 15 ] designed a DL training model which provides understandability to its prediction and guides the network to remain focused and precisely delineate the tumorous region of the image. Lim et al [ 16 ] developed a DL architecture called D'OraCa to categorize oral lesions with photographic images.…”

“…Next, compared and investigated the efficiency of regularization, convolution neural network, and transfer learning technique for classifying oral tumors. Figueroa et al [15] designed a DL training model which provides understandability to its prediction and guides the network to remain focused and precisely delineate the tumorous region of the image. Lim et al [16] developed a DL architecture called D'OraCa to categorize oral lesions with photographic images.…”