Visually interpretable deep network for diagnosis of breast masses on mammograms

Abstract

Recently, deep learning technology has achieved various successes in medical image analysis studies including computer-aided diagnosis (CADx). However, current CADx approaches based on deep learning have a limitation in interpreting diagnostic decisions. The limited interpretability is a major challenge for practical use of current deep learning approaches. In this paper, a novel visually interpretable deep network framework is proposed to provide diagnostic decisions with visual interpretation. The proposed method is motivated by the fact that the radiologists characterize breast masses according to the breast imaging reporting and data system (BIRADS). The proposed deep network framework consists of a BIRADS guided diagnosis network and a BIRADS critic network. A 2D map, named BIRADS guide map, is generated in the inference process of the deep network. The visual features extracted from the breast masses could be refined by the BIRADS guide map, which helps the deep network to focus on more informative areas. The BIRADS critic network makes the BIRADS guide map to be relevant to the characterization of masses in terms of BIRADS description. To verify the proposed method, comparative experiments have been conducted on public mammogram database. On the independent test set (170 malignant masses and 170 benign masses), the proposed method was found to have significantly higher performance compared to the deep network approach without using the BIRADS guide map (p < 0.05). Moreover, the visualization was conducted to show the location where the deep network exploited more information. This study demonstrated that the proposed visually interpretable CADx framework could be a promising approach for visually interpreting the diagnostic decision of the deep network.

Endometrium segmentation on transvaginal ultrasound image using key‐point discriminator

“…Recently, deep learning‐based approaches have shown great success in image processing and computer‐aided applications . In particular, many studies on automatic segmentation with natural images using deep convolutional neural networks have achieved outstanding performances.…”

“…Recently, deep learning-based approaches have shown great success in image processing and computer-aided applications. [10][11][12][13] In particular, many studies on automatic segmentation with natural images using deep convolutional neural networks have achieved outstanding performances. Long et al applied a fully convolutional network (FCN) to powerful existing CNNs (e.g., AlexNet and VGG16) for object segmentation by replacing fully connected layers with fully convolutional layers.…”

Interpretation and visualization techniques for deep learning models in medical imaging

“…Similar weakly-supervised approaches using CAMs for chest x-ray abnormality and breast mass localization is described in ( Hwang and Kim, 2016 ), and for ACL tear localization on knee MRI in ( Liu et al , 2019 ). Kim et al computed class activation maps for the classification of benign vs malignant breast masses on mammograms, but found them difficult to interpret for their task, as shown in Figure 6C ( Kim et al , 2018 ). Other applications of class activation maps to medical imaging tasks include localization of diabetic retinopathy lesions in retinal fundus images ( Gondal et al , 2017 ), and weakly supervised diagnosis of tuberculosis on chest x-rays ( Hwang and Kim, 2016 ).…”

“…Recently, artificial intelligence (AI) and, more specifically, deep learning (DL), approaches have achieved state of the art results for many medical imaging tasks including image segmentation ( Hu et al , 2017 ; Kamnitsas et al , 2017 ; Roth et al , 2015b ), disease detection and diagnosis ( Gao and Noble, 2017 ; Roth et al , 2016 ; Kim et al , 2018 ; Huynh et al , 2016 ; Roth et al , 2014 ), and image classification ( Yang et al , 2018 ; Chen and Shi, 2018 ; Van Molle et al , 2018 ; Shen and Gao, 2018 ; Yi et al , 2017 ). The workhorse of DL applications for medical imaging is the convolutional neural network (CNN).…”

Radiomics in Lung Cancer from Basic to Advanced: Current Status and Future Directions

“…However, the learned features are challenging to interpret, in stark contrast to radiomics features, where each feature has an analytical formula with possible physical interpretations. Many efforts have been made to address this interpretability issue (58,59). Another issue is the scarcity of pre-trained DL networks for medical imaging.…”