Assessing the (Un)Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging

Arun, Nishanth; Gaw, Nathan; Singh, Praveer; Chang, Ken; Aggarwal, Mehak; Chen, Bryan; Hoebel, Katharina; Gupta, Sharut; Patel, Jay; Gidwani, Mishka; Adebayo, Julius; Li, Matthew; Kalpathy-Cramer, Jayashree

doi:10.48550/arxiv.2008.02766

Cited by 8 publications

(9 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These methods arose in computer vision and have demonstrated empirical utility in producing nonlinear factor models where the factors are conceptually sensible. Yet, due to the black-box nature of deep learning, explanations for how the factors are generated from the data, using local saliency maps for instance, are unreliable or imprecise (Laugel et al, 2019;Slack et al, 2020;Arun et al, 2020). In imaging applications, where the features are raw pixels, this type of interpretability is unnecessary.…”

Section: Disentangled Autoencodersmentioning

confidence: 99%

Sparse encoding for more-interpretable feature-selecting representations in probabilistic matrix factorization

Chang,

Fletcher,

Han

et al. 2020

Preprint

View full text Add to dashboard Cite

Dimensionality reduction methods for count data are critical to a wide range of applications in medical informatics and other fields where model interpretability is paramount. For such data, hierarchical Poisson matrix factorization (HPF) and other sparse probabilistic non-negative matrix factorization (NMF) methods are considered to be interpretable generative models. They consist of sparse transformations for decoding their learned representations into predictions. However, sparsity in representation decoding does not necessarily imply sparsity in the encoding of representations from the original data features. HPF is often incorrectly interpreted in the literature as if it possesses encoder sparsity. The distinction between decoder sparsity and encoder sparsity is subtle but important. Due to the lack of encoder sparsity, HPF does not possess the column-clustering property of classical NMF -the factor loading matrix does not sufficiently define how each factor is formed from the original features. We address this deficiency by self-consistently enforcing encoder sparsity, using a generalized additive model (GAM), thereby allowing one to relate each representation coordinate to a subset of the original data features. In doing so, the method also gains the ability to perform feature selection. We demonstrate our method on simulated data and give an example of how encoder sparsity is of practical use in a concrete application of representing inpatient comorbidities in Medicare patients.

show abstract

Section: Disentangled Autoencodersmentioning

confidence: 99%

Sparse encoding for more-interpretable feature-selecting representations in probabilistic matrix factorization

Chang,

Fletcher,

Han

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Nevertheless, the diagnosis of COVID through imaging is a delicate problem, and its solution should present high robustness to risk in order to be useful. A number of authors have written on the challenges of such a use Arun et al [2020], DeGrave et al [2020], Wynants et al [2020] with some pointing that probably the most promising use of X-rays would be at assessing disease severity and progression in a prognostic approach Cohen et al [2020b], Manna et al [2020]. With this in mind, this methodology processes a different use of the data set, where classes of patients with different ICU-related outcomes are adopted.…”

Section: Data Setsmentioning

confidence: 99%

“…One popular technique for interpretation on convolutional networks is the saliency map, which provides a heatmap overlay of network attention calculated on a gradient basis Simonyan and Zisserman [2014]. Nevertheless, there are still serious concerns as to whether saliency mapping techniques accurately reflect trained model parameters Adebayo et al [2018] since they do not have shown to be robust under rigorous examination in the context of medial imaging Arun et al [2020].…”

mentioning

confidence: 99%

MAVIDH Score: A COVID-19 Severity Scoring using Chest X-Ray Pathology Features

Gomes,

Horry,

Ulhaq

et al. 2020

Preprint

View full text Add to dashboard Cite

The application of computer vision for COVID-19 diagnosis is complex and challenging, given the risks associated with patient misclassifications. Arguably, the primary value of medical imaging for COVID-19 lies rather on patient prognosis. Radiological images can guide physicians assessing the severity of the disease, and a series of images from the same patient at different stages can help to gauge disease progression. Hence, a simple method based on lung-pathology features for scoring disease severity from Chest X-rays is proposed here. As the primary contribution, this method shows to be correlated to patient severity in different stages of disease progression comparatively well to other existing methods. An original approach for data selection is also proposed, allowing the simple model to learn the severity-related features. It is hypothesized that the resulting competitive performance presented here is related to the method being feature-based rather than reliant on lung involvement or compromise as others in the literature. The fact that it is simpler and interpretable than other end-to-end, more complex models, also sets aside this work. As the data set is small, bias-inducing artifacts that could lead to overfitting are minimized through an image normalization and lung segmentation step at the learning phase. A second contribution comes from the validation of the results, conceptualized as the scoring of patients groups from different stages of the disease. Besides performing such validation on an independent data set, the results were also compared with other proposed scoring methods in the literature. The expressive results show that although imaging alone is not sufficient for assessing severity as a whole, there is a strong correlation with the scoring system, termed as MAVIDH score, with patient outcome. The method has the potential to help physicians rating and following disease progression in COVID-19 patients, especially when making decisions on resource-scarce environments.

show abstract

“…(2) Explain black box neural networks posthoc by creating approximations, saliency maps, or derivatives. Posthoc explanations can be problematic; for instance saliency maps highlight regions of the image (show attention), but can be unreliable and misleading, as they tend to highlight edges and do not show what computation is actually done with the highlighted pixels [Rudin, 2019;Adebayo et al, 2018;Arun et al, 2020]. There are several types of approaches in interpretable machine learning, including case-based reasoning (which we use here), forcing the network to use logical conditions within its last layers [e.g., Wu and Song, 2019], or disentangling the neural network's latent space [e.g., .…”

Section: Introductionmentioning

confidence: 99%

Interpretable Mammographic Image Classification using Case-Based Reasoning and Deep Learning

Barnett,

Schwartz,

Tao

et al. 2021

Preprint

View full text Add to dashboard Cite

When we deploy machine learning models in highstakes medical settings, we must ensure these models make accurate predictions that are consistent with known medical science. Inherently interpretable networks address this need by explaining the rationale behind each decision while maintaining equal or higher accuracy compared to blackbox models. In this work, we present a novel interpretable neural network algorithm that uses casebased reasoning for mammography. Designed to aid a radiologist in their decisions, our network presents both a prediction of malignancy and an explanation of that prediction using known medical features. In order to yield helpful explanations, the network is designed to mimic the reasoning processes of a radiologist: our network first detects the clinically relevant semantic features of each image by comparing each new image with a learned set of prototypical image parts from the training images, then uses those clinical features to predict malignancy. Compared to other methods, our model detects clinical features (mass margins) with equal or higher accuracy, provides a more detailed explanation of its prediction, and is better able to differentiate the classification-relevant parts of the image.

show abstract

Assessing the (Un)Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging

Cited by 8 publications

References 0 publications

Sparse encoding for more-interpretable feature-selecting representations in probabilistic matrix factorization

Sparse encoding for more-interpretable feature-selecting representations in probabilistic matrix factorization

MAVIDH Score: A COVID-19 Severity Scoring using Chest X-Ray Pathology Features

Interpretable Mammographic Image Classification using Case-Based Reasoning and Deep Learning

Contact Info

Product

Resources

About