Transparency of deep neural networks for medical image analysis: A review of interpretability methods

Salahuddin, Zohaib; Woodruff, Henry C.; Chatterjee, A.; Lambin, Philippe

doi:10.1016/j.compbiomed.2021.105111

Cited by 210 publications

(107 citation statements)

References 63 publications

Supporting

Mentioning

106

Contrasting

Unclassified

Order By: Relevance

“…However, one limitation of this model is that it does not satisfactorily explain its decisions. As deep learning models have been increasingly applied to medical image analysis, there is an evolving interest in the interpretability of these models ( Salahuddin et al, 2022 ; Lipton, 2017 ; Zech et al, 2018 ; Ghassemi et al, 2021 ). While an exhaustive interpretation of deep learning QC models is beyond the scope of this work, we provided a preliminary qualitative interpretation of the CNN-i model (Figure 6) that demonstrates the intuitive nature of its decisions.…”

Section: Discussionmentioning

confidence: 99%

“…Here, we expand the hybrid-QC approach to a large multi-site dMRI dataset. Moreover, one of the common critiques of deep learning is that it can learn irrelevant features of the data and does not provide information that is transparent enough to interpret ( Lipton, 2017 ; Salahuddin et al, 2022 ; Zech et al, 2018 ). To confirm that the hybrid-QC deep learning algorithm uses meaningful features of the diffusion-weighted images (DWI) to perform accurate QC, we used machine learning interpretation methods that pry open the “black box” of the neural network, thereby highlighting the features that lead to a specific QC score ( Sundararajan et al, 2017 ; Murdoch et al, 2019 ).…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An open, analysis-ready, and quality controlled resource for pediatric brain white-matter research

Richie-Halford

Cieslak

et al. 2022

Preprint

View full text Add to dashboard Cite

We created resources to facilitate research on the role of human brain microstructure in the development of mental health disorders, based on openly-available diffusion MRI (dMRI) data from the Healthy Brain Network (HBN) study. First, we curated the HBN dMRI data (N=2747) into the Brain Imaging Data Structure and preprocessed it according to best-practices, including denoising and correcting for motion effects, susceptibility-related distortions, and eddy currents. Preprocessed, analysis-ready data was made openly available. Data quality plays a key role in the analysis of dMRI, and we provide automated quality control (QC) scores for every scan, as part of the data release. To scale QC to this large dataset, we trained a neural network through the combination of a small data subset scored by experts and a larger set scored by community scientists. The network performs QC highly concordant with that of experts on a held out set (ROC-AUC=0.947). A further analysis of the neural network demonstrates that it relies on image features with relevance to QC. Altogether, this work both delivers a resource for transdiagnostic research in brain connectivity and pediatric mental health and serves as a novel tool for automated QC of large datasets.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An open, analysis-ready, and quality controlled resource for pediatric brain white-matter research

Richie-Halford

Cieslak

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…However, the projection of original samples in latent spaces with regularized dimensions for different attributes (see Figure 7) could be used as an interpretable attribute selection, identifying the ones better separating the analyzed classes such as the maximum 2D diameter of the myocardium and the LV volume attributes in our experiments. Further work will focus on fully integrating advanced feature selection techniques with the Attri-VAE model, as well as exploring alternative interpretability methods (see the recent review of Salahuddin et al (Salahuddin et al, 2022)) to better understand the role of clinical and imaging attributes on medical decisions in cardiovascular applications.…”

Section: Discussionmentioning

confidence: 99%

Attri-VAE: attribute-based, disentangled and interpretable representations of medical images with variational autoencoders

Çetin¹,

Cámara²,

Ballester³

2022

Preprint

View full text Add to dashboard Cite

Deep learning (DL) methods where interpretability is intrinsically considered as part of the model are required to better understand the relationship of clinical and imaging-based attributes with DL outcomes, thus facilitating their use in reasoning medical decisions. Latent space representations built with variational autoencoders (VAE) do not ensure individual control of data attributes. Attribute-based methods enforcing attribute disentanglement have been proposed in the literature for classical computer vision tasks in benchmark data. In this paper, we propose a VAE approach, the Attri-VAE, that includes an attribute regularization term to associate clinical and medical imaging attributes with different regularized dimensions in the generated latent space, enabling a better disentangled interpretation of the attributes. Furthermore, the generated attention maps explained the attribute encoding in the regularized latent space dimensions. The Attri-VAE approach analyzed healthy and myocardial infarction patients with clinical, cardiac morphology, and radiomics attributes. The proposed model provided an excellent trade-off between reconstruction fidelity, disentanglement, and interpretability, outperforming state-of-the-art VAE approaches according to several quantitative metrics. The resulting latent space allowed the generation of realistic synthetic data in the trajectory between two distinct input samples or along a specific attribute dimension to better interpret changes between different cardiac conditions.

show abstract

“…Recently, Salahuddin et al [107] review a set of interpretability methods which are grouped into nine different categories based on the type of explanations generated. They also discuss the problem of evaluating explanations and describe a set of evaluation strategies adopted to quantitative and qualitative measure the explanations' quality.…”

Section: General Reviews Tjoa and Guanmentioning

confidence: 99%

Explainable Deep Learning Methods in Medical Imaging Diagnosis: A Survey

Patrício¹,

Neves²,

Teixeira³

2022

Preprint

View full text Add to dashboard Cite

The remarkable success of deep learning has prompted interest in its application to medical diagnosis. Even tough state-ofthe-art deep learning models have achieved human-level accuracy on the classification of different types of medical data, these models are hardly adopted in clinical workflows, mainly due to their lack of interpretability. The black-box-ness of deep learning models has raised the need for devising strategies to explain the decision process of these models, leading to the creation of the topic of eXplainable Artificial Intelligence (XAI). In this context, we provide a thorough survey of XAI applied to medical diagnosis, including visual, textual, and example-based explanation methods. Moreover, this work reviews the existing medical imaging datasets and the existing metrics for evaluating the quality of the explanations . Complementary to most existing surveys, we include a performance comparison among a set of report generation-based methods. Finally, the major challenges in applying XAI to medical imaging are also discussed.CCS Concepts: • Applied computing → Health care information systems.

show abstract

Transparency of deep neural networks for medical image analysis: A review of interpretability methods

Cited by 210 publications

References 63 publications

An open, analysis-ready, and quality controlled resource for pediatric brain white-matter research

An open, analysis-ready, and quality controlled resource for pediatric brain white-matter research

Attri-VAE: attribute-based, disentangled and interpretable representations of medical images with variational autoencoders

Explainable Deep Learning Methods in Medical Imaging Diagnosis: A Survey

Contact Info

Product

Resources

About