A Survey on Deep Learning and Explainability for Automatic Report Generation from Medical Images

Messina, Pablo; Pino, Pablo del; Parra, Denis; Soto, Álvaro; Besa, Cecilia; Uribe, Sergio; andía, Marcelo; Tejos, Cristián; Prieto, Claudia; Capurro, Daniel

doi:10.48550/arxiv.2010.10563

Cited by 2 publications

(3 citation statements)

References 86 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast to previous sections that discuss applications of ViTs, this section focuses on transformers as powerful language models. It is also pertinent to note that even though multiple surveys exist covering the applications of deep learning in clinical report generation [335]- [338], to the best of our knowledge, none of these have covered the applications of transformer models in the area despite having transformers' phenomenal impact since their inception back in 2017. In this regard, we hope this section will serve as a valuable resource to the research community.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Transformers in Medical Imaging: A Survey

Shamshad¹,

Khan²,

Zamir³

et al. 2022

Preprint

View full text Add to dashboard Cite

Following unprecedented success on the natural language tasks, Transformers have been successfully applied to several computer vision problems, achieving state-of-the-art results and prompting researchers to reconsider the supremacy of convolutional neural networks (CNNs) as de facto operators. Capitalizing on these advances in computer vision, the medical imaging field has also witnessed growing interest for Transformers that can capture global context compared to CNNs with local receptive fields. Inspired from this transition, in this survey, we attempt to provide a comprehensive review of the applications of Transformers in medical imaging covering various aspects, ranging from recently proposed architectural designs to unsolved issues. Specifically, we survey the use of Transformers in medical image segmentation, detection, classification, reconstruction, synthesis, registration, clinical report generation, and other tasks. In particular, for each of these applications, we develop taxonomy, identify application-specific challenges as well as provide insights to solve them, and highlight recent trends. Further, we provide a critical discussion of the field's current state as a whole, including the identification of key challenges, open problems, and outlining promising future directions. We hope this survey will ignite further interest in the community and provide researchers with an up-to-date reference regarding applications of Transformer models in medical imaging. Finally, to cope with the rapid development in this field, we intend to regularly update the relevant latest papers and their open-source implementations at https://github.com/fahadshamshad/awesome-transformers-in-medical-imaging.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Recently, immense progress has been made to automatically generate clinical reports from medical images using deep learning [335]- [338]. This automatic report generation process can help clinicians in accurate decision-making.…”

Section: Clinical Report Generationmentioning

confidence: 99%

Transformers in Medical Imaging: A Survey

Shamshad¹,

Khan²,

Zamir³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…A number of studies have found an intimate connection between explainability and the safety of human life in security-critical areas, e.g., in medicine [9,17,2] or autonomous driving [25,8]. In the medical domain, decisions made by blackbox models are unreliable and thus unacceptable [28].…”

Section: Introductionmentioning

confidence: 99%

Surrogate Model-Based Explainability Methods for Point Cloud NNs

Tan¹,

Kotthaus²

2021

Preprint

View full text Add to dashboard Cite

In the field of autonomous driving and robotics, point clouds are showing their excellent real-time performance as raw data from most of the mainstream 3D sensors. Therefore, point cloud neural networks have become a popular research direction in recent years. So far, however, there has been little discussion about the explainability of deep neural networks for point clouds. In this paper, we propose new explainability approaches for point cloud deep neural networks based on local surrogate model-based methods to show which components make the main contribution to the classification. Moreover, we propose a quantitative validation method for explainability methods of point clouds which enhances the persuasive power of explainability by dropping the most positive or negative contributing features and monitoring how the classification scores of specific categories change. To enable an intuitive explanation of misclassified instances, we display features with confounding contributions. Our new explainability approach provides a fairly accurate, more intuitive and widely applicable explanation for point cloud classification tasks. Our code is available at https://github.com/Explain3D/Expla inable3D

show abstract

A Survey on Deep Learning and Explainability for Automatic Report Generation from Medical Images

Cited by 2 publications

References 86 publications

Transformers in Medical Imaging: A Survey

Transformers in Medical Imaging: A Survey

Surrogate Model-Based Explainability Methods for Point Cloud NNs

Contact Info

Product

Resources

About