Few-shot Structured Radiology Report Generation Using Natural Language Prompts

Keicher, Matthias; Mullakaeva, Kamilia; Czempiel, Tobias; Mach, Kristina; Khakzar, Ashkan; Navab, Nassir

doi:10.48550/arxiv.2203.15723

Cited by 1 publication

(1 citation statement)

References 23 publications

(29 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…settings. Recently, zero-shot [4,1,14,15,17] and few-shot [4,1,8] learning methods have been proposed as a potential solution, utilizing contrastive pretraining [19,13] on pairs of radiology reports and images, and achieving performance on par with radiologists [15]. However, these methods lack the level of detail of radiology reports and inherent explainability, impeding their adoption in clinical settings [7].…”

Section: Introductionmentioning

confidence: 99%

Xplainer: From X-Ray Observations to Explainable Zero-Shot Diagnosis

Pellegrini¹,

Keicher²,

Özsoy³

et al. 2023

Preprint

View full text Add to dashboard Cite

Automated diagnosis prediction from medical images is a valuable resource to support clinical decision-making. However, such systems usually need to be trained on large amounts of annotated data, which often is scarce in the medical domain. Zero-shot methods address this challenge by allowing a flexible adaption to new settings with different clinical findings without relying on labeled data. Further, to integrate automated diagnosis in the clinical workflow, methods should be transparent and explainable, increasing medical professionals' trust and facilitating correctness verification. In this work, we introduce Xplainer, a novel framework for explainable zero-shot diagnosis in the clinical setting. Xplainer adapts the classification-by-description approach of contrastive vision-language models to the multi-label medical diagnosis task. Specifically, instead of directly predicting a diagnosis, we prompt the model to classify the existence of descriptive observations, which a radiologist would look for on an X-Ray scan, and use the descriptor probabilities to estimate the likelihood of a diagnosis. Our model is explainable by design, as the final diagnosis prediction is directly based on the prediction of the underlying descriptors. We evaluate Xplainer on two chest X-ray datasets, CheXpert and ChestX-ray14, and demonstrate its effectiveness in improving the performance and explainability of zero-shot diagnosis. Our results suggest that Xplainer provides a more detailed understanding of the decision-making process and can be a valuable tool for clinical diagnosis.

show abstract