Non-task expert physicians benefit from correct explainable AI advice when reviewing X-rays

Gaube, Susanne; Suresh, Harini; Raue, Martina; Lermer, Eva; Koch, T.; Hudecek, Matthias F. C.; Ackery, Alun; Grover, Samir C.; Coughlin, Joseph F.; Frey, Dieter; Kitamura, Felipe; Ghassemi, Marzyeh; Colak, Errol

doi:10.1038/s41598-023-28633-w

Cited by 32 publications

(23 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is especially relevant for translating learning-based computer-aided diagnostic/screening systems to routine clinical care. 15 Particularly, multi-label classification tasks, where the co-occurrence of disease makes deep learning models more vulnerable to reliance on confounding factors. Typical approaches to multi-label image classification explainability, including GradCAM, 16 have been criticised for their inability to highlight smaller pathologies or structures with complex shapes, for example, mechanical wiring.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Learning disentangled representations for explainable chest x-ray classification using Dirichlet VAEs

Harkness

Frangi

Zucker

et al. 2023

Medical Imaging 2023: Image Processing

View full text Add to dashboard Cite

This study explores the use of the Dirichlet Variational Autoencoder (DirVAE) for learning disentangled latent representations of chest X-ray (CXR) images. Our working hypothesis is that distributional sparsity, as facilitated by the Dirichlet prior, will encourage disentangled feature learning for the complex task of multi-label classification of CXR images. The DirVAE is trained using CXR images from the CheXpert database, and the predictive capacity of multi-modal latent representations learned by DirVAE models is investigated through implementation of an auxiliary multi-label classification task, with a view to enforce separation of latent factors according to class-specific features. The predictive performance and explainability of the latent space learned using the DirVAE were quantitatively and qualitatively assessed, respectively, and compared with a standard Gaussian prior-VAE (GVAE). We introduce a new approach for explainable multi-label classification in which we conduct gradient-guided latent traversals for each class of interest. Study findings indicate that the DirVAE is able to disentangle latent factors into class-specific visual features, a property not afforded by the GVAE, and achieve a marginal increase in predictive performance relative to GVAE. We generate visual examples to show that our explainability method, when applied to the trained DirVAE, is able to highlight regions in CXR images that are clinically relevant to the class(es) of interest and additionally, can identify cases where classification relies on spurious feature correlations.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Figure 2: 3-dimensional probability density functions from the Dirichlet distribution, each with different α values.The values of α are set to (0.5, 0.5, 0.5), (0.9, 0.9, 0.9) and(2,5,15).…”

mentioning

confidence: 99%

Learning disentangled representations for explainable chest x-ray classification using Dirichlet VAEs

Harkness

Frangi

Zucker

et al. 2023

Medical Imaging 2023: Image Processing

View full text Add to dashboard Cite

show abstract

“…In all experiments, the three dependent variables were based on established measures (see Gaube et al, 2021Gaube et al, , 2023.…”

Section: Dependent Variablesmentioning

confidence: 99%

“…While some previous studies found that providing explanations can positively affect performance in a decision-making task (Gaube et al, 2023;Pessach et al, 2020), others did not (e.g., van der Waa et al, 2021. More research regarding this discrepancy is needed to investigate the role of explainable AI advice in decision-making and how explanations ought to be presented.…”

Section: Introductionmentioning

confidence: 95%

“…Receiving correct advice leads to higher trust in the advice and improved performance compared to incorrect advice (Gaube et al, 2021(Gaube et al, , 2023Goodyear et al, 2016;Strickland et al, 2021). Yet, research has also shown that recipients often blindly follow advice, even when it is incorrect (Gaube et al, 2021;Gino et al, 2012;Schultze et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Effect of AI-generated Advice on Decision-Making in Personnel Selection

Cecil¹,

Lermer²,

Hudecek³

et al. 2023

Preprint

View full text Add to dashboard Cite

Despite the rise of decision support systems enabled by artificial intelligence (AI) in personnel selection, their impact on decision-making processes is largely unknown. Consequently, we conducted five experiments (N = 1,403) investigating how people interact with AI-generated advice in a personnel selection task. In all pre-registered experiments, we presented correct and incorrect advice. In Experiments 1a and 1b, we manipulated the source of the advice (human vs. AI). In Experiments 2a, 2b, and 2c, we further manipulated the type of explainability of AI advice (2a and 2b: heatmaps and 2c: charts) to test if explainable advice improves decision-making. The independent variables were regressed on task performance, perceived advice quality and confidence ratings. The results consistently showed that incorrect advice negatively impacted performance, as people failed to dismiss it (i.e., overreliance). Additionally, we found that the effects of source and explainability of advice on the dependent variables were limited.

show abstract

Interaction between clinicians and artificial intelligence to detect fetal atrioventricular septal defects on ultrasound: how can we optimize collaborative performance?

Day,

Matthew,

Budd

et al. 2024

Ultrasound in Obstet & Gyne

View full text Add to dashboard Cite

ObjectivesArtificial intelligence (AI) has shown promise in improving the performance of fetal ultrasound screening in detecting congenital heart disease (CHD). The effect of giving AI advice to human operators has not been studied in this context. Giving additional information about AI model workings, such as confidence scores for AI predictions, may be a way of improving performance further. Our aims were to investigate whether AI advice improved overall diagnostic accuracy (using a single CHD lesion as an exemplar), and to see what, if any, additional information given to clinicians optimized the overall performance of the clinician‐AI team.MethodsAn AI model was trained to classify a single fetal CHD lesion (atrioventricular septal defect, AVSD), using a retrospective cohort of 121,130 cardiac four chamber images extracted from 173 ultrasound scan videos (98 with normal hearts, 75 with AVSD). A ResNet50 model architecture was used. Temperature scaling of model prediction probability was performed on a validation set, and gradient‐weighted class activation maps (grad‐CAMs) produced. Ten clinicians (two consultant fetal cardiologists, three trainees in pediatric cardiology, and five fetal cardiac sonographers) were recruited from a center of fetal cardiology to participate. Each participant was shown 2000 fetal four chamber images in a random order (1,000 normal and 1,000 AVSD). The dataset was comprised of 500 images, each shown in four conditions: 1) image alone without AI output; 2) image with binary AI classification; 3) image with AI model confidence; 4) image with gradient‐weighted class activation map image overlays. The clinicians were asked to classify each image as normal or AVSD.Results20,000 image classifications were recorded from 10 clinicians. The AI model alone achieved an accuracy of 0.798 (95% CI 0.760 – 0.832), sensitivity of 0.868 (95% CI 0.834 – 0.902) and specificity of 0.728 (95% CI 0.702 – 0.754, and the clinicians without AI achieved an accuracy of 0.844 (95% CI 0.834 – 0.854), sensitivity of 0.827 (95% CI 0.795 – 0.858) and specificity of 0.861 (95% CI 0.828 – 0.895). Showing a binary (normal or AVSD) AI model output resulted in significant improvement in accuracy to 0.865 (p <0.001). This effect was seen in both experienced and less experienced participants. Giving incorrect AI advice resulted in significant deterioration in overall accuracy from 0.761 to 0.693 (p <0.001), which was driven by an increase in both type I and type II error by the clinicians. This effect was worsened by showing model confidence (accuracy 0.649, p <0.001) or grad‐CAM (accuracy 0.644, p <0.001).ConclusionsAI has the potential to improve performance when used in collaboration with clinicians, even if the model performance does not reach expert level. Giving additional information about model workings such as model confidence and class activation map image overlays did not improve overall performance, and actually worsened performance for images where the AI model was incorrect.This article is protected by copyright. All rights reserved.

show abstract

Non-task expert physicians benefit from correct explainable AI advice when reviewing X-rays

Cited by 32 publications

References 26 publications

Learning disentangled representations for explainable chest x-ray classification using Dirichlet VAEs

Learning disentangled representations for explainable chest x-ray classification using Dirichlet VAEs

The Effect of AI-generated Advice on Decision-Making in Personnel Selection

Interaction between clinicians and artificial intelligence to detect fetal atrioventricular septal defects on ultrasound: how can we optimize collaborative performance?

Contact Info

Product

Resources

About