Clinician's guide to trustworthy and responsible artificial intelligence in cardiovascular imaging

Szabó, L; Raisi‐Estabragh, Zahra; Salih, Ahmed; McCracken, Celeste; Pujadas, Esmeralda Ruiz; Gkontra, Polyxeni; Kiss, Máté; Maurovich-Horvath, Pal; Vágó, Hajnalka; Merkely, Béla; Lee, Aaron M.; Lekadir, Karim; Petersen, Steffen E.

doi:10.3389/fcvm.2022.1016032

Cited by 13 publications

(5 citation statements)

References 114 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is also the wider issue of AI trustworthiness which is prevalent within cardiovascular imaging. Our model has some mitigating features as highlighted by Szabo et al (27) such as the inclusion of multicentre data in the initial training and creating results that are explainable. This study also adds to this by demonstrating that it performs consistently in a real-world patient population without any exhibition of bias.…”

Section: Discussionmentioning

confidence: 99%

Clinical assessment of an AI tool for measuring biventricular parameters on cardiac MR

Salehi,

Maiter,

Strickland

et al. 2024

Front. Cardiovasc. Med.

View full text Add to dashboard Cite

IntroductionCardiac magnetic resonance (CMR) is of diagnostic and prognostic value in a range of cardiopulmonary conditions. Current methods for evaluating CMR studies are laborious and time-consuming, contributing to delays for patients. As the demand for CMR increases, there is a growing need to automate this process. The application of artificial intelligence (AI) to CMR is promising, but the evaluation of these tools in clinical practice has been limited. This study assessed the clinical viability of an automatic tool for measuring cardiac volumes on CMR.MethodsConsecutive patients who underwent CMR for any indication between January 2022 and October 2022 at a single tertiary centre were included prospectively. For each case, short-axis CMR images were segmented by the AI tool and manually to yield volume, mass and ejection fraction measurements for both ventricles. Automated and manual measurements were compared for agreement and the quality of the automated contours was assessed visually by cardiac radiologists.Results462 CMR studies were included. No statistically significant difference was demonstrated between any automated and manual measurements (p > 0.05; independent T-test). Intraclass correlation coefficient and Bland-Altman analysis showed excellent agreement across all metrics (ICC > 0.85). The automated contours were evaluated visually in 251 cases, with agreement or minor disagreement in 229 cases (91.2%) and failed segmentation in only a single case (0.4%). The AI tool was able to provide automated contours in under 90 s.ConclusionsAutomated segmentation of both ventricles on CMR by an automatic tool shows excellent agreement with manual segmentation performed by CMR experts in a retrospective real-world clinical cohort. Implementation of the tool could improve the efficiency of CMR reporting and reduce delays between imaging and diagnosis.

show abstract

Section: Discussionmentioning

confidence: 99%

Clinical assessment of an AI tool for measuring biventricular parameters on cardiac MR

Salehi,

Maiter,

Strickland

et al. 2024

Front. Cardiovasc. Med.

View full text Add to dashboard Cite

show abstract

“…Examples include incorrect self-diagnosis and treatment, delayed seeking of medical help, potential disease transmission, and undermining trust in healthcare professionals and health institutions [1,6,25]. Thus, ensuring the generation of correct, reliable, and credible medical information is of high importance and should be considered by AI model developers, considering the current evidence showing a generation of inaccurate information by these AI-based models [26][27][28]. Additionally, such an approach is recommended in various health domains given the intricacies and peculiarities of each subject (e.g., maxillofacial surgery, dentistry, and pharmacy) [29][30][31][32].…”

Section: Discussionmentioning

confidence: 99%

Pilot Testing of a Tool to Standardize the Assessment of the Quality of Health Information Generated by Artificial Intelligence-Based Models

Sallam,

Barakat,

Sallam

2023

Cureus

View full text Add to dashboard Cite

BackgroundArtificial intelligence (AI)-based conversational models, such as Chat Generative Pre-trained Transformer (ChatGPT), Microsoft Bing, and Google Bard, have emerged as valuable sources of health information for lay individuals. However, the accuracy of the information provided by these AI models remains a significant concern. This pilot study aimed to test a new tool with key themes for inclusion as follows: Completeness of content, Lack of false information in the content, Evidence supporting the content, Appropriateness of the content, and Relevance, referred to as "CLEAR", designed to assess the quality of health information delivered by AI-based models. MethodsTool development involved a literature review on health information quality, followed by the initial establishment of the CLEAR tool, which comprised five items that aimed to assess the following: completeness, lack of false information, evidence support, appropriateness, and relevance. Each item was scored on a five-point Likert scale from excellent to poor. Content validity was checked by expert review. Pilot testing involved 32 healthcare professionals using the CLEAR tool to assess content on eight different health topics deliberately designed with varying qualities. The internal consistency was checked with Cronbach's alpha (α). Feedback from the pilot test resulted in language modifications to improve the clarity of the items. The final CLEAR tool was used to assess the quality of health information generated by four distinct AI models on five health topics. The AI models were ChatGPT 3.5, ChatGPT 4, Microsoft Bing, and Google Bard, and the content generated was scored by two independent raters with Cohen's kappa (κ) for inter-rater agreement. ResultsThe final five CLEAR items were: (1) Is the content sufficient?; (2) Is the content accurate?; (3) Is the content evidence-based?; (4) Is the content clear, concise, and easy to understand?; and (5) Is the content free from irrelevant information? Pilot testing on the eight health topics revealed acceptable internal consistency with a Cronbach's α range of 0.669-0.981. The use of the final CLEAR tool yielded the following average scores: Microsoft Bing (mean=24.4±0.42), Google Bard (mean=21.2±1.79),. The inter-rater agreement revealed the following Cohen κ values: for ChatGPT-3.5 (κ=0.875, P<.001), ChatGPT-4 (κ=0.780, P<.001), Microsoft Bing (κ=0.348, P=.037), and Google Bard (κ=.749, P<.001). ConclusionsThe CLEAR tool is a brief yet helpful tool that can aid in standardizing testing of the quality of health information generated by AI-based models. Future studies are recommended to validate the utility of the CLEAR tool in the quality assessment of AI-generated health-related content using a larger sample across various complex health topics.

show abstract

“…Examples include incorrect self-diagnosis and treatment, delayed seeking of medical help, potential disease transmission, and undermining trust in healthcare professionals and health institutions [1,6,25]. Thus, ensuring the generation of correct, reliable, and credible health information is of high importance and should be considered by AI-models' developers considering the current evidence showing the generation of inaccurate information by these AI-based models [26,27].…”

Section: Discussionmentioning

confidence: 99%

CLEAR: Pilot Testing of a Tool to Standardize Assessment of the Quality of Health Information Generated by Artificial Intelligence-Based Models

Sallam,

Barakat,

Sallam

2023

Preprint

View full text Add to dashboard Cite

BackgroundArtificial intelligence (AI)-based conversational models, such as ChatGPT, Microsoft Bing, and Google Bard, emerged as valuable sources of health information for the lay individuals. However, the accuracy of information provided by these AI models remains a significant concern. This pilot study aimed to test a new tool referred to as “CLEAR”, designed to assess the quality of health information delivered by AI-based models.MethodsTool development involved a literature review on health information quality, followed by initial establishment of the CLEAR tool comprising five items that aimed to assess the following: completeness of content in response to the prompt, lack of false information, evidence support, appropriateness, and relevance of the generated content. Each item was scored on a 5-point Likert scale from excellent to poor. Content validity was checked by expert review of the initial items. Pilot testing involved 32 healthcare professionals using the CLEAR tool to assess content on eight different health topics deliberately designed with varying qualities. The internal consistency was checked using the Cronbach α. Feedback through the pilot test resulted in language modifications to improve the clarity of the items. The final CLEAR tool was used to assess health information quality generated through four different AI-based models in five different, yet common health topics. The AI models were ChatGPT 3.5, ChatGPT 4, Bing, and Bard, and the content generated was scored by two independent raters with Cohen κ to assess the inter-rater agreement.ResultsThe final five CLEAR items were: (1) Is the content sufficient? (2) Is the content accurate? (3) Is the content evidence-based? (4) Is the content clear, concise, and easy to understand? and (5) Is the content free from irrelevant information? Pilot testing using the eight different health topics revealed an acceptable internal consistency with a Cronbach α range of 0.669–0.981. The use of the final CLEAR tool yielded the following average scores: Bing (mean=24.4±0.42), ChatGPT-4 (mean=23.6±0.96), Bard (mean=21.2±1.79), and ChatGPT-3.5 (mean=20.6±5.20). The inter-rater agreement revealed the following Cohen κ values: for ChatGPT-3.5 (κ=0.875, P<.001), ChatGPT-4 (κ=0.780, P<.001), Bing (κ=0.348, P=.037), and Bard (κ=.749, P<.001).ConclusionsThe CLEAR tool is a brief yet helpful tool that can aid to standardize testing of the quality of health information generated by the AI-based conversational models. Future studies are recommended to validate the utility of the CLEAR tool to assess the quality of the AI-generated health-related content using a larger sample across various complex health topics.

show abstract

Clinician's guide to trustworthy and responsible artificial intelligence in cardiovascular imaging

Cited by 13 publications

References 114 publications

Clinical assessment of an AI tool for measuring biventricular parameters on cardiac MR

Clinical assessment of an AI tool for measuring biventricular parameters on cardiac MR

Pilot Testing of a Tool to Standardize the Assessment of the Quality of Health Information Generated by Artificial Intelligence-Based Models

CLEAR: Pilot Testing of a Tool to Standardize Assessment of the Quality of Health Information Generated by Artificial Intelligence-Based Models

Contact Info

Product

Resources

About