Ability of ChatGPT to generate competent radiology reports for distal radius fracture by use of RSNA template items and integrated AO classifier

Bosbach, Wolfram A.; Senge, Jan F.; Németh, Bence; Omar, Siti H.; Mitrakovic, Milena; Beisbart, Claus; Horváth, András; Heverhagen, Johannes T.; Daneshvar, Keivan

doi:10.1067/j.cpradiol.2023.04.001

Cited by 16 publications

(25 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One relevant study has demonstrated limitations concerning accuracy of recommendations for IR procedures [15]. This result is similar to what we found in a previous study about the handling of technical and medical information in report drafting for distal radius fracture [16], [17]. For evaluating the ability of ChatGPT to handle the RSNA PICC line template [2], we defined 3 distinct cases and iterated those for a parameter study (n = 5).…”

Section: Introductionsupporting

confidence: 77%

“…The methodology of the presented study follows the concept of the previous work [16]: cases were defined within the framework of a current RSNA template. ChatGPT was tasked with report drafting.…”

Section: Methodsmentioning

confidence: 99%

“…The command was set to "Write a radiology report which contains this exact information:". No instruction was given on text structure, unlike before in [16]. The returned outputs were saved as txtfiles.…”

Section: Chatgpt Parameter Studymentioning

confidence: 99%

“…The returned outputs were saved as txtfiles. The previous study on distal radius fracture report drafting [16] relied on an earlier version of ChatGPT [18].…”

Section: Chatgpt Parameter Studymentioning

confidence: 99%

See 3 more Smart Citations

ChatGPT may free time needed by the interventional radiologist for administration / documentation: A study on the RSNA PICC line reporting template

Senge,

McMurray,

Haupt

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

Motive: Documentation and administration, unpleasant necessities, take a substantial part of the working time in the subspecialty of interventional radiology. With increasing future demand for clinical radiology predicted, time savings from use of text drafting technologies could be a valuable contribution towards our field. Method: Three cases of peripherally inserted central catheter (PICC) line insertion were defined for the present study. The current version of ChatGPT was tasked with drafting reports, following the Radiological Society of North America (RSNA) template. Key results: Score card evaluation by human radiologists indicates that time savings in documentation / administration can be expected without loss of quality from using ChatGPT. Further, automatically generated texts were not assessed to be clearly identifiable as AI-produced. Conclusions: Patients, doctors, and hospital administrators would welcome a reduction of the time that interventional radiologists need for documentation and administration these days. If AI-tools as tested in the present study are brought into clinical application, questions about trust into those systems eg with regard to medical complications will have to be addressed.

show abstract

Section: Introductionsupporting

confidence: 77%

Section: Methodsmentioning

confidence: 99%

Section: Chatgpt Parameter Studymentioning

confidence: 99%

“…The returned outputs were saved as txtfiles. The previous study on distal radius fracture report drafting [16] relied on an earlier version of ChatGPT [18].…”

Section: Chatgpt Parameter Studymentioning

confidence: 99%

See 2 more Smart Citations

ChatGPT may free time needed by the interventional radiologist for administration / documentation: A study on the RSNA PICC line reporting template

Senge,

McMurray,

Haupt

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Accuracy. Several studies highlighted that ChatGPT exhibited inaccuracies when asked to respond to certain questions [14,18,23,29,32,34,35,38,43,50,52,53,64,65,67,71,72]. For instance, ChatGPT could respond with incomplete information or exhibit an inability to distinguish between truth and falsehood [21,69].…”

Section: Reliabilitymentioning

confidence: 99%

A Systematic Review of ChatGPT and Other Conversational Large Language Models in Healthcare

Wang,

Wan,

et al. 2024

Preprint

View full text Add to dashboard Cite

Background: The launch of the Chat Generative Pre-trained Transformer (ChatGPT) in November 2022 has attracted public attention and academic interest to large language models (LLMs), facilitating the emergence of many other innovative LLMs. These LLMs have been applied in various fields, including healthcare. Numerous studies have since been conducted regarding how to employ state-of-the-art LLMs in health-related scenarios to assist patients, doctors, and public health administrators. Objective: This review aims to summarize the applications and concerns of applying conversational LLMs in healthcare and provide an agenda for future research on LLMs in healthcare. Methods: We utilized PubMed, ACM, and IEEE digital libraries as primary sources for this review. We followed the guidance of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRIMSA) to screen and select peer-reviewed research articles that (1) were related to both healthcare applications and conversational LLMs and (2) were published before September 1st, 2023, the date when we started paper collection and screening. We investigated these papers and classified them according to their applications and concerns. Results: Our search initially identified 820 papers according to targeted keywords, out of which 65 papers met our criteria and were included in the review. The most popular conversational LLM was ChatGPT from OpenAI (60), followed by Bard from Google (1), Large Language Model Meta AI (LLaMA) from Meta (1), and other LLMs (5). These papers were classified into four categories in terms of their applications: 1) summarization, 2) medical knowledge inquiry, 3) prediction, and 4) administration, and four categories of concerns: 1) reliability, 2) bias, 3) privacy, and 4) public acceptability. There are 49 (75%) research papers using LLMs for summarization and/or medical knowledge inquiry, and 58 (89%) research papers expressing concerns about reliability and/or bias. We found that conversational LLMs exhibit promising results in summarization and providing medical knowledge to patients with a relatively high accuracy. However, conversational LLMs like ChatGPT are not able to provide reliable answers to complex health-related tasks that require specialized domain expertise. Additionally, no experiments in our reviewed papers have been conducted to thoughtfully examine how conversational LLMs lead to bias or privacy issues in healthcare research. Conclusions: Future studies should focus on improving the reliability of LLM applications in complex health-related tasks, as well as investigating the mechanisms of how LLM applications brought bias and privacy issues. Considering the vast accessibility of LLMs, legal, social, and technical efforts are all needed to address concerns about LLMs to promote, improve, and regularize the application of LLMs in healthcare.

show abstract

Large language models’ responses to liver cancer surveillance, diagnosis, and management questions: accuracy, reliability, readability

Cao,

Kwon,

Ghaziani

et al. 2024

Abdom Radiol

View full text Add to dashboard Cite

Ability of ChatGPT to generate competent radiology reports for distal radius fracture by use of RSNA template items and integrated AO classifier

Cited by 16 publications

References 10 publications

ChatGPT may free time needed by the interventional radiologist for administration / documentation: A study on the RSNA PICC line reporting template

ChatGPT may free time needed by the interventional radiologist for administration / documentation: A study on the RSNA PICC line reporting template

A Systematic Review of ChatGPT and Other Conversational Large Language Models in Healthcare

Large language models’ responses to liver cancer surveillance, diagnosis, and management questions: accuracy, reliability, readability

Contact Info

Product

Resources

About