Using AI-generated suggestions from ChatGPT to optimize clinical decision support

Liu, Siru; Wright, Aileen P.; Patterson, Barron L.; Wanderer, Jonathan P.; Turer, Robert W; Nelson, S.; McCoy, Allison B.; Sittig, Dean F.; Wright, Adam

doi:10.1093/jamia/ocad072

Cited by 163 publications

(84 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a study, [24] a total of 36 AI-generated suggestions and 29 human-generated suggestions for 7 alerts were analyzed by 5 clinicians. Among the top 20 suggestions based on the survey scores, 9 were generated by ChatGPT.…”

Section: What This Knowledge Adds To What We Already Knewmentioning

confidence: 99%

“…[21] This study seeks to determine ChatGPT grade C for questions in the field of pediatrics, similar results to previous studies. [12][13][14]20,21,24] The potential for ChatGPT and future virtual assistants become important tools for patients and healthcare providers based on assessments ranging from basic fact-based questions to complex clinical questions in a study. [55] Although ChatGPT was able to provide interpretable responses that reduced the likelihood of causing alarm compared to Google's feature snippet, there is an urgent need for regulators and healthcare professionals to be involved in developing standards for minimum quality and to educate patients about the limitations of emerging AI assistants.…”

Section: What This Knowledge Adds To What We Already Knewmentioning

confidence: 99%

“…This approach aligns with the fundamental theorem of medical informatics, which emphasizes the creation of computer systems that augment human intelligence, leading to improved performance when humans and computers work together compared to humans alone. [24]…”

Section: What This Knowledge Adds To What We Already Knewmentioning

confidence: 99%

See 2 more Smart Citations

Assessing ChatGPT’s capacity for clinical decision support in pediatrics: A comparative study with pediatricians using KIDMAP of Rasch analysis

Kao

Chien

Wang³

et al. 2023

Medicine

View full text Add to dashboard Cite

Background: The application of large language models in clinical decision support (CDS) is an area that warrants further investigation. ChatGPT, a prominent large language models developed by OpenAI, has shown promising performance across various domains. However, there is limited research evaluating its use specifically in pediatric clinical decision-making. This study aimed to assess ChatGPT’s potential as a CDS tool in pediatrics by evCDSaluating its performance on 8 common clinical symptom prompts. Study objectives were to answer the 2 research questions: the ChatGPT’s overall grade in a range from A (high) to E (low) compared to a normal sample and the difference in assessment of ChatGPT between 2 pediatricians. Methods: We compared ChatGPT’s responses to 8 items related to clinical symptoms commonly encountered by pediatricians. Two pediatricians independently assessed the answers provided by ChatGPT in an open-ended format. The scoring system ranged from 0 to 100, which was then transformed into 5 ordinal categories. We simulated 300 virtual students with a normal distribution to provide scores on items based on Rasch rating scale model and their difficulties in a range between −2 to 2.5 logits. Two visual presentations (Wright map and KIDMAP) were generated to answer the 2 research questions outlined in the objectives of the study. Results: The 2 pediatricians’ assessments indicated that ChatGPT’s overall performance corresponded to a grade of C in a range from A to E, with average scores of −0.89 logits and 0.90 logits (=log odds), respectively. The assessments revealed a significant difference in performance between the 2 pediatricians (P < .05), with scores of −0.89 (SE = 0.37) and 0.90 (SE = 0.41) in log odds units (logits in Rasch analysis). Conclusion: This study demonstrates the feasibility of utilizing ChatGPT as a CDS tool for patients presenting with common pediatric symptoms. The findings suggest that ChatGPT has the potential to enhance clinical workflow and aid in responsible clinical decision-making. Further exploration and refinement of ChatGPT’s capabilities in pediatric care can potentially contribute to improved healthcare outcomes and patient management.

show abstract

Section: What This Knowledge Adds To What We Already Knewmentioning

confidence: 99%

Section: What This Knowledge Adds To What We Already Knewmentioning

confidence: 99%

See 1 more Smart Citation

Assessing ChatGPT’s capacity for clinical decision support in pediatrics: A comparative study with pediatricians using KIDMAP of Rasch analysis

Kao

Chien

Wang³

et al. 2023

Medicine

View full text Add to dashboard Cite

show abstract

“…Similarly, several studies have evaluated the value of ChatGPT for clinical decision support optimization. Their results showed that ChatGPT could be an important complementary part of optimizing clinical decision support and may even be able to assist experts in formulating their own suggestions [ 13 , 14 ].…”

Section: Conclusion and Future Perspectivesmentioning

confidence: 99%

Artificial Intelligence in Intensive Care Medicine: Toward a ChatGPT/GPT-4 Way?

et al. 2023

Ann Biomed Eng

View full text Add to dashboard Cite

“…reasoning processes 4,5 . Beyond answering general questions, GAI has demonstrated effectiveness in specialized medical contexts 6 , including U.S. Medical Licensing Examination queries 7 , clinical decisionmaking consultations 8,9 , and medical research assessments 10,11 . Notably, ChatGPT is already being leveraged by biotechnology companies to suggest novel pathways for drug targets 12 .…”

mentioning

confidence: 99%

Leveraging Generative AI to Prioritize Drug Repurposing Candidates: Validating Identified Candidates for Alzheimer's Disease in Real-World Clinical Datasets

Wei

Yan

Grabowska

et al. 2023

Preprint

View full text Add to dashboard Cite

Drug repurposing represents an attractive alternative to the costly and time-consuming process of new drug development, particularly for serious, widespread conditions with limited effective treatments, such as Alzheimer’s disease (AD). Emerging generative artificial intelligence (GAI) technologies like ChatGPT offer the promise of expediting the review and summary of scientific knowledge. To examine the feasibility of using GAI for identifying drug repurposing candidates, we iteratively tasked ChatGPT with proposing the twenty most promising drugs for repurposing in AD, and tested the top ten for risk of incident AD in exposed and unexposed individuals over age 65 in two large clinical datasets: 1) Vanderbilt University Medical Center and 2) the All of Us Research Program. Among the candidates suggested by ChatGPT, metformin, simvastatin, and losartan were associated with lower AD risk in meta-analysis. These findings suggest GAI technologies can assimilate scientific insights from an extensive Internet-based search space, helping to prioritize drug repurposing candidates and facilitate the treatment of diseases.

show abstract

Using AI-generated suggestions from ChatGPT to optimize clinical decision support

Cited by 163 publications

References 30 publications

Assessing ChatGPT’s capacity for clinical decision support in pediatrics: A comparative study with pediatricians using KIDMAP of Rasch analysis

Assessing ChatGPT’s capacity for clinical decision support in pediatrics: A comparative study with pediatricians using KIDMAP of Rasch analysis

Artificial Intelligence in Intensive Care Medicine: Toward a ChatGPT/GPT-4 Way?

Leveraging Generative AI to Prioritize Drug Repurposing Candidates: Validating Identified Candidates for Alzheimer's Disease in Real-World Clinical Datasets

Contact Info

Product

Resources

About