Diagnostic and Management Performance of ChatGPT in Obstetrics and Gynecology

Allahqoli, Leila; Ghiasvand, Mohammad Matin; Mazidimoradi, Afrooz; Salehiniya, Hamid; Alkatout, İbrahim

doi:10.1159/000533177

Cited by 14 publications

(10 citation statements)

References 15 publications

(15 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although some research suggests that ChatGPT needs to be specifically trained in the medical field to be effective ( 3 ). a substantial body of research has already demonstrated the excellent capabilities of ChatGPT in the medical field ( 4 , 5 ).…”

mentioning

confidence: 99%

Commentary: AI-based online chat and the future of oncology care: a promising technology or a solution in search of a problem?

Zhang,

Guan,

Chen

et al. 2023

Front. Oncol.

View full text Add to dashboard Cite

mentioning

confidence: 99%

Commentary: AI-based online chat and the future of oncology care: a promising technology or a solution in search of a problem?

Zhang,

Guan,

Chen

et al. 2023

Front. Oncol.

View full text Add to dashboard Cite

“…After screening, 54 studies were included in the metaanalysis of generative AI diagnostic performance 12,13, and 13 studies in the comparative analysis between generative AI models and physicians (Figure 1 and Table 1). 32,33,[35][36][37][38][39][40][41]49,52,58,62 The most evaluated models were GPT-4 4 (31 articles) and GPT-3.5 (28), while models such as GPT-4V ( 6), PaLM2 8 (3), Llama 2 6 (2), Prometheus (2), GPT-3 3 (1), Glass ( 1 were represented, as well as Gastroenterology, Cardiology, Pediatrics, Otolaryngology, Urology, Endocrinology, Gynecology, Orthopedic surgery, Rheumatology, Psychiatry, and Plastic surgery with one article each. Regarding model tasks, free text tasks were the most common, with 47 articles, followed by choice tasks at 13.…”

Section: Study Selection and Characteristicsmentioning

confidence: 99%

Diagnostic Performance Comparison between Generative AI and Physicians: A Systematic Review and Meta-Analysis

Takita,

Kabata,

Walston

et al. 2024

Preprint

View full text Add to dashboard Cite

Background: The rapid advancement of generative artificial intelligence (AI) has revolutionized understanding and generation of human language. Their integration into healthcare has shown potential for improving medical diagnostics, yet a comprehensive diagnostic performance evaluation of generative AI models and the comparison of their diagnostic performance with that of physicians has not been extensively explored. Methods: In this systematic review and meta-analysis, a comprehensive search of Medline, Scopus, Web of Science, Cochrane Central, and medRxiv was conducted for studies published from June 2018 through December 2023, focusing on those that validate generative AI models for diagnostic tasks. Meta-analysis was performed to summarize the performance of the models and to compare the accuracy of the models with that of physicians. The quality of studies was assessed using the Prediction Model Study Risk of Bias Assessment Tool. Results: The search resulted in 54 studies being included in the meta-analysis, with 13 of these also used in the comparative analysis. Eight models were evaluated across 17 medical specialties. The overall accuracy for generative AI models across 54 studies was 57% (95% confidence interval [CI]: 51–63%). The I-squared statistic of 96% signifies a high degree of heterogeneity among the study results. Meta-regression analysis of generative AI models revealed significantly improved accuracy for GPT-4, and reduced accuracy for some specialties such as Neurology, Endocrinology, Rheumatology, and Radiology. The comparison meta-analysis demonstrated that, on average, physicians exceeded the accuracy of the models (difference in accuracy: 14% [95% CI: 8–19%], p-value <0.001). However, in the performance comparison between GPT-4 and physicians, GPT-4 performed slightly higher than non-experts (-4% [95% CI: -10–2%], p-value = 0.173), and slightly underperformed compared to experts (6% [95% CI: -1–13%], p-value = 0.091). The quality assessment indicated a high risk of bias in the majority of studies, primarily due to small sample sizes. Conclusions: Generative AI exhibits promising diagnostic capabilities, with accuracy varying significantly by model and medical specialty. Although they have not reached the reliability of expert physicians, the findings suggest that generative AI models have the potential to enhance healthcare delivery and medical education, provided they are integrated with caution and their limitations are well-understood. This study also highlights the need for more rigorous research standards and a larger number of cases in the future.

show abstract

“…Allahqoli et al [ 16 ] reported a series of cases of OBGYN that were presented to ChatGPT, and their diagnostic and management performances were compared with those of a gynecologist. The results of 30 cases were provided, including early pregnancy, general gynecology, emergency gynecology obstetrics, peripartum care, obstetric emergencies, family planning, and sexual health, adapted from the book titled ‘100 cases in Obstetrics and Gynecology’.…”

Section: Chatgpt In Obgyn: Diagnostic Advancementsmentioning

confidence: 99%

Potential applications of ChatGPT in obstetrics and gynecology in Korea: a review article

Lee,

Kim

2024

Obstet Gynecol Sci

View full text Add to dashboard Cite

The use of chatbot technology, particularly chat generative pre-trained transformer (ChatGPT) with an impressive 175 billion parameters, has garnered significant attention across various domains, including Obstetrics and Gynecology (OBGYN). This comprehensive review delves into the transformative potential of chatbots with a special focus on ChatGPT as a leading artificial intelligence (AI) technology. Moreover, ChatGPT harnesses the power of deep learning algorithms to generate responses that closely mimic human language, opening up myriad applications in medicine, research, and education. In the field of medicine, ChatGPT plays a pivotal role in diagnosis, treatment, and personalized patient education. Notably, the technology has demonstrated remarkable capabilities, surpassing human performance in OBGYN examinations, and delivering highly accurate diagnoses. However, challenges remain, including the need to verify the accuracy of the information and address the ethical considerations and limitations. In the wide scope of chatbot technology, AI systems play a vital role in healthcare processes, including documentation, diagnosis, research, and education. Although promising, the limitations and occasional inaccuracies require validation by healthcare professionals. This review also examined global chatbot adoption in healthcare, emphasizing the need for user awareness to ensure patient safety. Chatbot technology holds great promise in OBGYN and medicine, offering innovative solutions while necessitating responsible integration to ensure patient care and safety.

show abstract

Diagnostic and Management Performance of ChatGPT in Obstetrics and Gynecology

Cited by 14 publications

References 15 publications

Commentary: AI-based online chat and the future of oncology care: a promising technology or a solution in search of a problem?

Commentary: AI-based online chat and the future of oncology care: a promising technology or a solution in search of a problem?

Diagnostic Performance Comparison between Generative AI and Physicians: A Systematic Review and Meta-Analysis

Potential applications of ChatGPT in obstetrics and gynecology in Korea: a review article

Contact Info

Product

Resources

About