Ultrasonographic and cytopathological characteristics of thyroid nodules in Graves' disease

Anıl, Cüneyd; Kirnap, Nazlı Gülsoy; İyidir, Özlem Turhan; Fırat, Sevde Nur; Tütüncü, Neslihan Başçıl

doi:10.1530/endoabs.37.ep992

Cited by 4 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another limitation is that we experimented on only one model, GPT‐4. However, since this is an early investigation and GPT‐4 is currently recognized as one of the strongest models, scoring highest on the Massive Multi‐task Language Understanding benchmark to date, we believe our study sufficiently sheds light on this potential application of large language models 16,33 . Lastly, as our reports were limited to Korean and English, our results may differ in the context of other languages, especially low‐resource languages in which GPT‐4 is not likely to be fluent.…”

Section: Discussionmentioning

confidence: 84%

Using GPT‐4 for LI‐RADS feature extraction and categorization with multilingual free‐text reports

Gu,

Lee,

Shin

et al. 2024

Liver International

View full text Add to dashboard Cite

Background and AimsThe Liver Imaging Reporting and Data System (LI‐RADS) offers a standardized approach for imaging hepatocellular carcinoma. However, the diverse styles and structures of radiology reports complicate automatic data extraction. Large language models hold the potential for structured data extraction from free‐text reports. Our objective was to evaluate the performance of Generative Pre‐trained Transformer (GPT)‐4 in extracting LI‐RADS features and categories from free‐text liver magnetic resonance imaging (MRI) reports.MethodsThree radiologists generated 160 fictitious free‐text liver MRI reports written in Korean and English, simulating real‐world practice. Of these, 20 were used for prompt engineering, and 140 formed the internal test cohort. Seventy‐two genuine reports, authored by 17 radiologists were collected and de‐identified for the external test cohort. LI‐RADS features were extracted using GPT‐4, with a Python script calculating categories. Accuracies in each test cohort were compared.ResultsOn the external test, the accuracy for the extraction of major LI‐RADS features, which encompass size, nonrim arterial phase hyperenhancement, nonperipheral ‘washout’, enhancing ‘capsule’ and threshold growth, ranged from .92 to .99. For the rest of the LI‐RADS features, the accuracy ranged from .86 to .97. For the LI‐RADS category, the model showed an accuracy of .85 (95% CI: .76, .93).ConclusionsGPT‐4 shows promise in extracting LI‐RADS features, yet further refinement of its prompting strategy and advancements in its neural network architecture are crucial for reliable use in processing complex real‐world MRI reports.

show abstract

Section: Discussionmentioning

confidence: 84%

Using GPT‐4 for LI‐RADS feature extraction and categorization with multilingual free‐text reports

Gu,

Lee,

Shin

et al. 2024

Liver International

View full text Add to dashboard Cite

show abstract

“…After screening, 54 studies were included in the meta-analysis 11–14,19–68 (Figure 1 and Table 1). The most evaluated models were GPT-4 3 (31 articles) and GPT-3.5 2 (28), while models such as GPT-4V 69 (6), PaLM2 7 (3), Llama 2 5 (2), Prometheus (2), GPT-3 2 (1), Glass AI 70 (1), and Med-42 56 (1) had less representation. GPT-3, GPT-3.5, GPT-4, and GPT-4V are available in ChatGPT or its application programming interface (Open AI, San Francisco, CA).…”

Section: Resultsmentioning

confidence: 99%

“…A search strategy was developed, including variations of the terms generative AI and diagnosis . The search strategy was as follows: articles in English that included the words “large language model”, “LLM”, “generative artificial intelligence”, “generative AI”, “generative pre-trained transformers” 1 , “GPT”, “Bing”, “Prometheus”, “Bard”, “PaLM” 6,7 , “Pathways Language Model”, “LaMDA” 8 , “Language Model for Dialogue Applications”, “Llama” 4,5 , or “Large Language Model Meta AI” and also “diagnosis”, “diagnostic”, “quiz”, “examination”, or “vignette” were included. We searched the following electronic databases for literature from June 2018 through December 2023: Medline, Scopus, Web of Science, Cochrane Central, and MedRxiv.…”

Section: Methodsmentioning

confidence: 99%

Diagnostic Performance Comparison between Generative AI and Physicians: A Systematic Review and Meta-Analysis

Takita,

Kabata,

Walston

et al. 2024

Preprint

View full text Add to dashboard Cite

Background: The rapid advancement of generative artificial intelligence (AI) has revolutionized understanding and generation of human language. Their integration into healthcare has shown potential for improving medical diagnostics, yet a comprehensive diagnostic performance evaluation of generative AI models and the comparison of their diagnostic performance with that of physicians has not been extensively explored. Methods: In this systematic review and meta-analysis, a comprehensive search of Medline, Scopus, Web of Science, Cochrane Central, and medRxiv was conducted for studies published from June 2018 through December 2023, focusing on those that validate generative AI models for diagnostic tasks. Meta-analysis was performed to summarize the performance of the models and to compare the accuracy of the models with that of physicians. The quality of studies was assessed using the Prediction Model Study Risk of Bias Assessment Tool. Results: The search resulted in 54 studies being included in the meta-analysis, with 13 of these also used in the comparative analysis. Eight models were evaluated across 17 medical specialties. The overall accuracy for generative AI models across 54 studies was 57% (95% confidence interval [CI]: 51–63%). The I-squared statistic of 96% signifies a high degree of heterogeneity among the study results. Meta-regression analysis of generative AI models revealed significantly improved accuracy for GPT-4, and reduced accuracy for some specialties such as Neurology, Endocrinology, Rheumatology, and Radiology. The comparison meta-analysis demonstrated that, on average, physicians exceeded the accuracy of the models (difference in accuracy: 14% [95% CI: 8–19%], p-value <0.001). However, in the performance comparison between GPT-4 and physicians, GPT-4 performed slightly higher than non-experts (-4% [95% CI: -10–2%], p-value = 0.173), and slightly underperformed compared to experts (6% [95% CI: -1–13%], p-value = 0.091). The quality assessment indicated a high risk of bias in the majority of studies, primarily due to small sample sizes. Conclusions: Generative AI exhibits promising diagnostic capabilities, with accuracy varying significantly by model and medical specialty. Although they have not reached the reliability of expert physicians, the findings suggest that generative AI models have the potential to enhance healthcare delivery and medical education, provided they are integrated with caution and their limitations are well-understood. This study also highlights the need for more rigorous research standards and a larger number of cases in the future.

show abstract

“…Questions were rephrased where appropriate for clarity and to make them more universally applicable. Each question was posed 3 times to the web-based chat interface for ChatGPT-3.5 (14) and Bard (15) between June 1 st 2023 and June 24 th 2023. The questions from each of the three categories (diagnosis, management and prognosis) were posed by one of the co-authors in the order they appeared in the table.…”

Section: Methodsmentioning

confidence: 99%

Accuracy and Consistency of Online Chat-based Artificial Intelligence Platforms in Answering Patients’ Questions About Heart Failure

Kozaily,

Geagea,

Akdogan

et al. 2023

Preprint

View full text Add to dashboard Cite

BackgroundHeart failure (HF) is a prevalent condition associated with significant morbidity. Patients may have questions that they feel embarrassed to ask or will face delays awaiting responses from their healthcare providers which may impact their health behavior. We aimed to investigate the potential of chat-based artificial intelligence (AI) platforms in complementing the delivery of patient-centered care.MethodsUsing online patient forums and physician experience, we created 30 questions related to diagnosis, management and prognosis of HF. The questions were posed to two artificial intelligence (AI) chatbots (OpenAI’s ChatGPT-3.5 and Google’s Bard). Each set of answers was evaluated by two HF experts, independently and blinded to each other, for accuracy (adequacy of content) and consistency of content.ResultsChatGPT provided mostly appropriate answers (27/30, 90%) and showed a high degree of consistency (93%). Bard provided a similar content in its answers and thus was evaluated only for adequacy (23/30, 77%). The two HF experts’ grades were concordant in 83% and 67% of the questions for ChatGPT and Bard, respectively. Both platforms suffered from issues related to “hallucination” of facts and/or difficulty with more contemporary recommendations.ConclusionAI based chatbots may have potential in improving HF education and empowering patients, but their limitations should be considered and addressed in future research.

show abstract

Ultrasonographic and cytopathological characteristics of thyroid nodules in Graves' disease

Cited by 4 publications

References 0 publications

Using GPT‐4 for LI‐RADS feature extraction and categorization with multilingual free‐text reports

Using GPT‐4 for LI‐RADS feature extraction and categorization with multilingual free‐text reports

Diagnostic Performance Comparison between Generative AI and Physicians: A Systematic Review and Meta-Analysis

Accuracy and Consistency of Online Chat-based Artificial Intelligence Platforms in Answering Patients’ Questions About Heart Failure

Contact Info

Product

Resources

About