Sixty seconds on . . . ChatGPT

Looi, Mun-Keat

doi:10.1136/bmj.p205

Cited by 24 publications

(14 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For instance, there are reports on how it stirs (medical) education, as it can generate convincing abstracts and essays, or write computer code. [12,13] There are also projects that use AI models to generate medical reports, for example for radiology, helping radiologists to quickly and accurately interpret and report the results of medical imaging. To stir up things more, researchers found that ChatGPT performed reasonably well on the US medical licensing exam.…”

Section: Prior Studiesmentioning

confidence: 99%

Performance of ChatGPT as an AI-assisted decision support tool in medicine: a proof-of-concept study for interpreting symptoms and management of common cardiac conditions (AMSTELHEART-2)

Harskamp

Clercq

2023

Preprint

View full text Add to dashboard Cite

Background: It is thought that ChatGPT, an advanced language model developed by OpenAI, may in the future serve as an AI-assisted decision support tool in medicine. Objective: To evaluate the accuracy of ChatGPTs recommendations on medical questions related to common cardiac symptoms or conditions. Methods: We tested ChatGPTs ability to address medical questions in two ways. First, we assessed its accuracy in correctly answering cardiovascular trivia questions (n=50), based on quizzes for medical professionals. Second, we entered 20 clinical case vignettes on the ChatGPT platform and evaluated its accuracy compared to expert opinion and clinical course. Results: We found that ChatGPT correctly answered 74% of the trivia questions, with slight variation in accuracy in the domains coronary artery disease (80%), pulmonary and venous thrombotic embolism (80%), atrial fibrillation (70%), heart failure (80%) and cardiovascular risk management (60%). In the case vignettes, ChatGPTs response matched in 90% of the cases with the actual advice given. In more complex cases, where physicians (general practitioners) asked other physicians (cardiologists) for assistance or decision support, ChatGPT was correct in 50% of cases, and often provided incomplete or inappropriate recommendations when compared with expert consultation. Conclusions: Our study suggests that ChatGPT has potential as an AI-assisted decision support tool in medicine, particularly for straightforward, low-complex medical questions, but further research is needed to fully evaluate its potential.

show abstract

Section: Prior Studiesmentioning

confidence: 99%

Performance of ChatGPT as an AI-assisted decision support tool in medicine: a proof-of-concept study for interpreting symptoms and management of common cardiac conditions (AMSTELHEART-2)

Harskamp

Clercq

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…. ChatGPT», що з'явилася на сторінках BMJ на початку цього року [7]. І дійсно, світ охопив новий тренд -використання штучного інтелекту для написання текстів.…”

Section: вступunclassified

Talk to me: A dialogue with artificial intelligence about its use in education and research

Suchikova¹

2023

Preprint

View full text Add to dashboard Cite

This article is devoted to the use of artificial intelligence in the educational process and when writing scientific articles, as well as ethical issues related to this use. The study is based on a cursory analysis of the chatbot dialogue with the ChatGPT and includes an analysis of the possibilities of using artificial intelligence in education. On the basis of this analysis, as well as the study of relevant literary sources, we come to the conclusion of the need for careful behaviour when using language models based on artificial intelligence. In the article, we raise the issue of academic integrity when using artificial intelligence to write texts. These issues, as well as ethical aspects, are still completely unresolved at the legislative and academic levels, because ChatGPT has existed for only a few months, and the ways of such a settlement are still not clear at all. We also come to the conclusion that ChatGPT is good at the technology of writing texts but often makes mistakes and provides completely false information. The results of the article can help to understand the potential of artificial intelligence in education and the problems associated with its use.

show abstract

“…2 Potential applications of ChatGPT have also received attention from the medical research community with numerous editorials and commentaries published on the topic in major scientific journals, while some authors already used and attributed co-authorship to ChatGPT. 3-6 Recent studies have demonstrated that ChatGPT and similar AI-based systems reached a level to be able to pass components or the full United States Medical Licensing Exam 7-9 and can answer questions in specific medical specialties like genetics and ophthalmology. 10,11 Such proof of clinical knowledge is promising, but studies and applications in patient-centered scenarios are still rare.…”

Section: Introductionmentioning

confidence: 99%

ChatGPT- versus human-generated answers to frequently asked questions about diabetes: a Turing test-inspired survey among employees of a Danish diabetes center

Hulmán

Dollerup

Mortensen

et al. 2023

Preprint

View full text Add to dashboard Cite

Background Large language models have received enormous attention recently with some studies demonstrating their potential clinical value, despite not being trained specifically for this domain. We aimed to investigate whether ChatGPT, a language model optimized for dialogue, can answer frequently asked questions about diabetes. Methods We conducted a closed e-survey among employees of a large Danish diabetes center. The study design was inspired by the Turing test and non-inferiority trials. Our survey included ten questions with two answers each. One of these was written by a human expert, while the other was generated by ChatGPT. Participants had the task to identify the ChatGPT-generated answer. Data was analyzed at the question-level using logistic regression with robust variance estimation with clustering at participant level. In secondary analyses, we investigated the effect of participant characteristics on the outcome. A 55% non-inferiority margin was pre-defined based on precision simulations and had been published as part of the study protocol before data collection began. Findings Among 311 invited individuals, 183 participated in the survey (59% response rate). 64% had heard of ChatGPT before, and 19% had tried it. Overall, participants could identify ChatGPT-generated answers 59.5% (95% CI: 57.0, 62.0) of the time. Among participant characteristics, previous ChatGPT use had the strongest association with the outcome (odds ratio: 1.52 (1.16, 2.00), p=0.003). Previous users answered 67.4% (61.7, 72.7) of the questions correctly, versus non-users 57.6% (54.9, 60.3). Interpretation Participants could distinguish between ChatGPT-generated and human-written answers somewhat better than flipping a fair coin. However, our results suggest a stronger predictive value of linguistic features rather than the actual content. Rigorously planned studies are needed to elucidate the risks and benefits of integrating such technologies in routine clinical practice.

show abstract

Sixty seconds on . . . ChatGPT

Cited by 24 publications

References 3 publications

Performance of ChatGPT as an AI-assisted decision support tool in medicine: a proof-of-concept study for interpreting symptoms and management of common cardiac conditions (AMSTELHEART-2)

Performance of ChatGPT as an AI-assisted decision support tool in medicine: a proof-of-concept study for interpreting symptoms and management of common cardiac conditions (AMSTELHEART-2)

Talk to me: A dialogue with artificial intelligence about its use in education and research

ChatGPT- versus human-generated answers to frequently asked questions about diabetes: a Turing test-inspired survey among employees of a Danish diabetes center

Contact Info

Product

Resources

About