Information Quality and Readability: ChatGPT's Responses to the Most Common Questions About Spinal Cord Injury

Temel, Mustafa Hüseyin; Erden, Yakup; Bağcıer, Fatih

doi:10.1016/j.wneu.2023.11.062

Cited by 11 publications

(4 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Utilizing the Flesch-Kincaid Reading Ease score and grade level, they demonstrated reading ease for treatment-related questions similar to our findings (47.67 ± 10.77 vs. 42.9 ± 12.4) but noted a higher grade level compared to ours. This echoes Tamel et al's findings [40]. To assess the response reliability, they applied the DISCERN tests for treatment-related questions in contrast to our approach, which involved compiling the most frequently asked patient questions.…”

Section: Comparative Analysis Of Chatgpt's Performancesupporting

confidence: 74%

“…In their study, Temel et al [40] evaluated the responses generated by ChatGPT to inquiries related to spinal cord injuries by using the most frequently searched keywords. They found that the complexity of ChatGPT's responses, as indicated by a Flesch-Kincaid grade level of (14.84 ± 1.79), was significantly higher than that in our study, which recorded a grade level of (10.8 ± 2.2).…”

Section: Comparative Analysis Of Chatgpt's Performancementioning

confidence: 99%

See 1 more Smart Citation

Comparative Analysis of Artificial Intelligence Virtual Assistant and Large Language Models in Post-Operative Care

Borna,

Gomez-Cabello,

Pressman

et al. 2024

EJIHPE

View full text Add to dashboard Cite

In postoperative care, patient education and follow-up are pivotal for enhancing the quality of care and satisfaction. Artificial intelligence virtual assistants (AIVA) and large language models (LLMs) like Google BARD and ChatGPT-4 offer avenues for addressing patient queries using natural language processing (NLP) techniques. However, the accuracy and appropriateness of the information vary across these platforms, necessitating a comparative study to evaluate their efficacy in this domain. We conducted a study comparing AIVA (using IBM Watson Assistant) with ChatGPT-4 and Google BARD, assessing the accuracy, knowledge gap, and response appropriateness. AIVA demonstrated superior performance, with significantly higher accuracy (mean: 0.9) and lower knowledge gap (mean: 0.1) compared to BARD and ChatGPT-4. Additionally, AIVA’s responses received higher Likert scores for appropriateness. Our findings suggest that specialized AI tools like AIVA are more effective in delivering precise and contextually relevant information for postoperative care compared to general-purpose LLMs. While ChatGPT-4 shows promise, its performance varies, particularly in verbal interactions. This underscores the importance of tailored AI solutions in healthcare, where accuracy and clarity are paramount. Our study highlights the necessity for further research and the development of customized AI solutions to address specific medical contexts and improve patient outcomes.

show abstract

Section: Comparative Analysis Of Chatgpt's Performancesupporting

confidence: 74%

Section: Comparative Analysis Of Chatgpt's Performancementioning

confidence: 99%

Comparative Analysis of Artificial Intelligence Virtual Assistant and Large Language Models in Post-Operative Care

Borna,

Gomez-Cabello,

Pressman

et al. 2024

EJIHPE

View full text Add to dashboard Cite

show abstract

“…Rather than presenting users with an overwhelming array of web pages, LLMs synthesize information into concise, coherent paragraphs that respond directly to the user's query, thereby facilitating patient education and setting the stage for more substantive discussions within the framework of SDM For LLMs to effectively contribute to SDM, it is imperative that the information they provide is accessible to patients, necessitating a literacy level commensurate with that of the intended audience. Recent analyses have indicated that outputs from models like ChatGPT often demand high literacy levels for comprehension, posing challenges in the context of patient education on complex medical topics (Dash et al 2023 ; Haver et al 2024 ; Onder et al 2024 ; Temel et al 2024 ). However, this limitation can be addressed through strategic prompting—the method by which a query is formulated by the user (Gao 2023 ).…”

Section: Bridging the Gap With Llmsmentioning

confidence: 99%

Large language models as decision aids in neuro-oncology: a review of shared decision-making applications

Lawson McLean,

Wu,

Lawson McLean

et al. 2024

J Cancer Res Clin Oncol

View full text Add to dashboard Cite

Shared decision-making (SDM) is crucial in neuro-oncology, fostering collaborations between patients and healthcare professionals to navigate treatment options. However, the complexity of neuro-oncological conditions and the cognitive and emotional burdens on patients present significant barriers to achieving effective SDM. This discussion explores the potential of large language models (LLMs) such as OpenAI's ChatGPT and Google's Bard to overcome these barriers, offering a means to enhance patient understanding and engagement in their care. LLMs, by providing accessible, personalized information, could support but not supplant the critical insights of healthcare professionals. The hypothesis suggests that patients, better informed through LLMs, may participate more actively in their treatment choices. Integrating LLMs into neuro-oncology requires navigating ethical considerations, including safeguarding patient data and ensuring informed consent, alongside the judicious use of AI technologies. Future efforts should focus on establishing ethical guidelines, adapting healthcare workflows, promoting patient-oriented research, and developing training programs for clinicians on the use of LLMs. Continuous evaluation of LLM applications will be vital to maintain their effectiveness and alignment with patient needs. Ultimately, this exploration contends that the thoughtful integration of LLMs into SDM processes could significantly enhance patient involvement and strengthen the patient-physician relationship in neuro-oncology care.

show abstract

“…Numerous authors in various fields have tried to pose clinical questions. The results are variable but all authors conclude that thus far AI can't compete with a real doctor [30][31][32][33][34]. In a study on paediatric emergencies for example, ChatGPT/GPT-4 reliably advised to call emergency services only in 54% of the cases, gave correct first aid instructions in 45% and incorrectly advised advanced life support techniques to parents 13.6% [35].…”

Section: Plos Digital Healthmentioning

confidence: 99%

Microsoft Bing outperforms five other generative artificial intelligence chatbots in the Antwerp University multiple choice medical license exam

Morreel,

Verhoeven,

Mathysen

2024

PLOS Digit Health

View full text Add to dashboard Cite

Recently developed chatbots based on large language models (further called bots) have promising features which could facilitate medical education. Several bots are freely available, but their proficiency has been insufficiently evaluated. In this study the authors have tested the current performance on the multiple-choice medical licensing exam of University of Antwerp (Belgium) of six widely used bots: ChatGPT (OpenAI), Bard (Google), New Bing (Microsoft), Claude instant (Anthropic), Claude+ (Anthropic) and GPT-4 (OpenAI). The primary outcome was the performance on the exam expressed as a proportion of correct answers. Secondary analyses were done for a variety of features in the exam questions: easy versus difficult questions, grammatically positive versus negative questions, and clinical vignettes versus theoretical questions. Reasoning errors and untruthful statements (hallucinations) in the bots’ answers were examined. All bots passed the exam; Bing and GPT-4 (both 76% correct answers) outperformed the other bots (62–67%, p = 0.03) and students (61%). Bots performed worse on difficult questions (62%, p = 0.06), but outperformed students (32%) on those questions even more (p<0.01). Hallucinations were found in 7% of Bing’s and GPT4’s answers, significantly lower than Bard (22%, p<0.01) and Claude Instant (19%, p = 0.02). Although the creators of all bots try to some extent to avoid their bots being used as a medical doctor, none of the tested bots succeeded as none refused to answer all clinical case questions.Bing was able to detect weak or ambiguous exam questions. Bots could be used as a time efficient tool to improve the quality of a multiple-choice exam.

show abstract

Information Quality and Readability: ChatGPT's Responses to the Most Common Questions About Spinal Cord Injury

Cited by 11 publications

References 20 publications

Comparative Analysis of Artificial Intelligence Virtual Assistant and Large Language Models in Post-Operative Care

Comparative Analysis of Artificial Intelligence Virtual Assistant and Large Language Models in Post-Operative Care

Large language models as decision aids in neuro-oncology: a review of shared decision-making applications

Microsoft Bing outperforms five other generative artificial intelligence chatbots in the Antwerp University multiple choice medical license exam

Contact Info

Product

Resources

About