AI-based online chat and the future of oncology care: a promising technology or a solution in search of a problem?

Kassab, Joseph; Nasr, Lewis; Gebrael, Georges; Helou, Michel Chedid El; Saba, Ludovic; Haroun, Elio; Dahdah, Joseph El; Nasr, Fadi

doi:10.3389/fonc.2023.1176617

Cited by 7 publications

(4 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The present study adds to the existing body of literature analyzing the use of LLMs, such as ChatGPT, to answer patient questions regarding spine surgery. In contrast to other studies that utilized questions from social networks such as Reddit 5,14 , hypothetical questions devised by clinicians 10,15-18 , or items from licensing and student examinations 19-23 , the questions posed to the model in the present study were bona fide, commonly searched questions, as evidenced by their presence in the PAA section of Google search results. A recent study in orthopaedic surgery regarding total knee and hip arthroplasty applied a similar approach to the one utilized herein; however, far fewer questions were analyzed (20 versus 71) and the underlying ChatGPT model was utilized directly 24 .…”

Section: Discussionmentioning

confidence: 93%

Assessing the Accuracy and Reliability of AI-Generated Responses to Patient Questions Regarding Spine Surgery

Kasthuri,

Glueck,

Pham

et al. 2024

Journal of Bone and Joint Surgery

View full text Add to dashboard Cite

Background: In today’s digital age, patients increasingly rely on online search engines for medical information. The integration of large language models such as GPT-4 into search engines such as Bing raises concerns over the potential transmission of misinformation when patients search for information online regarding spine surgery. Methods: SearchResponse.io, a database that archives People Also Ask (PAA) data from Google, was utilized to determine the most popular patient questions regarding 4 specific spine surgery topics: anterior cervical discectomy and fusion, lumbar fusion, laminectomy, and spinal deformity. Bing’s responses to these questions, along with the cited sources, were recorded for analysis. Two fellowship-trained spine surgeons assessed the accuracy of the answers on a 6-point scale and the completeness of the answers on a 3-point scale. Inaccurate answers were re-queried 2 weeks later. Cited sources were categorized and evaluated against Journal of the American Medical Association (JAMA) benchmark criteria. Interrater reliability was measured with use of the kappa statistic. A linear regression analysis was utilized to explore the relationship between answer accuracy and the type of source, number of sources, and mean JAMA benchmark score. Results: Bing’s responses to 71 PAA questions were analyzed. The average completeness score was 2.03 (standard deviation [SD], 0.36), and the average accuracy score was 4.49 (SD, 1.10). Among the question topics, spinal deformity had the lowest mean completeness score. Re-querying the questions that initially had answers with low accuracy scores resulted in responses with improved accuracy. Among the cited sources, commercial sources were the most prevalent. The JAMA benchmark score across all sources averaged 2.63. Government sources had the highest mean benchmark score (3.30), whereas social media had the lowest (1.75). Conclusions: Bing’s answers were generally accurate and adequately complete, with incorrect responses rectified upon re-querying. The plurality of information was sourced from commercial websites. The type of source, number of sources, and mean JAMA benchmark score were not significantly correlated with answer accuracy. These findings underscore the importance of ongoing evaluation and improvement of large language models to ensure reliable and informative results for patients seeking information regarding spine surgery online amid the integration of these models in the search experience.

show abstract

Section: Discussionmentioning

confidence: 93%

Assessing the Accuracy and Reliability of AI-Generated Responses to Patient Questions Regarding Spine Surgery

Kasthuri,

Glueck,

Pham

et al. 2024

Journal of Bone and Joint Surgery

View full text Add to dashboard Cite

show abstract

“…Joseph Kassab’s recent article ( 1 ) inspired us to consider the potential impact of AI chatbots like ChatGPT on the global healthcare and the challenges that must be addressed to harness their full potential. One of the key challenges is the severe imbalance in the distribution of global healthcare resources, particularly in some regions of Asia, Africa, and Latin America.…”

mentioning

confidence: 99%

Commentary: AI-based online chat and the future of oncology care: a promising technology or a solution in search of a problem?

Zhang,

Guan,

Chen

et al. 2023

Front. Oncol.

View full text Add to dashboard Cite

“…Oncology witnesses ChatGPT as a vital tool for disseminating information on various cancer types, treatment options, and potential side effects. Beyond its informative role, the model serves as a source of emotional support for patients and their families, addressing concerns related to cancer diagnosis, treatment plans, and survivorship [ 6 ]. In neurology, ChatGPT acts as an educational guide, simplifying complex concepts, explaining diagnostic procedures, and providing information on available treatment options.…”

Section: Introductionmentioning

confidence: 99%

ChatGPT as a New Tool to Select a Biological for Chronic Rhino Sinusitis with Polyps, “Caution Advised” or “Distant Reality”?

Sireci,

Lorusso,

Immordino

et al. 2024

JPM

View full text Add to dashboard Cite

ChatGPT is an advanced language model developed by OpenAI, designed for natural language understanding and generation. It employs deep learning technology to comprehend and generate human-like text, making it versatile for various applications. The aim of this study is to assess the alignment between the Rhinology Board’s indications and ChatGPT’s recommendations for treating patients with chronic rhinosinusitis with nasal polyps (CRSwNP) using biologic therapy. An observational cohort study involving 72 patients was conducted to evaluate various parameters of type 2 inflammation and assess the concordance in therapy choices between ChatGPT and the Rhinology Board. The observed results highlight the potential of Chat-GPT in guiding optimal biological therapy selection, with a concordance percentage = 68% and a Kappa coefficient = 0.69 (CI95% [0.50; 0.75]). In particular, the concordance was, respectively, 79.6% for dupilumab, 20% for mepolizumab, and 0% for omalizumab. This research represents a significant advancement in managing CRSwNP, addressing a condition lacking robust biomarkers. It provides valuable insights into the potential of AI, specifically ChatGPT, to assist otolaryngologists in determining the optimal biological therapy for personalized patient care. Our results demonstrate the need to implement the use of this tool to effectively aid clinicians.

show abstract

AI-based online chat and the future of oncology care: a promising technology or a solution in search of a problem?

Cited by 7 publications

References 11 publications

Assessing the Accuracy and Reliability of AI-Generated Responses to Patient Questions Regarding Spine Surgery

Assessing the Accuracy and Reliability of AI-Generated Responses to Patient Questions Regarding Spine Surgery

Commentary: AI-based online chat and the future of oncology care: a promising technology or a solution in search of a problem?

ChatGPT as a New Tool to Select a Biological for Chronic Rhino Sinusitis with Polyps, “Caution Advised” or “Distant Reality”?

Contact Info

Product

Resources

About