ChatGPT and its Role in the Decision-Making for the Diagnosis and Treatment of Lumbar Spinal Stenosis: A Comparative Analysis and Narrative Review

Abstract

Study Design Comparative Analysis and Narrative Review. Objective To assess and compare ChatGPT’s responses to the clinical questions and recommendations proposed by The 2011 North American Spine Society (NASS) Clinical Guideline for the Diagnosis and Treatment of Degenerative Lumbar Spinal Stenosis (LSS). We explore the advantages and disadvantages of ChatGPT’s responses through an updated literature review on spinal stenosis. Methods We prompted ChatGPT with questions from the NASS Evidence-based Clinical Guidelines for LSS and compared its generated responses with the recommendations provided by the guidelines. A review of the literature was performed via PubMed, OVID, and Cochrane on the diagnosis and treatment of lumbar spinal stenosis between January 2012 and April 2023. Results 14 questions proposed by the NASS guidelines for LSS were uploaded into ChatGPT and directly compared to the responses offered by NASS. Three questions were on the definition and history of LSS, one on diagnostic tests, seven on non-surgical interventions and three on surgical interventions. The review process found 40 articles that were selected for inclusion that helped corroborate or contradict the responses that were generated by ChatGPT. Conclusions ChatGPT’s responses were similar to findings in the current literature on LSS. These results demonstrate the potential for implementing ChatGPT into the spine surgeon’s workplace as a means of supporting the decision-making process for LSS diagnosis and treatment. However, our narrative summary only provides a limited literature review and additional research is needed to standardize our findings as means of validating ChatGPT’s use in the clinical space.

ChatGPT fails challenging the recent ESCMID brain abscess guideline

“…On the topic of post-colonoscopy management, ChatGPT provided responses with 90% adherence to guidelines and 85% accuracy [ 13 ], suggesting beneficial use for healthcare providers and patients. ChatGPT’s recommendations on the management of lumbar spinal stenosis were also in line with findings in the current literature [ 9 ]. When asked guideline-based questions on urological topics, ChatGPT provided only 60% appropriate responses [ 10 ].…”

“…ChatGPT has already been tested and compared to several clinical guidelines from varying medical departments, for example for treatment of advanced solid tumors [ 7 ], spine surgery [ 8 , 9 ], urology [ 10 ], and diabetic ketoacidosis [ 11 , 12 ].…”

Automating Responses to Patient Portal Messages Using Generative AI

“…Already, several studies are exploring feasibility of integrating systems such as GPT to generate highquality responses to patient inquiries and aid clinical decisions-making across various medical specialties. 10,15,16 Of note, crafting effective prompts entailed iterative trial and error. The potential for performance variation underscores the importance of understanding the model's reliance on training data patterns and ensuring the relevance and quality of examples provided.…”

“…Already, several studies are exploring feasibility of integrating systems such as GPT to generate high-quality responses to patient inquiries and aid clinical decisions-making across various medical specialties. 10,15,16…”

“…[7][8][9] Several studies have highlighted generative AI's effectiveness in clinical decision support. [9][10][11] Recent improvements in prompt engineering through techniques such as few-shot learning have allowed automation of some in-basket message work such as patient portal message responses. 7 Several studies have aimed to develop and evaluate the effectiveness of fine-tuned large language models (LLMs) in generating responses to patient queries.…”