Performance of ChatGPT 3.5 and 4 as a tool for patient support before and after DBS surgery for Parkinson’s disease

Abstract

Deep brain stimulation (DBS) is a neurosurgical procedure that involves implanting electrodes into specific areas of the brain to treat a variety of medical conditions, including Parkinson’s disease. Doubts and questions from patients prior to or following surgery should be addressed in line with the most recent scientific and clinical practice. ChatGPT emerges as an example of how artificial intelligence can be used, with its ability to comprehend and answer medical questions in an understandable way, accessible to everyone. However, the risks of these resources still need to be fully understood.ChatGPT models 3.5 and 4 responses to 40 questions in English and Portuguese were independently graded by two experienced specialists in functional neurosurgery and neurological movement disorders and resolved by a third reviewer. ChatGPT 3.5 and 4 demonstrated a good level of accuracy in responding to 80 questions in both English and Portuguese, related to DBS surgery for Parkinson’s disease. The proportion of responses graded as correct was 57.5% and 83.8% for GPT 3.5 and GPT 4, respectively. GPT 3.5 provided potentially harmful answers for 6.3% (5/80) of its responses. No responses from GPT 4 were graded as harmful. In general, ChatGPT 3.5 and 4 demonstrated good performance in terms of quality and reliability across two different languages. Nonetheless, harmful responses should not be scorned, and it’s crucial to consider this aspect when addressing patients using these resources. Considering the current safety concerns, it’s not advisable for patients to use such models for DBS surgery guidance. Performance of ChatGPT 3.5 and 4 as a tool for patient support before and after DBS surgery for Parkinson’s disease.