Predicting dementia from spontaneous speech using large language models

Agbavor, Felix; Liang, Hualou

doi:10.1371/journal.pdig.0000168

Cited by 49 publications

(58 citation statements)

References 35 publications

(39 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our own group previously showed that frameworks such as BERT and neural network–based sentence encoding can be used to automatically transcribe digital voice recordings and differentiate cognitively impaired persons from those with normal cognition. 22 Agbavor and Liang 23 similarly leveraged GPT-3 to develop a model to predict dementia in persons using their spontaneous speech.…”

Section: How Large Language Models Workmentioning

confidence: 99%

Large Language Models in Neurology Research and Future Practice

Romano,

Shih,

Paschalidis

et al. 2023

Neurology

View full text Add to dashboard Cite

Recent advancements in generative artificial intelligence, particularly using large language models (LLMs), are gaining increased public attention. We provide a perspective on the potential of LLMs to analyze enormous amounts of data from medical records and gain insights on specific topics in neurology. Additionally, we explore use cases for LLMs such as early diagnosis, supporting patient and caregivers, and acting as an assistant for clinicians. We point to the potential ethical and technical challenges raised by LLMs, such as concerns about privacy and data security, potential biases in the data for model training, and the need for careful validation of results. Researchers must consider these challenges and take steps to address them to ensure that their work is conducted in a safe and responsible manner. Despite these challenges, LLMs offer promising opportunities for improving care and treatment of various neurological disorders.

show abstract

Section: How Large Language Models Workmentioning

confidence: 99%

Large Language Models in Neurology Research and Future Practice

Romano,

Shih,

Paschalidis

et al. 2023

Neurology

View full text Add to dashboard Cite

show abstract

“…Therefore, medical research articles related to it are limited. Some GPT-3-related medical articles have been reported about a medical dialogue summarizer [ 28 ], navigation application for electronic health record systems [ 5 ], and its implementation in clinical use [ 7 ] including ophthalmology [ 4 ] and dementia prediction [ 29 ].…”

Section: Introductionmentioning

confidence: 99%

Diagnostic Accuracy of Differential-Diagnosis Lists Generated by Generative Pretrained Transformer 3 Chatbot for Clinical Vignettes with Common Chief Complaints: A Pilot Study

Hirosawa

Harada

Yokose

et al. 2023

IJERPH

201

114

View full text Add to dashboard Cite

The diagnostic accuracy of differential diagnoses generated by artificial intelligence (AI) chatbots, including the generative pretrained transformer 3 (GPT-3) chatbot (ChatGPT-3) is unknown. This study evaluated the accuracy of differential-diagnosis lists generated by ChatGPT-3 for clinical vignettes with common chief complaints. General internal medicine physicians created clinical cases, correct diagnoses, and five differential diagnoses for ten common chief complaints. The rate of correct diagnosis by ChatGPT-3 within the ten differential-diagnosis lists was 28/30 (93.3%). The rate of correct diagnosis by physicians was still superior to that by ChatGPT-3 within the five differential-diagnosis lists (98.3% vs. 83.3%, p = 0.03). The rate of correct diagnosis by physicians was also superior to that by ChatGPT-3 in the top diagnosis (53.3% vs. 93.3%, p < 0.001). The rate of consistent differential diagnoses among physicians within the ten differential-diagnosis lists generated by ChatGPT-3 was 62/88 (70.5%). In summary, this study demonstrates the high diagnostic accuracy of differential-diagnosis lists generated by ChatGPT-3 for clinical cases with common chief complaints. This suggests that AI chatbots such as ChatGPT-3 can generate a well-differentiated diagnosis list for common chief complaints. However, the order of these lists can be improved in the future.

show abstract

“…More researchers tend to explore the application of deep learning-based embedding vectors in dementia detection. For example, Agbavor’s research team recently explored the utility of text-embedding vectors of the GPT-3 model in dementia detection for the first time in the ADReSSo-based dataset [39]. In addition, the team constructed a new end-to-end detection system by training the data2vec model to combine multimodal information effectively [40].…”

Section: Discussionmentioning

confidence: 99%

Explainable Alzheimer’s Disease Detection Using Linguistic Features from Automatic Speech Recognition

Tang¹,

Zhang²,

Feng³

et al. 2023

Dement Geriatr Cogn Disord

View full text Add to dashboard Cite

Introduction: Alzheimer’s disease (AD) is the most prevalent type of dementia and can cause abnormal cognitive function and progressive loss of essential life skills. Early screening is thus necessary for the prevention and intervention of AD. Speech dysfunction is an early onset symptom of AD patients. Recent studies have demonstrated the promise of automated acoustic assessment using acoustic or linguistic features extracted from speech. However, most previous studies have relied on manual transcription of text to extract linguistic features, which weakens the efficiency of automated assessment. The present study thus investigates the effectiveness of automatic speech recognition (ASR) in building an end-to-end automated speech analysis model for AD detection. Methods: We implemented three publicly available ASR engines and compared the classification performance using the ADReSS-IS2020 dataset. Besides, the SHapley Additive exPlanations algorithm was then used to identify critical features that contributed most to model performance. Results: Three automatic transcription tools obtained mean word error rate texts of 32%, 43%, and 40%, respectively. These automated texts achieved similar or even better results than manual texts in model performance for detecting dementia, achieving classification accuracies of 89.58%, 83.33%, and 81.25%, respectively. Conclusion: Our best model, using ensemble learning, is comparable to the state-of-the-art manual transcription-based methods, suggesting the possibility of an end-to-end medical assistance system for AD detection with ASR engines. Moreover, the critical linguistic features might provide insight into further studies on the mechanism of AD.

show abstract

Predicting dementia from spontaneous speech using large language models

Cited by 49 publications

References 35 publications

Large Language Models in Neurology Research and Future Practice

Large Language Models in Neurology Research and Future Practice

Diagnostic Accuracy of Differential-Diagnosis Lists Generated by Generative Pretrained Transformer 3 Chatbot for Clinical Vignettes with Common Chief Complaints: A Pilot Study

Explainable Alzheimer’s Disease Detection Using Linguistic Features from Automatic Speech Recognition

Contact Info

Product

Resources

About