Tackling the ADReSS Challenge: A Multimodal Approach to the Automated Recognition of Alzheimer’s Dementia

Martinc, Matej; Pollak, Senja

doi:10.21437/interspeech.2020-2202

Cited by 27 publications

(22 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 3 shows the classification results achieved by our system on the DementiaBank dataset. With LDA classifier and using only linguistic features, we achieve an accuracy of 96%, 97% sensitivity and 92.3% specificity, providing the higher classifi- [11,14] -86/-/-96/-/-86/-/-- [28] 77/-/-51/-/-55/-/--Luz et al [10] -75/-/---cation metrics among all the classifiers and features combinations. Furthermore, the best classification results with Feature Selection are obtained when using only 15% of the features.…”

Section: Discussionmentioning

confidence: 89%

An Automatic System for Dementia Detection using Acoustic and Linguistic Features

Gonzalez-Atienza¹,

Peinado²,

Gonzalez-Lopez³

2021

IberSPEECH 2021

View full text Add to dashboard Cite

Early diagnosis of dementia is crucial for mitigating the consequences of this disease in patients. Previous studies have demonstrated that it is possible to detect the symptoms of dementia, in some cases even years before the onset of the disease, by detecting neurodegeneration-associated characteristics in a person's speech. This paper presents an automatic method for detecting dementia caused by Alzheimer's disease (AD) through a wide range of acoustic and linguistic features extracted from the person's speech. Two well-known databases containing speech for patients with AD and healthy controls are used to this end: DementiaBank and ADReSS. The experimental results show that our system is able to achieve state-of-theart performance on both databases. Furthermore, our results also show that the linguistic features extracted from the speech transcription are significantly better for detecting dementia.

show abstract

Section: Discussionmentioning

confidence: 89%

An Automatic System for Dementia Detection using Acoustic and Linguistic Features

Gonzalez-Atienza¹,

Peinado²,

Gonzalez-Lopez³

2021

IberSPEECH 2021

View full text Add to dashboard Cite

show abstract

“…Clustering of feature vectors : All word level feature vectors were aggregated into clusters using k-means clustering 4 . This is in contrast with the original implementation (Haider et al, 2020 ), which employed self-organising maps (SOM) clustering (Kohonen, 1990 ) but in line with the work done by Martinc and Pollak ( 2020 ).…”

Section: Ad Detectionmentioning

confidence: 83%

“…• Centroid: Includes four ADR features that model structural, semantic and temporal aspects of the data, namely cluster counts, duration, audio-textual centroid embeddings and audio-textual centroid velocity. • New: Includes only the four new ADR features which have not been used in the previous studies where ADR was employed (Haider et al, 2020;Martinc and Pollak, 2020), namely audio-textual centroid embeddings, audio-textual centroid velocity, audio-textual word/pause embeddings and audiotextual centroid embeddings. • All: Includes all 6 ADR features described in section 5.1.2.…”

Section: Feature Engineeringmentioning

confidence: 99%

“…We show that these features are capable of modelling temporal aspects of text and audio, but fall short of semantic modelling. To address this shortcoming, we propose combining ADR features with term frequency-inverse document frequency weighted bag-of-n-grams features, which proved effective in modelling semantics in previous studies (Martinc and Pollak, 2020 ). The final combination of ADR and bag-of-n-grams features leads to state-of-the-art performance on the AD classification task 1 .…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Temporal Integration of Text Transcripts and Acoustic Features for Alzheimer's Diagnosis Based on Spontaneous Speech

Martinc

Haider

Pollak

et al. 2021

Front. Aging Neurosci.

Self Cite

View full text Add to dashboard Cite

Background: Advances in machine learning (ML) technology have opened new avenues for detection and monitoring of cognitive decline. In this study, a multimodal approach to Alzheimer's dementia detection based on the patient's spontaneous speech is presented. This approach was tested on a standard, publicly available Alzheimer's speech dataset for comparability. The data comprise voice samples from 156 participants (1:1 ratio of Alzheimer's to control), matched by age and gender.Materials and Methods: A recently developed Active Data Representation (ADR) technique for voice processing was employed as a framework for fusion of acoustic and textual features at sentence and word level. Temporal aspects of textual features were investigated in conjunction with acoustic features in order to shed light on the temporal interplay between paralinguistic (acoustic) and linguistic (textual) aspects of Alzheimer's speech. Combinations between several configurations of ADR features and more traditional bag-of-n-grams approaches were used in an ensemble of classifiers built and evaluated on a standardised dataset containing recorded speech of scene descriptions and textual transcripts.Results: Employing only semantic bag-of-n-grams features, an accuracy of 89.58% was achieved in distinguishing between Alzheimer's patients and healthy controls. Adding temporal and structural information by combining bag-of-n-grams features with ADR audio/textual features, the accuracy could be improved to 91.67% on the test set. An accuracy of 93.75% was achieved through late fusion of the three best feature configurations, which corresponds to a 4.7% improvement over the best result reported in the literature for this dataset.Conclusion: The proposed combination of ADR audio and textual features is capable of successfully modelling temporal aspects of the data. The machine learning approach toward dementia detection achieves best performance when ADR features are combined with strong semantic bag-of-n-grams features. This combination leads to state-of-the-art performance on the AD classification task.

show abstract

“…Previous work has been done using the ADReSS dataset. Some researchers only participated in the AD classification task (Edwards et al, 2020 ; Pompili et al, 2020 ; Yuan et al, 2020 ), others only participated in the Mini-Mental State Examination (MMSE) prediction task (Farzana and Parde, 2020 ), and others participated in both tasks (Balagopalan et al, 2020 ; Cummins et al, 2020 ; Koo et al, 2020 ; Luz et al, 2020 ; Martinc and Pollak, 2020 ; Pappagari et al, 2020 ; Rohanian et al, 2020 ; Sarawgi et al, 2020 ; Searle et al, 2020 ; Syed et al, 2020 ). The best performance on the AD classification task was achieved by Yuan et al ( 2020 ), who obtained an accuracy of 89.6% on the test set using linguistic features extracted from the transcripts, as well as encoded pauses.…”

Section: Introductionmentioning

confidence: 99%

Classifying Alzheimer's Disease Using Audio and Text-Based Representations of Speech

Haulcy

Glass

2021

Front. Psychol.

View full text Add to dashboard Cite

Alzheimer's Disease (AD) is a form of dementia that affects the memory, cognition, and motor skills of patients. Extensive research has been done to develop accessible, cost-effective, and non-invasive techniques for the automatic detection of AD. Previous research has shown that speech can be used to distinguish between healthy patients and afflicted patients. In this paper, the ADReSS dataset, a dataset balanced by gender and age, was used to automatically classify AD from spontaneous speech. The performance of five classifiers, as well as a convolutional neural network and long short-term memory network, was compared when trained on audio features (i-vectors and x-vectors) and text features (word vectors, BERT embeddings, LIWC features, and CLAN features). The same audio and text features were used to train five regression models to predict the Mini-Mental State Examination score for each patient, a score that has a maximum value of 30. The top-performing classification models were the support vector machine and random forest classifiers trained on BERT embeddings, which both achieved an accuracy of 85.4% on the test set. The best-performing regression model was the gradient boosting regression model trained on BERT embeddings and CLAN features, which had a root mean squared error of 4.56 on the test set. The performance on both tasks illustrates the feasibility of using speech to classify AD and predict neuropsychological scores.

show abstract

Tackling the ADReSS Challenge: A Multimodal Approach to the Automated Recognition of Alzheimer’s Dementia

Cited by 27 publications

References 11 publications

An Automatic System for Dementia Detection using Acoustic and Linguistic Features

An Automatic System for Dementia Detection using Acoustic and Linguistic Features

Temporal Integration of Text Transcripts and Acoustic Features for Alzheimer's Diagnosis Based on Spontaneous Speech

Classifying Alzheimer's Disease Using Audio and Text-Based Representations of Speech

Contact Info

Product

Resources

About