Automatic Detection and Assessment of Alzheimer Disease Using Speech and Language Technologies in Low-Resource Scenarios

Pappagari, Raghavendra; Cho, Jaejin; Joshi, Sonal; Moro-Velázquez, Laureano; Żelasko, Piotr; Villalba, Jesús; Dehak, Najim

doi:10.21437/interspeech.2021-1850

Cited by 30 publications

(23 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The used subset has 87 recordings from speakers with AD and 79 from control subjects. Considering the small amount of the utterances in the data, we ran 10-fold CV as used in [26] where each fold has class-balanced distribution.…”

Section: Methodsmentioning

confidence: 99%

“…Using a multitask objective, e.g., emotion classification loss plus Alzheimer's disease (AD) classification loss, can be one way to increase the total number of samples, but an AD dataset as a medical data also usually has a small number of samples, thus making the multitask objective training still not solving the data scarcity problem better than transfer learning. In [25], [26], the authors used pre-trained speaker classification, and encoder-decoder ASR models for AD detection.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Non-Contrastive Self-Supervised Learning for Utterance-Level Information Extraction From Speech

Cho

Villalba

Moro-Velázquez

et al. 2022

IEEE J. Sel. Top. Signal Process.

Self Cite

View full text Add to dashboard Cite

In recent studies, self-supervised pre-trained models tend to outperform supervised pre-trained models in transfer learning. In particular, self-supervised learning of utterance-level speech representation can be used in speech applications that require discriminative representation of consistent attributes within an utterance: speaker, language, emotion, and age. Existing frame-level self-supervised speech representation, e.g., wav2vec, can be used as utterance-level representation with pooling, but the models are usually large. There are also self-supervised learning techniques to learn utterance-level representation. One of the most successful is a contrastive method, which requires negative sampling: selecting alternative samples to contrast with the current sample (anchor). However, this does not ensure that all the negative samples belong to classes different from the anchor class without labels. This paper applies a non-contrastive self-supervised method to learn utterance-level embeddings. We adapted DIstillation with NO labels (DINO) from computer vision to speech. Unlike contrastive methods, DINO does not require negative sampling. We compared DINO to x-vector trained in a supervised manner. When transferred to downstream tasks (speaker verification, speech emotion recognition, and Alzheimer's disease detection), DINO outperformed x-vector. We studied the influence of several aspects during transfer learning such as dividing the fine-tuning process into steps, chunk lengths, or augmentation. During fine-tuning, tuning the last affine layers first and then the whole network surpassed fine-tuning all at once. Using shorter chunk lengths, although they generate more diverse inputs, did not necessarily improve performance, implying speech segments at least with a specific length are required for better performance per application. Augmentation was helpful in speech emotion recognition.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Non-Contrastive Self-Supervised Learning for Utterance-Level Information Extraction From Speech

Cho

Villalba

Moro-Velázquez

et al. 2022

IEEE J. Sel. Top. Signal Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Pappagari et al ( 2021 ) proposed several acoustic and language models and fused the two modalities by using the output probabilities of the individual models as the inputs to a Logistic Regression classifier for obtaining a final prediction. Regarding the acoustic models, they used an end-to-end classifier by fine-tuning an x-vector model and trained a Logistic Regression and XGBoost classifier using features extracted via several open-source libraries.…”

Section: Related Workmentioning

confidence: 99%

Multimodal Deep Learning Models for Detecting Dementia From Speech and Transcripts

Ilias

Askounis

2022

Front. Aging Neurosci.

View full text Add to dashboard Cite

Alzheimer's dementia (AD) entails negative psychological, social, and economic consequences not only for the patients but also for their families, relatives, and society in general. Despite the significance of this phenomenon and the importance for an early diagnosis, there are still limitations. Specifically, the main limitation is pertinent to the way the modalities of speech and transcripts are combined in a single neural network. Existing research works add/concatenate the image and text representations, employ majority voting approaches or average the predictions after training many textual and speech models separately. To address these limitations, in this article we present some new methods to detect AD patients and predict the Mini-Mental State Examination (MMSE) scores in an end-to-end trainable manner consisting of a combination of BERT, Vision Transformer, Co-Attention, Multimodal Shifting Gate, and a variant of the self-attention mechanism. Specifically, we convert audio to Log-Mel spectrograms, their delta, and delta-delta (acceleration values). First, we pass each transcript and image through a BERT model and Vision Transformer, respectively, adding a co-attention layer at the top, which generates image and word attention simultaneously. Secondly, we propose an architecture, which integrates multimodal information to a BERT model via a Multimodal Shifting Gate. Finally, we introduce an approach to capture both the inter- and intra-modal interactions by concatenating the textual and visual representations and utilizing a self-attention mechanism, which includes a gate model. Experiments conducted on the ADReSS Challenge dataset indicate that our introduced models demonstrate valuable advantages over existing research initiatives achieving competitive results in both the AD classification and MMSE regression task. Specifically, our best performing model attains an accuracy of 90.00% and a Root Mean Squared Error (RMSE) of 3.61 in the AD classification task and MMSE regression task, respectively, achieving a new state-of-the-art performance in the MMSE regression task.

show abstract

“…2) EGEMAPSV02 : The extended Geneva Minimalistic Acoustic Parameter Set (eGeMAPS) features are a selected standardized set of statistical features that characterize affective physiological changes in voice production. We extracted these features for the entire recording, as this feature set was shown to be usable for atypical speech [22] and was successfully used for classifying AD from speech [23,24]. 3) WAV2VEC : in order to create audio representations using this approach, we make use of the huggingface 1 implementation of the wav2vec 2.0 [25] base model wav2vec2-base-960h.…”

Section: Feature Extractionmentioning

confidence: 99%

Impact of Acoustic Noise on Alzheimer's Disease Detection from Speech: Should You Let Baby Cry?

Novikova¹

2022

Preprint

View full text Add to dashboard Cite

Research related to automatically detecting Alzheimer's disease (AD) is important, given the high prevalence of AD and the high cost of traditional methods. Since AD significantly affects the acoustics of spontaneous speech, speech processing and machine learning (ML) provide promising techniques for reliably detecting AD. However, speech audio may be affected by different types of background noise and it is important to understand how the noise influences the accuracy of ML models detecting AD from speech. In this paper, we study the effect of fifteen types of noise from five different categories on the performance of four ML models trained with three types of acoustic representations. We perform a thorough analysis showing how ML models and acoustic features are affected by different types of acoustic noise. We show that acoustic noise is not necessarily harmful -certain types of noise are beneficial for AD detection models and help increasing accuracy by up to 4.8%. We provide recommendations on how to utilize acoustic noise in order to achieve the best performance results with the ML models deployed in real world.

show abstract

Automatic Detection and Assessment of Alzheimer Disease Using Speech and Language Technologies in Low-Resource Scenarios

Cited by 30 publications

References 23 publications

Non-Contrastive Self-Supervised Learning for Utterance-Level Information Extraction From Speech

Non-Contrastive Self-Supervised Learning for Utterance-Level Information Extraction From Speech

Multimodal Deep Learning Models for Detecting Dementia From Speech and Transcripts

Impact of Acoustic Noise on Alzheimer's Disease Detection from Speech: Should You Let Baby Cry?

Contact Info

Product

Resources

About