Multimodal Capture of Patient Behaviour for Improved Detection of Early Dementia: Clinical Feasibility and Preliminary Results

Jonell, Patrik; Moëll, Birger; Håkansson, Krister; Henter, Gustav Eje; Kucherenko, Taras; Mikheeva, Olga; Hagman, Göran; Holleman, Jasper; Kivipelto, Miia; Gustafson, Joakim; Beskow, Jonas

doi:10.3389/fcomp.2021.642633

Cited by 12 publications

(10 citation statements)

References 96 publications

(114 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Martinc et al ( 34 ) also used a multimodal approach to detect AD on the ADReSS dataset, using an active data representation approach ( 13 ), combining linguistic, acoustic, and temporal features and obtaining an accuracy of 93.75%. Jonell et al ( 35 ) recorded participants' language, speech, motor signs, pupil dilation, thermal emission, facial gestures, gaze, and heart rate variability of 25 patients with AD and found that multi-modality improved clinical discrimination. Recently, the transfer learning model has been widely used to diagnose AD.…”

Section: Related Workmentioning

confidence: 99%

A Transfer Learning Method for Detecting Alzheimer's Disease Based on Speech and Natural Language Processing

Liu

Luo

Yuan

et al. 2022

Front. Public Health

View full text Add to dashboard Cite

Alzheimer's disease (AD) is a neurodegenerative disease that is difficult to be detected using convenient and reliable methods. The language change in patients with AD is an important signal of their cognitive status, which potentially helps in early diagnosis. In this study, we developed a transfer learning model based on speech and natural language processing (NLP) technology for the early diagnosis of AD. The lack of large datasets limits the use of complex neural network models without feature engineering, while transfer learning can effectively solve this problem. The transfer learning model is firstly pre-trained on large text datasets to get the pre-trained language model, and then, based on such a model, an AD classification model is performed on small training sets. Concretely, a distilled bidirectional encoder representation (distilBert) embedding, combined with a logistic regression classifier, is used to distinguish AD from normal controls. The model experiment was evaluated on Alzheimer's dementia recognition through spontaneous speech datasets in 2020, including the balanced 78 healthy controls (HC) and 78 patients with AD. The accuracy of the proposed model is 0.88, which is almost equivalent to the champion score in the challenge and a considerable improvement over the baseline of 75% established by organizers of the challenge. As a result, the transfer learning method in this study improves AD prediction, which does not only reduces the need for feature engineering but also addresses the lack of sufficiently large datasets.

show abstract

Section: Related Workmentioning

confidence: 99%

A Transfer Learning Method for Detecting Alzheimer's Disease Based on Speech and Natural Language Processing

Liu

Luo

Yuan

et al. 2022

Front. Public Health

View full text Add to dashboard Cite

show abstract

“…The wearable biosensor offers many advantages, such as portability, comfort, convenience, and allowing for continuous point-of-care testing (Zhang et al, 2021a). At the same time, it Frontiers in Bioengineering and Biotechnology frontiersin.org has some drawbacks, for instance, some of them are bulky to wear, distracting, and cause tension (Jonell et al, 2021). As a result, in certain cases, some non-wearable biosensors are introduced to create an experimental environment with minimal interference to subjects and maintain the ecological validity of the recorded data, such as ambient-based in-home wireless sensors and Kinect (depth) sensors, etc.…”

Section: Body Motion Behavior Detection Of Alzheimer's Diseasementioning

confidence: 99%

“…Frontiers in Bioengineering and Biotechnology frontiersin.org clinical condition. In this way, the clinical feasibility of this sensor system is demonstrated by relating these digital biomarkers to traditional clinical assessment methods and established biomarkers (Jonell et al, 2021).…”

Section: Multimodal Detection Of Alzheimer's Diseasementioning

confidence: 99%

“…As a result, in certain cases, some non-wearable biosensors are introduced to create an experimental environment with minimal interference to subjects and maintain the ecological validity of the recorded data, such as ambient-based in-home wireless sensors and Kinect (depth) sensors, etc. ( Jonell et al, 2021 ). For example, Maselli et al designed a sensor-based tapestry to assess and train cognitive functions by combining the measurement of episodic memory and motor (upper limb articulation movement) in the tasks.…”

Section: Applications Of Biosensors Toward Behavior Detection Of Alzh...mentioning

confidence: 99%

See 1 more Smart Citation

Biosensors toward behavior detection in diagnosis of alzheimer’s disease

Sun

Sun²,

Wang³

et al. 2022

Front. Bioeng. Biotechnol.

View full text Add to dashboard Cite

In recent years, a huge number of individuals all over the world, elderly people, in particular, have been suffering from Alzheimer’s disease (AD), which has had a significant negative impact on their quality of life. To intervene early in the progression of the disease, accurate, convenient, and low-cost detection technologies are gaining increased attention. As a result of their multiple merits in the detection and assessment of AD, biosensors are being frequently utilized in this field. Behavioral detection is a prospective way to diagnose AD at an early stage, which is a more objective and quantitative approach than conventional neuropsychological scales. Furthermore, it provides a safer and more comfortable environment than those invasive methods (such as blood and cerebrospinal fluid tests) and is more economical than neuroimaging tests. Behavior detection is gaining increasing attention in AD diagnosis. In this review, cutting-edge biosensor-based devices for AD diagnosis together with their measurement parameters and diagnostic effectiveness have been discussed in four application subtopics: body movement behavior detection, eye movement behavior detection, speech behavior detection, and multi-behavior detection. Finally, the characteristics of behavior detection sensors in various application scenarios are summarized and the prospects of their application in AD diagnostics are presented as well.

show abstract

“…More recently, multimodal representations have been explored, combining linguistic and paralinguistic aspects of communication (Haider et al, 2020;Mahajan and Baths, 2021), as well as eye-tracking and other sensor modalities (Jonell et al, 2021). Those studies combined signal processing and machine learning to detect subtle acoustic signs of neurodegeneration which may be imperceptible to human diagnosticians.…”

Section: Related Workmentioning

confidence: 99%

Temporal Integration of Text Transcripts and Acoustic Features for Alzheimer's Diagnosis Based on Spontaneous Speech

Martinc

Haider

Pollak

et al. 2021

Front. Aging Neurosci.

View full text Add to dashboard Cite

Background: Advances in machine learning (ML) technology have opened new avenues for detection and monitoring of cognitive decline. In this study, a multimodal approach to Alzheimer's dementia detection based on the patient's spontaneous speech is presented. This approach was tested on a standard, publicly available Alzheimer's speech dataset for comparability. The data comprise voice samples from 156 participants (1:1 ratio of Alzheimer's to control), matched by age and gender.Materials and Methods: A recently developed Active Data Representation (ADR) technique for voice processing was employed as a framework for fusion of acoustic and textual features at sentence and word level. Temporal aspects of textual features were investigated in conjunction with acoustic features in order to shed light on the temporal interplay between paralinguistic (acoustic) and linguistic (textual) aspects of Alzheimer's speech. Combinations between several configurations of ADR features and more traditional bag-of-n-grams approaches were used in an ensemble of classifiers built and evaluated on a standardised dataset containing recorded speech of scene descriptions and textual transcripts.Results: Employing only semantic bag-of-n-grams features, an accuracy of 89.58% was achieved in distinguishing between Alzheimer's patients and healthy controls. Adding temporal and structural information by combining bag-of-n-grams features with ADR audio/textual features, the accuracy could be improved to 91.67% on the test set. An accuracy of 93.75% was achieved through late fusion of the three best feature configurations, which corresponds to a 4.7% improvement over the best result reported in the literature for this dataset.Conclusion: The proposed combination of ADR audio and textual features is capable of successfully modelling temporal aspects of the data. The machine learning approach toward dementia detection achieves best performance when ADR features are combined with strong semantic bag-of-n-grams features. This combination leads to state-of-the-art performance on the AD classification task.

show abstract

Multimodal Capture of Patient Behaviour for Improved Detection of Early Dementia: Clinical Feasibility and Preliminary Results

Cited by 12 publications

References 96 publications

A Transfer Learning Method for Detecting Alzheimer's Disease Based on Speech and Natural Language Processing

A Transfer Learning Method for Detecting Alzheimer's Disease Based on Speech and Natural Language Processing

Biosensors toward behavior detection in diagnosis of alzheimer’s disease

Temporal Integration of Text Transcripts and Acoustic Features for Alzheimer's Diagnosis Based on Spontaneous Speech

Contact Info

Product

Resources

About