Measuring Cognitive Status from Speech in a Smart Home Environment

Fraser, Kathleen; Komeili, Majid

doi:10.1109/mim.2021.9513645

Cited by 4 publications

(3 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…AD disrupts morphological, lexical, semantic, syntactic and structural features of language [27], and can even be detected in acoustic, temporal and other paralinguistic features [28][29][30]. Speech is simple to acquire using modern smart devices and therefore is an attractive target for longitudinal in-home monitoring of cognitive decline [31]. Advances in automated speech recognition (ASR), natural language processing (NLP) and other ML technologies have contributed to promising results in classifying AD from recorded speech [32], and recent work has also shown that linguistic and paralinguistic features can help detect MCI [33].…”

Section: Introductionmentioning

confidence: 99%

The use of home-based conversations for detecting early dementia: Protocol for the CUBOId TV task

Kumpik

Santos-Rodríguez

Selwood

et al. 2022

Preprint

View full text Add to dashboard Cite

IntroductionLimitations in effective dementia therapies mean that early diagnosis and monitoring are critical for disease management, but current clinical tools are impractical and/or unreliable, and disregard short-term symptom variability. Behavioural biomarkers of cognitive decline, such as speech, sleep and activity patterns, can manifest prodromal pathological changes. They can be continuously measured at home with smart sensing technologies, and permit leveraging of interpersonal interactions for optimising diagnostic and prognostic performance. Here we describe the ContinUous behavioural Biomarkers Of cognitive Impairment (CUBOId) study, which explores the feasibility of multimodal data fusion for in-home monitoring of mild cognitive impairment (MCI) and early Alzheimer’s disease (AD). The report focuses on a subset of CUBOId participants who perform a novel speech task, the “TV task”, designed to track changes in ecologically valid conversations with disease progression.Methods and AnalysisParticipants have diagnoses of MCI or AD, and controls are their live-in partners with no such diagnosis. Multimodal activity data were passively acquired from wearables and in-home fixed sensors over periods ranging from 8 to 25 months. At 2 time points participants completed the TV task over 5 days by recording audio of their conversations as they watched a favourite TV programme, with further testing to be completed after removal of the sensor installations. Behavioural testing is supported by neuropsychological assessment for deriving behavioural ground truths on cognitive status. Deep learning will be used to generate fused multimodal activity-speech embeddings for optimisation of diagnostic and predictive performance from speech alone.Ethics and disseminationCUBOId was approved by an NHS Research Ethics Committee (Wales REC; ref: 18/WA/0158) and is sponsored by University of Bristol. It is also supported by the National Institute for Health Research (NIHR) Clinical Research Network West of England. Results will be reported at conferences and in peer-reviewed scientific journals.ARTICLE SUMMARYStrengths and limitationsTo our knowledge, this is the first study to simultaneously characterise longitudinal, ecologically valid diagnostic trajectories across a broad range of behavioural domains relevant to cognitive decline “in the wild”, allowing construction of shared multimodal embeddings for dementia diagnosis from speech aloneParticipants’ live-in partners are contextually matched controls, accounting for differences across participants’ home environments and lifestylesMeasurements of cognitive status from neuropsychological testing form ground truths for behavioural biomarkers of cognitive declineLimited sample size and restricted demographics may confer limited generalisability of findings to other populationsBehavioural and neuropsychological testing timelines disrupted and desynchronised due to COVID-19

show abstract

Section: Introductionmentioning

confidence: 99%

The use of home-based conversations for detecting early dementia: Protocol for the CUBOId TV task

Kumpik

Santos-Rodríguez

Selwood

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“… 29–31 Speech is simple to acquire using modern smart devices and therefore is an attractive target for longitudinal in-home monitoring of cognitive decline. 32 33 Advances in automated speech recognition, natural language processing (NLP) and other ML technologies have contributed to promising results in classifying AD from recorded speech, 34 and recent work has also shown that linguistic and paralinguistic features can help detect MCI. 35 However, classification remains more challenging for MCI than AD, probably because MCI patients’ cognitive profiles are heterogenous and not as far removed from those of normal ageing.…”

Section: Introductionmentioning

confidence: 99%

“…AD disrupts morphological, lexical, semantic, syntactic and structural features of language,28 and can even be detected in acoustic, temporal and other paralinguistic features 29–31. Speech is simple to acquire using modern smart devices and therefore is an attractive target for longitudinal in-home monitoring of cognitive decline 32 33. Advances in automated speech recognition, natural language processing (NLP) and other ML technologies have contributed to promising results in classifying AD from recorded speech,34 and recent work has also shown that linguistic and paralinguistic features can help detect MCI 35.…”

Section: Introductionmentioning

confidence: 99%

A longitudinal observational study of home-based conversations for detecting early dementia: protocol for the CUBOId TV task

et al. 2022

View full text Add to dashboard Cite

IntroductionLimitations in effective dementia therapies mean that early diagnosis and monitoring are critical for disease management, but current clinical tools are impractical and/or unreliable, and disregard short-term symptom variability. Behavioural biomarkers of cognitive decline, such as speech, sleep and activity patterns, can manifest prodromal pathological changes. They can be continuously measured at home with smart sensing technologies, and permit leveraging of interpersonal interactions for optimising diagnostic and prognostic performance. Here we describe the ContinUous behavioural Biomarkers Of cognitive Impairment (CUBOId) study, which explores the feasibility of multimodal data fusion for in-home monitoring of mild cognitive impairment (MCI) and early Alzheimer’s disease (AD). The report focuses on a subset of CUBOId participants who perform a novel speech task, the ‘TV task’, designed to track changes in ecologically valid conversations with disease progression.Methods and analysisCUBOId is a longitudinal observational study. Participants have diagnoses of MCI or AD, and controls are their live-in partners with no such diagnosis. Multimodal activity data were passively acquired from wearables and in-home fixed sensors over timespans of 8–25 months. At two time points participants completed the TV task over 5 days by recording audio of their conversations as they watched a favourite TV programme, with further testing to be completed after removal of the sensor installations. Behavioural testing is supported by neuropsychological assessment for deriving ground truths on cognitive status. Deep learning will be used to generate fused multimodal activity-speech embeddings for optimisation of diagnostic and predictive performance from speech alone.Ethics and disseminationCUBOId was approved by an NHS Research Ethics Committee (Wales REC; ref: 18/WA/0158) and is sponsored by University of Bristol. It is supported by the National Institute for Health Research Clinical Research Network West of England. Results will be reported at conferences and in peer-reviewed scientific journals.

show abstract

A Novel Telerehabilitation Approach for Cognitive-Language Therapy in Chronic Stroke Subjects With Aphasia: Neurocognitive Intervention Through Neurorehabilitation Exercises and Assessments

Infarinato¹,

Romano²,

Adinolfi

et al. 2023

IEEE Robot. Automat. Mag.

View full text Add to dashboard Cite

Measuring Cognitive Status from Speech in a Smart Home Environment

Cited by 4 publications

References 35 publications

The use of home-based conversations for detecting early dementia: Protocol for the CUBOId TV task

The use of home-based conversations for detecting early dementia: Protocol for the CUBOId TV task

A longitudinal observational study of home-based conversations for detecting early dementia: protocol for the CUBOId TV task

A Novel Telerehabilitation Approach for Cognitive-Language Therapy in Chronic Stroke Subjects With Aphasia: Neurocognitive Intervention Through Neurorehabilitation Exercises and Assessments

Contact Info

Product

Resources

About