Emotional Changes Detection for Dementia People with Spectrograms from Physiological Signals

Abstract

Due to aging society, there has recently been an increasing percentage of people with serious cognitive decline and dementia around the world. Such patients often lose their diversity of facial expressions and even their ability to speak, rendering them unable to express their feelings to their caregivers. However, emotions and feelings are strongly correlated with physiological signals, detectable with EEG and ECG etc. Therefore, this research develops an emotion predicting system for people with dementia using bio-signals to support their interaction with their caregivers. In this paper, we focused on a previous study for binary classification of emotional changes using spectrograms of EEG and RRI by CNN, verifying the effectiveness of the method. Firstly, the participants were required to watch simulating videos while collecting their EEG and ECG data. Then, STFT was performed, processing the raw data signals by extracting the time-frequency domain features to get the spectrograms. Finally, deep learning was used to detect the emotional changes. CNN was used for arousal classification, with an accuracy of 90.00% with EEG spectrograms, 91.67% with RRI spectrograms, and 93.33% with EEG and RRI spectrograms.

Emotion Detection Using EEG and ECG Signals from Wearable Textile Devices for Elderly People

“…The reasons for this are: LSTM has powerful ability to learn features from raw data directly; integrating EEG and ECG together provides more information for emotion detection; achieving high average accuracy over patients. The achievement of highly accurate on arousal [15] and valence classification in the study, suggests that emotion detection in dimensional emotion models, even discrete emotions, will yield good results.…”

“…The purpose of the research in our team is to detect emotions for the elderly patients who lose speaking ability using smart textile devices in real-time with comfort and convenience. In our previous studies, constructed neural networks were used to detect emotional changes on arousal level using EEG and RRI spectrograms on young subjects [15]; collecting signals with wet electrodes. Next, we developed wearable textile devices (textile EEG cap and ECG chest band) for capturing EEG and ECG signals of elderly people with ease.…”

“…In the late stage of dementia, the patients lose the ability to speak; preventing them from expressing their feelings to the caregivers. Our research aim is to detect the emotions of elderly people using physiological signals -EEG and ECG [15] -to improve the quality of care between care receivers and caregivers; emotions are fundamental to human life [16].…”

Evaluation of electrocardiogram: numerical vs. image data for emotion recognition system

“…For example, Ref. 15 determines the R-peaks of the electrocardiogram prior to generating the R-R interval (RRI) spectrogram. Following that, CNN was used to classify the emotions, with an accuracy rate greater than 90%.…”

“…Previous research on human emotions has primarily relied on either direct analysis of 1-D data 12 – 14 or the conversion of 1-D data to a 2-D spectral image 15 prior to identifying the emotions. Despite this, majority of the portable devices record the ECG signal as images (2-D images) in a PDF file rather than as raw numerical data (1-D data).…”