A Convolution Neural Network Based Emotion Recognition System using Multimodal Physiological Signals

Yang, Cheng-Jie; Fahier, Nicolas; Li, Wei-Chih; Fang, Wai-Chi

doi:10.1109/icce-taiwan49838.2020.9258341

Cited by 12 publications

(3 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For comparison of experimental results, the same dataset consisting of 1,100 samples for PPG data and 12 channels for EEG signal was used. In addition, time, frequency, and time-frequency domain characteristics were extracted from the data in the same way as the existing research method [27,46]. For the 1-channel PPG signal, a scalogram that could check time change according to the frequency band was used [47].…”

Section: B Comparison Of Bio-signal-based Precedent Studies and Resultsmentioning

confidence: 99%

“…Yang et al extracted EEG and PPG signals into 11 statistical time zones and five frequency zones and performed emotion classification using a 1D CNN. Furthermore, in our previous study, we calculated the pulse transit time (PTT) between the ECG and PPG signals to improve the final classification accuracy [27]. However, it required a minimum signal latency of 10 s and a maximum of 60 s for feature extraction and recognition of emotions.…”

Section: B Emotion Recognition and Classification Based On The Ansmentioning

confidence: 99%

See 1 more Smart Citation

1D Convolutional Autoencoder-Based PPG and GSR Signals for Real-Time Emotion Classification

Kang

Kim

2022

IEEE Access

View full text Add to dashboard Cite

To apply emotion recognition and classification technology to the field of human-robot interaction, it is necessary to implement fast data processing and model weight reduction. This paper proposes a new photoplethysmogram (PPG) and galvanic skin response (GSR) signals-based labeling method using Asian multimodal data, a real-time emotion classification method, a 1d convolutional neural network autoencoder model, and a lightweight model obtained using knowledge distillation. In addition, the model performance was verified using the public DEAP dataset and the Asian multi-modal dataset 'MERTI-Apps'. For emotion classification, bio-signal data were window-sliced in 1-pulse units, and the label was reset to reflect the characteristics of the PPG and GSR signals. Simple data pre-processing, such as the prevention of loss and waveform duplication, was performed without using handcrafted features. The experiment showed that the accuracy of the proposed model using MERTI-Apps was 79.18% and 74.84% in the case of arousal and valence, respectively, for 3-class criteria, and the accuracy of the proposed model using DEAP was 81.33% and 80.25% in the case of arousal and valence, respectively, for 2-class criteria. The accuracy of the lightweight model was 77.87% and 73.49% in the case of arousal and valence, respectively, for 3-class criteria and its calculation time was reduced by more than 80% compared to the proposed 1d convolutional autoencoder model. We also confirmed that the proposed model improved computational time and accuracy compared to previous studies using MERTI-Apps and the lightweight model used in limited hardware environments enabled fast computation and real-time emotion classification.

show abstract

Section: B Comparison Of Bio-signal-based Precedent Studies and Resultsmentioning

confidence: 99%

Section: B Emotion Recognition and Classification Based On The Ansmentioning

confidence: 99%

1D Convolutional Autoencoder-Based PPG and GSR Signals for Real-Time Emotion Classification

Kang

Kim

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Consequently, investigators utilize physiological signals for emotion classification. C. -J. Yang [7] aimed *Corresponding Author 1 Shanghai Jiao Tong University (SJTU) to generate an emotion recognition system based on the electrocardiogram (ECG) and photoplethysmography (PPG) signals for three emotional states including positive, negative, and neutral. Mahrukh [8] proposed an automatic labelsgenerating approach from movie subtitles and used brain fMRI images for positive, negative, and neural sentiment analysis.…”

Section: Introductionmentioning

confidence: 99%

Exploring Emotions in EEG: Deep Learning Approach with Feature Fusion

Tasaouf Mridula,

Ferdaus,

Pias

2023

Preprint

View full text Add to dashboard Cite

Emotion is an intricate physiological response that plays a crucial role in how we respond and cooperate with others in our daily affairs. Numerous experiments have been evolved to recognize emotion, however still require exploration to intensify the performance. To enhance the performance of effective emotion recognition, this study proposes a subject-dependent robust end-to-end emotion recognition system based on a 1D convolutional neural network (1D-CNN). We evaluate the SJTU1Emotion EEG Dataset SEED-V with five emotions (happy, sad, neural, fear, and disgust). To begin with, we utilize the Fast Fourier Transform (FFT) to decompose the raw EEG signals into six frequency bands and extract the power spectrum feature from the frequency bands. After that, we combine the extracted power spectrum feature with eye movement and differential entropy (DE) features. Finally, for classification, we apply the combined data to our proposed system. Consequently, it attains 99.80% accuracy which surpasses each prior state-of-the-art system.

show abstract

Biosignal based emotion-oriented video summarization

Derdiyok

Akbulut

2023

Multimedia Systems

View full text Add to dashboard Cite

A Convolution Neural Network Based Emotion Recognition System using Multimodal Physiological Signals

Cited by 12 publications

References 6 publications

1D Convolutional Autoencoder-Based PPG and GSR Signals for Real-Time Emotion Classification

1D Convolutional Autoencoder-Based PPG and GSR Signals for Real-Time Emotion Classification

Exploring Emotions in EEG: Deep Learning Approach with Feature Fusion

Biosignal based emotion-oriented video summarization

Contact Info

Product

Resources

About