CorrNet: Fine-Grained Emotion Recognition for Video Watching Using Wearable Physiological Sensors

Zhang, Tianyi; Ali, Abdallah El; Wang, Chen; Hanjalic, Alan; César, Pablo

doi:10.3390/s21010052

Cited by 40 publications

(43 citation statements)

References 109 publications

(143 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Out of the studies in this research area, autoencoder is a widely used technique for unsupervised representation learning. Recently published CorrNet [16] uses autoencoder based automatic feature extraction in a wearable signal-based emotion recognition task and outperforms the state-of-the-art baseline for CASE dataset for arousal (74.03%) and valence (76.37%) detection. Martinez et al [13] deploy a denoising autoencoder network to learn features from blood volume pulse (BVP) and electrodermal activity (EDA) signals and reuse the learned features to classify affective states.…”

Section: A Feature Engineering and Representation Learning For Emotio...mentioning

confidence: 99%

“…However, in the literature [16], researchers have binned nine levels into two class and three class configurations for evaluation. For the comparison purpose, we follow the class configuration proposed by Zhang et al [16] 1 https:// www.empatica.com/ research/ e4/ 2 https:// wearabletech.io/ zephyr-bioharness-3/ in our evaluations. We use the CASE for signal representation learning as well as the emotion recognition tasks.…”

Section: Experimental Setup a Datasetsmentioning

confidence: 99%

“…We train representations using all eight proxy tasks and use them in the downstream emotion recognition task. Initially, we use data chunks with 30 seconds Prior work [16] suggests sampling with smaller window sizes results in better emotion recognition accuracy in the context of the wearable signal-based emotion recognition. Therefore we attempt to evaluate with a smaller sample size.…”

Section: Self-supervised Benchmarkmentioning

confidence: 99%

See 2 more Smart Citations

SigRep: Toward Robust Wearable Emotion Recognition With Contrastive Representation Learning

et al. 2022

View full text Add to dashboard Cite

Extracting emotions from physiological signals has become popular over the past decade. Recent advancements in wearable smart devices have enabled capturing physiological signals continuously and unobtrusively. However, signal readings from different smart wearables are lossy due to user activities, making it difficult to develop robust models for emotion recognition. Also, the limited availability of data labels is an inherent challenge for developing machine learning techniques for emotion classification. This paper presents a novel self-supervised approach inspired by contrastive learning to address the above challenges. In particular, our proposed approach develops a method to learn representations of individual physiological signals, which can be used for downstream classification tasks. Our evaluation with four publicly available datasets shows that the proposed method surpasses the emotion recognition performance of state-of-the-art techniques for emotion classification. In addition, we show that our method is more robust to losses in the input signal.

show abstract

Section: A Feature Engineering and Representation Learning For Emotio...mentioning

confidence: 99%

Section: Experimental Setup a Datasetsmentioning

confidence: 99%

Section: Self-supervised Benchmarkmentioning

confidence: 99%

See 1 more Smart Citation

SigRep: Toward Robust Wearable Emotion Recognition With Contrastive Representation Learning

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Nowadays, there are even systems that perform the analysis of biomarkers from sweat and saliva directly during sports activities [ 211 ]. The current literature summary of stress assessment using wearable multi-sensors in the natural environment includes: emotion recognition by neural networks from portable eyetracker and Empatica E4 [ 212 ], ANS research using again E4 but now with ECG and respiration sensors [ 213 ], development of cognitive load tracker using machine learning [ 109 ], smart stress reduction system using E4 combined with accelerometers [ 214 ], validation of wireless sensors for psychophysiological studies and stress detection [ 100 , 215 ], prediction of relative physical activity [ 216 ], real-time monitoring of passenger psychological stress [ 147 ], classification of calm/distress condition [ 217 ], assessment of mental stress of fighters [ 218 ], and others. A comprehensive overview about pain and stress detection using available wearable sensors was actually made very recently by Jerry Chen et al [ 150 ].…”

Section: Advanced Wearable Stress-metersmentioning

confidence: 99%

The Concept of Advanced Multi-Sensor Monitoring of Human Stress

Vavrinský

Stopjaková

Kopáni

et al. 2021

Sensors

View full text Add to dashboard Cite

Many people live under stressful conditions which has an adverse effect on their health. Human stress, especially long-term one, can lead to a serious illness. Therefore, monitoring of human stress influence can be very useful. We can monitor stress in strictly controlled laboratory conditions, but it is time-consuming and does not capture reactions, on everyday stressors or in natural environment using wearable sensors, but with limited accuracy. Therefore, we began to analyze the current state of promising wearable stress-meters and the latest advances in the record of related physiological variables. Based on these results, we present the concept of an accurate, reliable and easier to use telemedicine device for long-term monitoring of people in a real life. In our concept, we ratify with two synchronized devices, one on the finger and the second on the chest. The results will be obtained from several physiological variables including electrodermal activity, heart rate and respiration, body temperature, blood pressure and others. All these variables will be measured using a coherent multi-sensors device. Our goal is to show possibilities and trends towards the production of new telemedicine equipment and thus, opening the door to a widespread application of human stress-meters.

show abstract

“…These responses can be measured using physiological sensors, thus giving us an objective window into the realm of emotions. In recent years, advances in Deep Learning (DL) techniques has enabled researchers to directly model the mappings between physiological signals (e.g., galvanic skin response (GSR) and electrocardiogram (ECG)) and human emotions [5,6], without resorting to crafting features manually that require expert knowledge.…”

Section: Introductionmentioning

confidence: 99%

AC-WGAN-GP: Augmenting ECG and GSR Signals using Conditional Generative Models for Arousal Classification

Furdui

Zhang

Worring

et al. 2021

Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of The

Self Cite

View full text Add to dashboard Cite

Computational recognition of human emotion using Deep Learning techniques requires learning from large collections of data. However, the complex processes involved in collecting and annotating physiological data lead to datasets with small sample sizes. Models trained on such limited data often do not generalize well to realworld settings. To address the problem of data scarcity, we use an Auxiliary Conditioned Wasserstein Generative Adversarial Network with Gradient Penalty (AC-WGAN-GP) to generate synthetic data. We compare the recognition performance between real and synthetic signals as training data in the task of binary arousal classification. Experiments on GSR and ECG signals show that generative data augmentation significantly improves model performance (avg. 16.5%) for binary arousal classification in a subject-independent setting. CCS CONCEPTS• Computing methodologies → Machine learning; • Humancentered computing → Human computer interaction (HCI).

show abstract

CorrNet: Fine-Grained Emotion Recognition for Video Watching Using Wearable Physiological Sensors

Cited by 40 publications

References 109 publications

SigRep: Toward Robust Wearable Emotion Recognition With Contrastive Representation Learning

SigRep: Toward Robust Wearable Emotion Recognition With Contrastive Representation Learning

The Concept of Advanced Multi-Sensor Monitoring of Human Stress

AC-WGAN-GP: Augmenting ECG and GSR Signals using Conditional Generative Models for Arousal Classification

Contact Info

Product

Resources

About