A Survey on Physiological Signal-Based Emotion Recognition

Abstract

Physiological signals are the most reliable form of signals for emotion recognition, as they cannot be controlled deliberately by the subject. Existing review papers on emotion recognition based on physiological signals surveyed only the regular steps involved in the workflow of emotion recognition such as pre-processing, feature extraction, and classification. While these are important steps, such steps are required for any signal processing application. Emotion recognition poses its own set of challenges that are very important to address for a robust system. Thus, to bridge the gap in the existing literature, in this paper, we review the effect of inter-subject data variance on emotion recognition, important data annotation techniques for emotion recognition and their comparison, data pre-processing techniques for each physiological signal, data splitting techniques for improving the generalization of emotion recognition models and different multimodal fusion techniques and their comparison. Finally, we discuss key challenges and future directions in this field.

Multimodal and Multidomain Feature Fusion for Emotion Classification Based on Electrocardiogram and Galvanic Skin Response Signals

“…Ahmad et al mentioned a gap in the literature regarding using fusion techniques to improve classification accuracy. Moreover, no standard set of features works for all situations, and methods must be developed to select the best features automatically [35]. Khateeb et al fused EEG signals' time, frequency, and wavelet domain features using concatenation before classification.…”

Enhancing the Potential of Machine Learning for Immersive Emotion Recognition in Virtual Environment

“…Controlled experiments, surveys, or the use of pre-existing emotion datasets can all be used to collect this data. The effectiveness and generalizability of the emotion recognition algorithms within immersive environments are guaranteed by careful data collection and annotation processes [24].…”

I DARE: IULM Dataset of Affective Responses

“…A recent survey (Ahmad and Khan, 2022 ) compared the stimuli, modalities, and sample size of the eight most popular datasets for the training of physiological-based AD models: AMIGOS (Miranda-Correa et al, 2018 ), ASCERTRAIN (Subramanian et al, 2016 ), BIO-VID-EMO DB (Zhang et al, 2016 ), DEAP (Koelstra et al, 2011 ), DREAMER (Katsigiannis and Ramzan, 2017 ), MAHNOB-HCI (Soleymani et al, 2011 ), MPED (Song et al, 2019 ), and SEED (Zheng et al, 2017 ). The stimuli, varying from 10 to 40 (M = 23.5, SD = 9.943) are all induced and spontaneous.…”