Electrocardiogram‐based emotion recognition system using empirical mode decomposition and discrete Fourier transform

Abstract

Emotion recognition using physiological signals has gained momentum in the field of human computer–interaction. This work focuses on developing a user‐independent emotion recognition system that would classify five emotions (happiness, sadness, fear, surprise and disgust) and neutral state. The various stages such as design of emotion elicitation protocol, data acquisition, pre‐processing, feature extraction and classification are discussed. Emotional data were obtained from 30 undergraduate students by using emotional video clips. Power and entropy features were obtained in three ways – by decomposing and reconstructing the signal using empirical mode decomposition, by using a Hilbert–Huang transform and by applying a discrete Fourier transform to the intrinsic mode functions (IMFs). Statistical analysis using analysis of variance indicates significant differences among the six emotional states (p < 0.001). Classification results indicate that applying the discrete Fourier transform instead of the Hilbert transform to the IMFs provides comparatively better accuracy for all the six classes with an overall accuracy of 52%. Although the accuracy is less, it reveals the possibility of developing a system that could identify the six emotional states in a user‐independent manner using electrocardiogram signals. The accuracy of the system can be improved by investigating the power and entropy of the individual IMFs.

An Ensemble Learning Approach for Electrocardiogram Sensor Based Human Emotion Recognition

“…Numerous studies have attempted to use different types of bio-signals for detecting emotions [9,10,27]. Kim and André [8] developed an emotion recognition model incorporating four different biosensors: electrocardiogram, skin conductivity, electromyogram and respiration.…”

“…A number of studies have examined the use of ECG signals for emotion recognition [7,10,27,34,35]. An ECG based method is an adequate solution due to four important reasons.…”

“…Some of the studies have used discrete Fourier transform (DFT) to extract frequency domain features from the ECG signal. Jerritta et al [10] discuss the advantages of using frequency domain features compared to EMD based features. They claim that, unlike EMD features that provide an idea about the local properties of the ECG wave, the DFT method provides information about the frequency content of the signal.…”

“…Some of the widely used bio-signals include the electrocardiogram (ECG), galvanic skin response (GSR), electromyogram and respiration. Some studies have combined multiple bio-signal devices to recognize emotions, whereas other studies have used a single biosensor such as ECG for capturing data for emotion recognition [8,9,10]. Despite the increased recognition accuracy, using multiple sensors might lead to user dissatisfaction.…”

“…Previous studies conducted to investigate the use of ECG signals have used different types of feature extraction methods. Some of those methods include heart rate variability (HRV), empirical mode decomposition (EMD), with-in beat analysis (WIB) and wavelet transforms [8,10]. Although those feature extraction methods are sophisticated methods, they suffer from low prediction accuracy.…”