Emotion Classification Based on Biophysical Signals and Machine Learning Techniques

Abstract

Emotions constitute an indispensable component of our everyday life. They consist of conscious mental reactions towards objects or situations and are associated with various physiological, behavioral, and cognitive changes. In this paper, we propose a comparative analysis between different machine learning and deep learning techniques, with and without feature selection, for binarily classifying the six basic emotions, namely anger, disgust, fear, joy, sadness, and surprise, into two symmetrical categorical classes (emotion and no emotion), using the physiological recordings and subjective ratings of valence, arousal, and dominance from the DEAP (Dataset for Emotion Analysis using EEG, Physiological and Video Signals) database. The results showed that the maximum classification accuracies for each emotion were: anger: 98.02%, joy:100%, surprise: 96%, disgust: 95%, fear: 90.75%, and sadness: 90.08%. In the case of four emotions (anger, disgust, fear, and sadness), the classification accuracies were higher without feature selection. Our approach to emotion classification has future applicability in the field of affective computing, which includes all the methods used for the automatic assessment of emotions and their applications in healthcare, education, marketing, website personalization, recommender systems, video games, and social media.In the 1970s, Paul Ekman identified six basic emotions [2], namely anger, disgust, fear, joy, sadness, and surprise. Russell and Mehrabian proposed a dimensional approach [3] which states that any emotion is represented relative to three fundamental dimensions, namely valence (positive/pleasurable or negative/unpleasurable), arousal (engaged or not engaged), and dominance (degree of control that a person has over their affective states).Joy or happiness is a pleasant emotional state, synonymous with contentment, satisfaction and well-being. Sadness is the opposite of happiness, being characterized by grief, disappointment, and distress. Fear emerges in the presence of a stressful or dangerous stimulus perceived by the sensory organs. When the fight or flight response appears, heart rate and respiration rate increase. Also, the muscles become more tense in order to contend with threats in the environment. Anger is defined by fury, frustration, and resentment towards others. Surprise is triggered by an unexpected outcome to a situation, ranging from amazement to shock, whereas disgust is synonymous with dislike, distaste, or repugnance, being the most visceral of all six emotions.The DEAP database [4] was created with the purpose of developing a music video recommendation system based on the users' emotional responses. The biophysical signals of 32 subjects have been recorded while they were watching 40 one-minute long excerpts of music videos eliciting various emotions. The participants rated each video in terms of valence, arousal, dominance, like/dislike and familiarity on a scale from one to nine. The physiological signals were: galvanic skin response (GSR), plethysmograph (PP...

An Investigation of Various Machine and Deep Learning Techniques Applied in Automatic Fear Level Detection and Acrophobia Virtual Therapy

“…A more detailed investigation of ML techniques used in emotions classification was performed in [95].…”

Expressure: Detect Expressions Related to Emotional and Cognitive Activities Using Forehead Textile Pressure Mechanomyography

“…EMG placed at the temples has shown to detect a smile with around 90% accuracy in [60]. EEG has also been studied for emotion detection with above 90% accuracy among four emotions in [61]. We argue that while EEG or EMG relies on stable electrode contact with the skin as mentioned in Section 1, the proposed Expressure approach does not require any electrical skin contact since the muscle activity is mechanically coupled with the pressure sensors.…”

Cognitive and Affective Assessment of Navigation and Mobility Tasks for the Visually Impaired via Electroencephalography and Behavioral Signals

“…In terms of evaluating emotions, it is well known that the amygdala is responsible for the perception of emotions, such as anger, fear, and sadness. The pre-frontal cortex and the hippocampus (located in the medial region of the temporal lobe) are highly correlated to emotional activity [ 56 , 61 ]. Because the right hemisphere is associated with negative emotions (i.e., fear or disgust), and the left hemisphere is highly activated by positive emotions and motivation (i.e., happiness and satisfaction), the EEG asymmetries in the frontal and parietal lobes are relevant for valence and arousal assessment [ 56 ].…”