Personalized Emotion Recognition by Personality-Aware High-Order Learning of Physiological Signals

Abstract

Due to the subjective responses of different subjects to physical stimuli, emotion recognition methodologies from physiological signals are increasingly becoming personalized. Existing works mainly focused on modeling the involved physiological corpus of each subject, without considering the psychological factors, such as interest and personality. The latent correlation among different subjects has also been rarely examined. In this article, we propose to investigate the influence of personality on emotional behavior in a hypergraph learning framework. Assuming that each vertex is a compound tuple (subject, stimuli), multi-modal hypergraphs can be constructed based on the personality correlation among different subjects and on the physiological correlation among corresponding stimuli. To reveal the different importance of vertices, hyperedges, and modalities, we learn the weights for each of them. As the hypergraphs connect different subjects on the compound vertices, the emotions of multiple subjects can be simultaneously recognized. In this way, the constructed hypergraphs are vertex-weighted multi-modal multi-task ones. The estimated factors, referred to as emotion relevance, are employed for emotion recognition. We carry out extensive experiments on the ASCERTAIN dataset and the results demonstrate the superiority of the proposed method, as compared to the state-of-the-art emotion recognition approaches.

Personality-Based Affective Adaptation Methods for Intelligent Systems

“…We discovered several significant links between different personality traits, as measured by the NEO-FFI questionnaire, and the characteristics of psychophysiological reactions (ECG, GSR). These are in line with the existing research [ 33 ]. Furthermore, we discovered links between personality traits and reactions to complex stimulus environment (games).…”

“…The authors also presented the preliminary results on their data sets indicating the existence of significant relationships between personality factors and the characteristics of physiological signals. Further analysis of the ASCERTAIN collection led to the creation of a model that was based on hypergraph learning demonstrating the usefulness of personality assessment in the prediction of emotions [ 33 ].…”

“…Several significant associations between game activities and personality traits are a promising result. It indicates that personality can be useful in assessing emotions, not only with simple stimuli, such as movies [ 33 ], but also in a more complex experimental environment, like games. The most interesting seems to be the extroversion, generally connected to the increase of activity in game (i.e., number of shots, number of killed enemies).…”

Deep Transfer Learning for Image Emotion Analysis: Reducing Marginal and Joint Distribution Discrepancies Together

“…Therefore, emotion researchers pay more attention from the psychology analysis to the image emotion analysis. Some research works even extent the analysis from dominant emotion to personalized emotion [32,34,35]. Traditional method classifying emotion contained in images based on low-level crafted features [8,17,19,37].…”

The Development of Web-Based Emotion Detection System Using Keyboard Actions (EDS-KA)

“…• Psychological approaches such as self-reports (verbal explanations of an emotional state, rating scales, and graphical differentials methods). The physiological approach and its non-verbal equipment have been applied in several previous studies such as Li, Xu and Feng [11], Balters and Steinert [12], Zhao et al [13]. The main benefits of this approach are as follows:…”