Physiological-Based Emotion Detection and Recognition in a Video Game Context

Abstract

Affective gaming is a hot field of research that exploits human emotion for the enhancement of player's experience during gameplay. Physiological signal is an effective modality that can provide a better understanding of the emotional states and is very promising to be applied to affective gaming. Most physiological-based affective gaming applications evaluate player's emotion on an overall game fragment. These approaches fail to capture the emotion change in the dynamic game context. In order to achieve a better understanding of psychophysiological response with a better time sensitivity, we present a study that evaluates the psychophysiological responses related to the game events. More specifically, we present a multi-modal database DAG that contains peripheral physiological signals (ECG, EDA, respiration, EMG, temperature), accelerometer signals, facial and screening recordings as well as player's self-reported eventrelated emotion assessment through game playing. We then investigate physiological-based emotion detection and recognition by using machine learning techniques. Common challenges for physiological-based affective model such as signal segmentation, feature normalization, relevant features are addressed. We also discuss factors that influence the performance of the affective models.

A Review, Current Challenges, and Future Possibilities on Emotion Recognition Using Machine Learning and Physiological Signals

“…Within the state-of-the-art, affect elicitation is commonly performed via pictures [28], films [29], VR videos [30], games [31]- [36], music videos [21], sound [37], [38], words [39], recall [40]- [43] or in well controlled settings, although real-world scenarios have started to be explored [44].…”

Feature Extraction and Selection for Pain Recognition Using Peripheral Physiological Signals

“…In particular, the CC features involved computing a statistical difference in a physiological signal between a pain-free state (from a baseline measure) and an unknown state measurement window. In a study of Yang et al (2018), arousal and valence classification was also improved when features were normalized using the difference between an annotation segment and the precedent before, or a neutral state baseline segment. Through this adaptive normalization process, the interactions between dependent variables (autonomic parameters) and random variable (between-subject effect, between trial effect) are minimized, resulting in a direct measurable relationship between the dependent variable and independent variable (pain).…”

Electrocardiogram-Based Emotion Recognition Systems and Their Applications in Healthcare—A Review

“…There are five common elicitation techniques which are audio visual, imagery, music, memory recall, and the situational procedure [ 50 ]. The less common approaches are naturalistic conversations or debates [ 51 ], driving [ 52 ], video games [ 53 ], and virtual reality [ 54 ].…”

“…Considering that one cycle or one beat can be measured between two successive R peaks, the HR can be derived simply through averaging the overall signals collected through a period. The HR is proven to show distinct feature changes [ 87 ] and has been used in various ECG-based affective studies [ 6 , 24 , 39 , 51 , 53 , 55 , 56 , 57 , 67 , 71 , 75 , 78 , 85 , 88 ]. The benefits of HR over other features are the simplicity of the calculation and not requiring a highly accurate measurement.…”

“…The applications of PCA were viewed in [ 20 , 55 , 67 , 85 , 89 , 108 , 111 ]. LDA, also known as Fisher’s linear discriminant analysis, was used in [ 20 , 24 , 53 , 87 ] as a dimension reduction procedure.…”