Towards Emotional Interaction: Using Movies to Automatically Learn Users’ Emotional States

Abstract

The HCI community is actively seeking novel methodologies to gain insight into the user's experience during interaction with both the application and the content. We propose an emotional recognition engine capable of automatically recognizing a set of human emotional states using psychophysiological measures of the autonomous nervous system, including galvanic skin response, respiration, and heart rate. A novel pattern recognition system, based on discriminant analysis and support vector machine classifiers is trained using movies' scenes selected to induce emotions ranging from the positive to the negative valence dimension, including happiness, anger, disgust, sadness, and fear. In this paper we introduce an emotion recognition system and evaluate its accuracy by presenting the results of an experiment conducted with three physiologic sensors.

Evaluating User Experience for Interactive Television: Towards the Development of a Domain-Specific User Experience Questionnaire

“…Dimensions that are addressed include emotions [36,43], social factors [15], values and requirements [5,42,44], the perception of the quality of the interaction or representation [53,47], or service quality and content [16,31]. These user experience evaluations have been applying standard UX methods, like the AttrakDiff questionnaire [45].…”

“…Experiences in gaming and TV applications were evaluated using psycho-physiological measurements [36]. These measurements were also used to evaluate the users' experiences with multi-view 3D displays [47] or to classify emotional reactions to video content [43]. Obrist et al [42] investigated users' requirement and experiences within an ethnographic study and identified patterns how iTV services can support people.…”

Accessing movies based on emotional impact

“…They used four physiological measures (heart rate, galvanic skin response, muscle activity and respiration) that were processed through the sequential floating feature selection (SFFS) algorithm used to choose best features of each physiological measure and linear discriminant analysis (LDA) to build a statistical model of each emotional class. Like Picard, we used sequential forward selection (a variant of sequential floating forward selection) [31].…”

“…The SVM classification score shows promise that the iFelt recognition system can be used to automatically evaluate human emotions. Two key positive aspects of the system emerged: (1) the use of easily computed statistical features, which can be used to develop real-time classification systems [31]; and (2) a quite reasonable recognition rate when compared with the other studies that also used movies (Table 1). This table summarizes most relevant studies regarding automatic emotion recognition [52], using different approaches, where we added the elicitation methods and the iFelt emotion recognition study [31].…”

“…Two key positive aspects of the system emerged: (1) the use of easily computed statistical features, which can be used to develop real-time classification systems [31]; and (2) a quite reasonable recognition rate when compared with the other studies that also used movies (Table 1). This table summarizes most relevant studies regarding automatic emotion recognition [52], using different approaches, where we added the elicitation methods and the iFelt emotion recognition study [31]. These studies are hardly comparable, since they used different types of sensors, a varied number of participants, different elicitation media and features and even different classification techniques.…”