A Multimodal Database as a Background for Emotional Synthesis, Recognition and Training in E-Learning Systems

Abstract

Abstract. This paper presents a multimodal database developed within the EUfunded project MYSELF. The project aims at developing an e-learning platform endowed with affective computing capabilities for the training of relational skills through interactive simulations. The database includes data coming from 34 participants and concerning physiological parameters, vocal nonverbal features, facial expression and posture. Ten different emotions were considered (anger, joy, sadness, fear, contempt, shame, guilt, pride, frustration and boredom), ranging from primary to self-conscious emotions of particular relevance in learning process and interpersonal relationships. Preliminary results and analyses are presented, together with directions for future work.

Review of Studies on Emotion Recognition and Judgment Based on Physiological Signals

“…The results showed that it was necessary to use event-related features and emphasized the significance of improving the adaptive decision fusion method for emotion classification. Anolli et al [72] developed an E-learning system endowed with emotion computing ability for the cultivation of relational skills based on 10 different kinds of emotions detected from the facial expressions, vocal nonverbal features and postures of 34 subjects. Picard et al [73] developed a machine with the ability to recognize daily eight emotional states from EMG, EDA, blood volume pressure and respiration signals over multiple weeks.…”

Addressing Emotions within E-Learning Systems

“…All the modules provide information about behavior and physiological variations displayed by the user at a certain stage of the interaction: Variations should be processed by the inferential system in order to assess emotional state using an advanced statistical approach, including different learning and classification algorithms. To this end, it is critically necessary to define a training database, as a fundamental prerequisite for developing reliable classification algorithms (Anolli et al, 2005).…”