FILTWAM and Voice Emotion Recognition

Abstract

Abstract. This paper introduces the voice emotion recognition part of our framework for improving learning through webcams and microphones (FILTWAM). This framework enables multimodal emotion recognition of learners during game-based learning. The main goal of this study is to validate the use of microphone data for a real-time and adequate interpretation of vocal expressions into emotional states were the software is calibrated with end users. FILTWAM already incorporates a valid face emotion recognition module and is extended with a voice emotion recognition module. This extension aims to provide relevant and timely feedback based upon learner's vocal intonations. The feedback is expected to enhance learner's awareness of his or her own behavior. Six test persons received the same computer-based tasks in which they were requested to mimic specific vocal expressions. Each test person mimicked 82 emotions, which led to a dataset of 492 emotions. All sessions were recorded on video. An overall accuracy of our software based on the requested emotions and the recognized emotions is a pretty good 74.6% for the emotions happy and neutral emotions; but will be improved for the lower values of an extended set of emotions. In contrast with existing software our solution allows to continuously and unobtrusively monitor learners' intonations and convert these intonations into emotional states. This paves the way for enhancing the quality and efficacy of game-based learning by including the learner's emotional states, and links these to pedagogical scaffolding.

Vocal-based emotion recognition using random forests and decision tree

“…It has numerous applications in many fields such as business, entertainment (El Ayadi et al 2011), call centre environments (Petrushin 2000), games (Bahreini et al 2013) and education (Cowie et al 2001). Besides, the correct perception of the emotions can encourage the timely detection of diseases that are known to affect the expressions of the feelings (Yoon and Park 2007;Petrushin 2000;Zeng et al 2007).…”