Use of Agreement/Disagreement Classification in Dyadic Interactions for Continuous Emotion Recognition

Abstract

Natural and affective handshakes of two participants define the course of dyadic interaction. Affective states of the participants are expected to be correlated with the nature or type of the dyadic interaction. In this study, we investigate relationship between affective attributes and nature of dyadic interaction. In this investigation we use the JESTKOD database, which consists of speech and full-body motion capture data recordings for dyadic interactions under agreement and disagreement scenarios. The dataset also has affective annotations in activation, valence and dominance (AVD) attributes. We pose the continuous affect recognition problem under agreement and disagreement scenarios of dyadic interactions. We define a statistical mapping using the support vector regression (SVR) from speech and motion modalities to affective attributes with and without the dyadic interaction type (DIT) information. We observe an improvement in estimation of the valence attribute when the DIT is available. Furthermore this improvement sustains even we estimate the DIT from the speech and motion modalities of the dyadic interaction.

Cross-Subject Continuous Emotion Recognition Using Speech and Body Motion in Dyadic Interactions

“…Subsequently, we exploit Gaussian Mixture Regression (GMR) to construct a statistical mapping between the underlying observed summarized speech, motion and multimodal features and the hidden window-level emotional attributes, a. After GMR, the estimated attribute,â, is low-pass filtered for smoothing as in [11,14].…”

“…Continuous emotion recognition (CER) has increasingly attracted interest over the past few years [5,6]. Speech and motion modality information are extensively used for continuous emotion recognition [7,8,9] based on a wide variety of techniques such as k-means clustering, support vector regression, neural networks and Gaussian mixture model (GMM) regression [10,11,12,13,14]. The prevalent challenges in CER to date necessitate improved techniques because the estimated performance is context-dependent and varies between databases.…”