Robust Unsupervised Arousal Rating:A Rule-Based Framework withKnowledge-Inspired Vocal Features

Abstract

Studies in classifying affect from vocal cues have produced exceptional within-corpus results, especially for arousal (activation or stress); yet cross-corpora affect recognition has only recently garnered attention. An essential requirement of many behavioral studies is affect scoring that generalizes across different social contexts and data conditions. We present a robust, unsupervised (rule-based) method for providing a scale-continuous, bounded arousal rating operating on the vocal signal. The method incorporates just three knowledge-inspired features chosen based on empirical and theoretical evidence. It constructs a speaker’s baseline model for each feature separately, and then computes single-feature arousal scores. Lastly, it advantageously fuses the single-feature arousal scores into a final rating without knowledge of the true affect. The baseline data is preferably labeled as neutral, but some initial evidence is provided to suggest that no labeled data is required in certain cases. The proposed method is compared to a state-of-the-art supervised technique which employs a high-dimensional feature set. The proposed framework achieves highly-competitive performance with additional benefits. The measure is interpretable, scale-continuous as opposed to discrete, and can operate without any affective labeling. An accompanying Matlab tool is made available with the paper.

The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing

“…Further, speech and articulation rate was found to be important for all emotional expressions. For the case of automatic arousal recognition, [22] successfully builds an unsupervised recognition framework with these descriptors. [16] perform acoustic analysis of various fundamental frequency and harmonics related parameters 1. http://www.speech.kth.se/wavesurfer/ on a small set of emotional speech utterances.…”

Emotion Recognition in the Wild

“…Predictions on the test partition were submitted with the linear (% U AR = 32.16) and the gaussian (% U AR = 31.61) kernels, and we obtained again a better performance than the baseline (% U AR = 23.82); absolute improvement with the linear kernel is 8.34%. Therefore, a smaller, expert-knowledge based acoustic feature set shows higher robustness for emotion recognition than a large scale brute-force feature set, as found in [3]. …”

“…In contrast to large scale brute-force feature sets, which have been successfully applied to many speech and music classification tasks, e. g., [20,23], smaller, expert-knowledge based feature sets have shown high robustness for emotion recognition [3]. In this light, we assembled a small acoustic feature set for the EmotiW14 Challenge, using our openS-MILE toolkit [12].…”

Prediction of Emotion Change From Speech

“…Given the advantages of relative emotions over absolute emotions, one wonders whether changes in emotion ratings can be better predicted than absolute ratings. Secondly, despite the increasing popularity of predicting emotion dimensions either at utterance level Bone et al, 2014) or at frame level Metallinou et al, 2011;Nicolaou et al, 2011), all of the studies focus on prediction of absolute emotions across time. From these studies, it seems that predicting absolute emotion dimension remains challenging, and predicting absolute emotion alone may not provide insight into dynamic components, properties and regularities of emotion changes.…”