Discriminating Emotions in the Valence Dimension from Speech Using Timbre Features

Abstract

The most used and well-known acoustic features of a speech signal, the Mel frequency cepstral coefficients (MFCC), cannot characterize emotions in speech sufficiently when a classification is performed to classify both discrete emotions (i.e., anger, happiness, sadness, and neutral) and emotions in valence dimension (positive and negative). The main reason for this is that some of the discrete emotions, such as anger and happiness, share similar acoustic features in the arousal dimension (high and low) but are different in the valence dimension. Timbre is a sound quality that can discriminate between two sounds even with the same pitch and loudness. In this paper, we analyzed timbre acoustic features to improve the classification performance of discrete emotions as well as emotions in the valence dimension. Sequential forward selection (SFS) was used to find the most relevant acoustic features among timbre acoustic features. The experiments were carried out on the Berlin Emotional Speech Database and the Interactive Emotional Dyadic Motion Capture Database. Support vector machine (SVM) and long short-term memory recurrent neural network (LSTM-RNN) were used to classify emotions. The significant classification performance improvements were achieved using a combination of baseline and the most relevant timbre acoustic features, which were found by applying SFS on a classification of emotions for the Berlin Emotional Speech Database. From extensive experiments, it was found that timbre acoustic features could characterize emotions sufficiently in a speech in the valence dimension.

Deep-Net: A Lightweight CNN-Based Speech Emotion Recognition System Using Deep Frequency Features

“…Furthermore, ref. [ 13 ] presented a SER based on the key segments to reduce the cost and increase the accuracy utilizing three challenging speech emotion datasets [ 41 ]. However, in this research, we proposed a novel CNN architecture for the SER based on the deep frequency pixel features to recognize the emotional features in the speech spectrograms.…”

The Acoustic Features and Didactic Function of Foreigner-Directed Speech: A Scoping Review

“…In studies exploring this question, participants were asked to listen to continuous speech samples and rate how positive or negative they sounded. The emotional valence of a speech signal is a complex combination of several acoustic features including, but not limited to, speech tempo, pitch height and range, and intensity (see Liscombe et al, 2003;Tursunov et al, 2019). While the perceived affect of a speech register can be directly related to speakers' desire to transmit emotion or to their communicative intent, it can also be a by-product of the exaggeration of prosodic and acoustic components intended to enhance a register's clarity or its didactic purpose.…”

EmoLexPros: A database of prosodically incongruent emotional words in European Portuguese

“…Among the timbral measures suggested by Tursunov et al (2019), spectral descriptors seemed particularly relevant. In addition, timbral descriptors were clearly more helpful to distinguish positive from neutral utterances than the other pairs (positive-negative and negative-neutral).…”