A Comparative Analysis of Modeling and Predicting Perceived and Induced Emotions in Sonification

Abstract

Sonification is the utilization of sounds to convey information about data or events. There are two types of emotions associated with sounds: (1) “perceived” emotions, in which listeners recognize the emotions expressed by the sound, and (2) “induced” emotions, in which listeners feel emotions induced by the sound. Although listeners may widely agree on the perceived emotion for a given sound, they often do not agree about the induced emotion of a given sound, so it is difficult to model induced emotions. This paper describes the development of several machine and deep learning models that predict the perceived and induced emotions associated with certain sounds, and it analyzes and compares the accuracy of those predictions. The results revealed that models built for predicting perceived emotions are more accurate than ones built for predicting induced emotions. However, the gap in predictive power between such models can be narrowed substantially through the optimization of the machine and deep learning models. This research has several applications in automated configurations of hardware devices and their integration with software components in the context of the Internet of Things, for which security is of utmost importance.

An Exhaustive Variable Selection Study for Linear Models of Soundscape Emotions: Rankings and Gibbs Analysis

“…However, time and frequency-domain features have been also included into the suggested models. These results are in line with other studies which also highlight that subjective perception and time-dynamics of the signals (jointly embedded in indicators) lead to better model scores [1], [20]. The valence model provides worse performance than arousal one, even involving more features.…”

“…In [2], a comparison of four LM and four NLM is explored and a dimension reduction is performed by a principal components analysis (PCA). 1 Furthermore, the authors in [10] also show that a RF model outperforms the rest of the models with only 25 features. In [1], a fine-tuned RF model with 14 features overcomes the previous RF model, and convolution neural networks (CNNs).…”

“…1 Furthermore, the authors in [10] also show that a RF model outperforms the rest of the models with only 25 features. In [1], a fine-tuned RF model with 14 features overcomes the previous RF model, and convolution neural networks (CNNs). Deep learning techniques have been also applied to SER through CNN and 23 simplified mel-frequency cepstral coefficients (MFCC) in [31], and the combination with SVM (Transfer learning) in [11].…”

“…Thus, soundscape emotion recognition (SER) requires further research to support perception and context descriptors [12], [25]. Additionally to environmental acoustics and urban planning, there is an increasing research interest in SER for certain domains like sound design in films and digital games [22], or sonification in the Internet of Things (IoT) [1]. Soundscapes descriptors are identified with perceived emotions and SER becomes a relatively new sub-field of research in affective computing [10].…”

“…The goal of this work is to design simple and interpretable SER linear models. Additionally, this study offers an 1 In order to avoid confusions, it is important to remark that the dimension reduction obtained by a PCA is different from a dimension reduction obtained by applying a variable selection scheme. The dimension reduction by PCA is obtained by suitable linear combinations of variables.…”