A Review on Five Recent and Near-Future Developments in Computational Processing of Emotion in the Human Voice

Schuller, Dagmar; Schuller, Björn W.

doi:10.1177/1754073919898526

Cited by 46 publications

(25 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To effectively establish an emotional identification and expression system, emotional identification and synthesis based on DL have considerable potential in human–machine interactions (Schuller and Schuller, 2021 ). Recognizing emotions through the automatic extraction of acoustic features and generating expressions through emotions are the main strategies for relevant research development.…”

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

Affective Voice Interaction and Artificial Intelligence: A Research Study on the Acoustic Features of Gender and the Emotional States of the PAD Model

2021

View full text Add to dashboard Cite

New types of artificial intelligence products are gradually transferring to voice interaction modes with the demand for intelligent products expanding from communication to recognizing users' emotions and instantaneous feedback. At present, affective acoustic models are constructed through deep learning and abstracted into a mathematical model, making computers learn from data and equipping them with prediction abilities. Although this method can result in accurate predictions, it has a limitation in that it lacks explanatory capability; there is an urgent need for an empirical study of the connection between acoustic features and psychology as the theoretical basis for the adjustment of model parameters. Accordingly, this study focuses on exploring the differences between seven major “acoustic features” and their physical characteristics during voice interaction with the recognition and expression of “gender” and “emotional states of the pleasure-arousal-dominance (PAD) model.” In this study, 31 females and 31 males aged between 21 and 60 were invited using the stratified random sampling method for the audio recording of different emotions. Subsequently, parameter values of acoustic features were extracted using Praat voice software. Finally, parameter values were analyzed using a Two-way ANOVA, mixed-design analysis in SPSS software. Results show that gender and emotional states of the PAD model vary among seven major acoustic features. Moreover, their difference values and rankings also vary. The research conclusions lay a theoretical foundation for AI emotional voice interaction and solve deep learning's current dilemma in emotional recognition and parameter optimization of the emotional synthesis model due to the lack of explanatory power.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

“…It has been proven that a generative adversarial network (GAN) can improve the machine's performance in emotional analysis tasks (Han et al, 2019 ). Additionally, people begin to think about transfer learning applications in relevant tasks and voice emotional computing modes (Schuller and Schuller, 2021 ).…”

Section: Literature Reviewmentioning

confidence: 99%

Affective Voice Interaction and Artificial Intelligence: A Research Study on the Acoustic Features of Gender and the Emotional States of the PAD Model

2021

View full text Add to dashboard Cite

show abstract

“…Expressions can be produced spontaneously in ecologically valid scenarios (Tracy & Matsumoto, 2008), with poses in controlled laboratory settings (Cordaro et al, 2018), or elicited with stimuli that are meant to induce specific emotions (Levenson et al, 1991). The expressive cues, such as facial muscle movements or vocalizations, are then empirically mapped using established coding schemes (Dael et al, 2012a(Dael et al, , 2012bEkman et al, 2002) and automatic detection software using algorithms (Krumhuber et al, 2019;Schuller & Schuller, 2020). A broad range of positive emotions have been found to have specific nonverbal cues (for a review, see Sauter, 2017).…”

Section: Nonverbal Expressions Of Positive Emotionsmentioning

confidence: 99%

Do People Agree on How Positive Emotions Are Expressed? A Survey of Four Emotions and Five Modalities Across 11 Cultures

et al. 2021

View full text Add to dashboard Cite

While much is known about how negative emotions are expressed in different modalities, our understanding of the nonverbal expressions of positive emotions remains limited. In the present research, we draw upon disparate lines of theoretical and empirical work on positive emotions, and systematically examine which channels are thought to be used for expressing four positive emotions: feeling moved, gratitude, interest, and triumph. Employing the intersubjective approach, an established method in cross-cultural psychology, we first explored how the four positive emotions were reported to be expressed in two North American community samples (Studies 1a and 1b: n = 1466). We next confirmed the cross-cultural generalizability of our findings by surveying respondents from ten countries that diverged on cultural values (Study 2: n = 1826). Feeling moved was thought to be signaled with facial expressions, gratitude with the use of words, interest with words, face and voice, and triumph with body posture, vocal cues, facial expressions, and words. These findings provide cross-culturally consistent findings of differential expressions across positive emotions. Notably, positive emotions were thought to be expressed via modalities that go beyond the face.

show abstract

“…Expressions can be produced spontaneously in ecologically valid scenarios (Tracy & Matsumoto, 2008), with poses in controlled laboratory settings (Cordaro et al, 2018), or elicited with stimuli that are meant to induce specific emotions (Levenson et al, 1991). The expressive cues, such as facial muscle movements or vocalizations, are then empirically mapped using established coding schemes (Ekman et al, 2002; and automatic detection software using algorithms (Schuller & Schuller, 2020;Krumhuber et al, 2019). A broad range of positive emotions have been found to have specific nonverbal cues (for a review, see Sauter, 2017).…”

Section: Nonverbal Expressions Of Positive Emotionsmentioning

confidence: 99%

Do People Agree on How Positive Emotions are Expressed? A Survey of Four Emotions and Five Modalities across 11 Cultures

Manokara¹,

Đurić²,

Fischer³

et al. 2020

Preprint

View full text Add to dashboard Cite

While much is known about how negative emotions are expressed in different modalities, our understanding of the nonverbal expressions of positive emotions remains limited. In the present research, we draw upon disparate lines of theoretical and empirical work on positive emotions, and systematically examine which channels are thought to be used for expressing four positive emotions: feeling moved, gratitude, interest, and triumph. Employing the intersubjective approach, an established tool in cross-cultural psychology, we first examined how the four positive emotions were reported to be expressed in a U.S.A. community sample (Study 1: n = 1015). We next confirmed the cross-cultural generalizability of our findings by surveying respondents from ten countries that diverged on cultural values (Study 2: n = 1834). Feeling moved was thought to be signaled with facial expressions, gratitude with the use of words, interest with words, face and voice, and triumph with body posture, vocal cues, facial expressions, and words. These findings provide cross-culturally consistent findings of differential expressions across positive emotions. Notably, positive emotions were mostly thought to be expressed via modalities that go beyond the face. In addition, we hope that the intersubjective approach will constitute a useful tool for researchers studying nonverbal expressions.

show abstract

A Review on Five Recent and Near-Future Developments in Computational Processing of Emotion in the Human Voice

Cited by 46 publications

References 44 publications

Affective Voice Interaction and Artificial Intelligence: A Research Study on the Acoustic Features of Gender and the Emotional States of the PAD Model

Affective Voice Interaction and Artificial Intelligence: A Research Study on the Acoustic Features of Gender and the Emotional States of the PAD Model

Do People Agree on How Positive Emotions Are Expressed? A Survey of Four Emotions and Five Modalities Across 11 Cultures

Do People Agree on How Positive Emotions are Expressed? A Survey of Four Emotions and Five Modalities across 11 Cultures

Contact Info

Product

Resources

About