The influence of language and culture on the understanding of vocal emotions

Altrov, Rene; Pajupuu, Hille

doi:10.12697/jeful.2015.6.3.01

Cited by 9 publications

(9 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For a speech-only system, automatic emotion recognition should understand the fundamental dynamics of emotional cues and be able to identify emotional states from utterances. As recognizing emotions in speech among humans is difficult when cultural differences exist among speakers, the same difficulty remains for SER [ 5 ]. Coupled with speech variations among speakers and dynamical features with low saliency, SER is a challenging problem.…”

Section: Introductionmentioning

confidence: 99%

Fusion-ConvBERT: Parallel Convolution and BERT Fusion for Speech Emotion Recognition

Lee

Han

2020

Sensors

View full text Add to dashboard Cite

Speech emotion recognition predicts the emotional state of a speaker based on the person’s speech. It brings an additional element for creating more natural human–computer interactions. Earlier studies on emotional recognition have been primarily based on handcrafted features and manual labels. With the advent of deep learning, there have been some efforts in applying the deep-network-based approach to the problem of emotion recognition. As deep learning automatically extracts salient features correlated to speaker emotion, it brings certain advantages over the handcrafted-feature-based methods. There are, however, some challenges in applying them to the emotion recognition problem, because data required for properly training deep networks are often lacking. Therefore, there is a need for a new deep-learning-based approach which can exploit available information from given speech signals to the maximum extent possible. Our proposed method, called “Fusion-ConvBERT”, is a parallel fusion model consisting of bidirectional encoder representations from transformers and convolutional neural networks. Extensive experiments were conducted on the proposed model using the EMO-DB and Interactive Emotional Dyadic Motion Capture Database emotion corpus, and it was shown that the proposed method outperformed state-of-the-art techniques in most of the test configurations.

show abstract

Section: Introductionmentioning

confidence: 99%

Fusion-ConvBERT: Parallel Convolution and BERT Fusion for Speech Emotion Recognition

Lee

Han

2020

Sensors

View full text Add to dashboard Cite

show abstract

“…It was also found that the difference between the prosodic features in Estonian and Australian English was a significant obstacle for the Australians attempting to read their partners' affective state. This finding supports the results ofAltrov and Pajupuu (2015) that the acoustic properties characteristic of Estonian speech can provide an extra challenge in the interpretation of the vocal expression of emotion by other cultural groups.With regard to the Australian partners' emotional behaviour, none of the Estonian partners reported misreading their partners' expression of emotion. However, in some cases, the Estonian partners found the explicit expression of emotion confronting.…”

supporting

confidence: 90%

“…This indicates that some vocal expressions can be characteristic of a particular language, which, in turn, will provide extra challenge for the representatives of the different linguistic and cultural backgrounds to interpret the emotion expressed. For example, the results from the recent study by Altrov and Pajupuu (2015) demonstrate that except for Estonians, all other cultural groups under investigation (Latvians, Italians, Finns, Swedes, Danes, Norwegians, Russians) generally misinterpreted the Estonian vocal expression of basic emotions such as anger and joy, confusing them with neutral speech. Moreover, some of the Latvians, Swedes, Norwegians, and Russians in the study perceived the Estonian vocal expression of anger as the expression of joy.…”

Section: The Role Of Various Channels In Communication Of Emotionmentioning

confidence: 99%

Perceived communication of emotion in intercultural Estonian-Australian couples in Australia

Kull¹

View full text Add to dashboard Cite

This thesis investigates the communication of emotion in intercultural Estonian-Australian families in Australia from an applied linguistics perspective, situating the study within the fields of intercultural communication and acculturation with a specific focus on emotion. It adds to a growing body of literature on the communication of emotion in intercultural and bilingual contexts (most prominently by Dewaele, Grosjean, Panayiotou, Pavlenko, Piller and Wierzbicka) by drawing attention to a less frequently studied combination of cultures and focusing on the mutual perceptions and interpretations of the informants. Even though globalisation has led to an increase in the number of intercultural families, so far little research has focused on mixed family communication, and even less on the expression and perception of emotion between partners. To contribute towards filling this gap, the present study focuses on cultural differences in emotional behaviour and in its interpretation, specifically on Estonian and Australian partners' perception of their own and each other's emotional behaviour. Since emotional behavioural patterns not only vary among cultures and individuals but may also be affected by sustained contact with cultural others, this study also explores the multidirectional influences and patterns of negotiation in the communication of emotion in intercultural Estonian-Australian families. An embedded mixed methods approach was used to gain a detailed picture of the perception of emotional behaviour in Estonian-Australian families. Semi-structured interviews were adopted as the main data-gathering method in combination with an online questionnaire, where the latter provided background information about the participants and context. The interviews were conducted with both partners simultaneously, which enabled the partners to add to or comment on each other's responses, and therefore compensated for a potential bias in the self-reports. The analysis explored the relationship between the partners' linguistic and cultural backgrounds and the interpretation of the differing forms of emotional behaviour, together with the perceived changes in the communication of emotion which may occur in intercultural contact. The analysis revealed cultural differences in the expressiveness of the Estonian and Australian partners. Specifically, the Estonian partners were found to be more reserved in their expression of emotion in comparison with their Australian partners. This difference, in turn, emerged as a source of misunderstandings in Estonian-Australian families. The Australians in particular experienced difficulty in interpreting the affective states of their Estonian partners, expecting their partners' expression of emotion to resemble that of their home culture. One specific area which emerged from the data as problematic was the interpretation of vocal cues accompanying the Estonian partners' speech. This was related to the different acoustic properties characteristic of Estonian and Australian English, which cre...

show abstract

“…Generally speaking, it is expensive to invite actors to perform emotions in the studio and it is easier to collect noisy data in real life. On the other hand, emotions are subjective and influenced by many factors, language and culture have an important influence on the judgment of emotions in speech [19], which increases the cost of data labeling. In addition, improvised speech is easier to be recognized than those performed in the studio according to the script, which has been demonstrated in previous research [20], [21].…”

Section: Introductionmentioning

confidence: 99%

Head Fusion: Improving the Accuracy and Robustness of Speech Emotion Recognition on the IEMOCAP and RAVDESS Dataset

Zhang²,

Zhang³

2021

IEEE Access

View full text Add to dashboard Cite

Speech Emotion Recognition (SER) refers to the use of machines to recognize the emotions of a speaker from his (or her) speech. SER benefits Human-Computer Interaction(HCI). But there are still many problems in SER research, e.g., the lack of high-quality data, insufficient model accuracy, little research under noisy environments, etc. In this paper, we proposed a method called Head Fusion based on the multi-head attention mechanism to improve the accuracy of SER. We implemented an attentionbased convolutional neural network(ACNN) model and conducted experiments on the Interactive Emotional Dyadic Motion Capture (IEMOCAP) data set. The accuracy is improved to 76.18% (weighted accuracy, WA) and 76.36% (unweighted accuracy, UA). To the best of our knowledge, compared with the state-ofthe-art result on this dataset (76.4% of WA and 70.1% of WA), we achieved a UA improvement of about 6% absolute while achieving a similar WA. Furthermore, We conducted empirical experiments by injecting speech data with 50 types of common noises. We inject the noises by altering the noise intensity, timeshifting the noises, and mixing different noise types, to identify their varied impacts on the SER accuracy and verify the robustness of our model. This work will also help researchers and engineers properly add their training data by using speech data with the appropriate types of noises to alleviate the problem of insufficient high-quality data.

show abstract

The influence of language and culture on the understanding of vocal emotions

Cited by 9 publications

References 15 publications

Fusion-ConvBERT: Parallel Convolution and BERT Fusion for Speech Emotion Recognition

Fusion-ConvBERT: Parallel Convolution and BERT Fusion for Speech Emotion Recognition

Perceived communication of emotion in intercultural Estonian-Australian couples in Australia

Head Fusion: Improving the Accuracy and Robustness of Speech Emotion Recognition on the IEMOCAP and RAVDESS Dataset

Contact Info

Product

Resources

About