An experimental study of emotional speech in Mandarin and English

Abstract

This study reports our initial results on whether the use of pitch for expressing emotions differs between Mandarin and English. The production experiment was conducted using five emotions (anger, fear, happiness, sadness, and neutral) by comparing both prosodic cues and phonation cues between Mandarin and English emotional speech. Results demonstrated that within each language, each vocal emotion had specific acoustic patterns. Moreover, Mandarin and English showed different mechanisms of utilizing pitch for encoding emotions. The differences in pitch variation between neutral and other emotions were significantly larger in English than in Mandarin. However, the variations of speech rate and certain phonation cues (e.g., CPP and CQ) were significantly larger in Mandarin than in English. The differences in emotional speech between the two languages may be due to the restriction of pitch variation by the presence of lexical tones in Mandarin. This study reveals an interesting finding that when a certain parameter (e.g., pitch) is restricted in one language, other cues turned out to be strengthened for compensation. Therefore, we posit that the acoustic realizations of emotional speech are multidimensional.

Emotional tones of voice affect the acoustics and perception of Mandarin tones

“…There is ample evidence that different emotions result in distinct acoustic characteristics [2,[5][6][7][41][42][43][44][45][46][47][48][49][50][51][52][53][54]. Physiologically, the sympathetic nervous system is aroused by emotions such as anger, fear, or happiness, resulting in a higher heart rate and blood pressure, a dry mouth, and occasionally muscle tremors [55,56].…”

“…Wang, Lee, and Ma (2016, 2018) [ 46 , 66 ] examined acoustic correlates of angry, fear, happy, sad, and neutral emotions in Mandarin and English. Semantically-neutral declarative sentences were embedded in different contexts to elicit angry, fear, happy, sad, and neutral emotions.…”

“…There is limited evidence on how lexical tones affect the perception of emotions. Wang, Ding, & Gu (2012) [ 80 ] investigated emotion recognition from Mandarin sentences by native and non-native speakers. Mandarin words with various tones ( qi4che1 “car”, zhao4pian4 “picture”, xin1fang2 “new house”, dian4nao3 “computer”, and xue2xiao3 “school”) were embedded in a semantically neutral carrier phrase ( zhe4 shi4 ta1 de0 [target word] “This is his [target word]”).…”

Within and Across-Language Comparison of Vocal Emotions in Mandarin and English

“…(b) whether Mandarin shows a restricted pitch variation in encoding vocal emotions compared to English; and (c) if so, how other cues change to supplement the restricted use of pitch. The preliminary results of this study were presented at the 8th International Conference on Speech Prosody as shown in Reference [31].…”

“…Prosodic cues of four emotions in Mandarin and English. Points indicate mean values and error bars 95% confidence intervals (Adapted fromFigure 1in Reference[31]).…”