Sequence-to-Sequence Emotional Voice Conversion With Strength Control

Abstract

This paper proposes an improved emotional voice conversion (EVC) method with emotional strength and duration controllability. EVC methods without duration mapping generate emotional speech with identical duration to that of the neutral input speech. In reality, even the same sentences would have different speeds and rhythms depending on the emotions. To solve this, the proposed method adopts a sequence-to-sequence network with an attention module that enables the network to learn attention in the neutral input sequence should be focused on which part of the emotional output sequence. Besides, to capture the multi-attribute aspects of emotional variations, an emotion encoder is designed for transforming acoustic features into emotion embedding vectors. By aggregating the emotion embedding vectors for each emotion, a representative vector for the target emotion is obtained and weighted to reflect emotion strength. By introducing a speaker encoder, the proposed method can preserve speaker identity even after the emotion conversion. Objective and subjective evaluation results confirm that the proposed method is superior to other previous works. Especially, in emotion strength control, we achieve in getting successful results.

Textless Speech Emotion Conversion using Discrete and Decomposed Representations

“…Studies have also revealed that the emotions can be expressed through universal principles that are shared across different individuals and cultures (Ekman, 1992;Manokara et al, 2021). This motivates the study of multispeaker (Shankar et al, 2019b(Shankar et al, , 2020, and speaker-independent emotional voice conversion (Zhou et al, 2020b;Choi and Hahn, 2021).…”

Emotion Intensity and its Control for Emotional Voice Conversion

“…There are only few studies on sequence-to-sequence emotional voice conversion [20], [42], [43], [59]. In [42], the authors jointly model pitch and duration with parallel data, where the model is conditioned on the syllable position in the phrase.…”

“…One uses auxiliary features such as a state of voiced, unvoiced, and silence (VUS) [17], attention weights or a saliency map [18]. Another manipulates the internal emotion representations through interpolation [19] or scaling [20]. Despite these methods, emotion intensity control is still an under-explored topic in emotional voice conversion.…”

Emotional Voice Conversion: Theory, Databases and ESD

“…Such framework generally works well in speaker-dependent tasks. Studies have also revealed that the emotions can be expressed through some universal principles that are shared across different individuals and cultures [55,56,57], that motivates the study of multispeaker [58,59,54], and speaker-independent emotional voice conversion [60,61].…”