Introducing Prosodic Speaker Identity for a Better Expressive Speech Synthesis Control

Abstract

To have more control over Text-to-Speech (TTS) synthesis and to improve expressivity, it is necessary to disentangle prosodic information carried by the speaker's voice identity from the one belonging to linguistic properties. In this paper, we propose to analyze how information related to speaker voice identity affects a Deep Neural Network (DNN) based multi-speaker speech synthesis model. To do so, we feed the network with a vector encoding speaker information in addition to a set of basic linguistic features. We then compare three main speaker coding configurations: a) simple one-hot vector describing the speaker gender and identifier ; b) an embedding vector extracted from a speaker recognition pre-trained model ; c) a prosodic vector which summarizes information such as melody, intensity, and duration. To measure the impact of the input feature vector, we investigate the representation of the latent space at the output of the first layer of the network. The aim is to have an overview of our data representation and model behavior. Furthermore, we conducted a subjective assessment to validate the result. Results show that the prosodic identity of the speaker is captured by the model and therefore allows the user to control more precisely synthesis.

Evaluating Acoustic Parameters for DeepFake Audio Identification

“…To the data generated from the various models listed representing the fake audio datasets, we added audio snippets from various recordings and datasets like synplaflex which is a corpus of audiobooks in French composed of 87 hours of good quality speech [14] and other recorded audio messages mainly in French language thus representing the authentic audio snippets dataset. Then, from this dataset, we segmented the audios into 10 seconds snippets and 02 seconds snippets.…”

LIUM-TTS entry for Blizzard 2023

“…NEB has read numerous books whose recordings are available on Librivox. In the Syn-PaFlex project, more than 87 hours of this voice were extracted and annotated according to various expressive aspects in order to build a corpus dedicated to French expressive TTS [9]. Indeed, the speaker is able to change her prosody and modify her voice in order to personify some characters with a distinct style from the indirect speech [10].…”

Controllable Data Generation by Deep Learning: A Review

“…LibriSpeech was also borrowed in TTS-related task to control emotion of generated speech [212,305]. LibriVox is a collection of public audiobooks that can be used in controllable deep audio synthesis [213,306]. Emotional Speech Database consists of 350 parallel utterances spoken by 10 native English and 10 native Chinese speakers and covers 5 emotion categories (neutral, happy, angry, sad and surprise) [307].…”