Expressive Singing Synthesis Based on Unit Selection for the Singing Synthesis Challenge 2016

Abstract

Sample and statistically based singing synthesizers typically require a large amount of data for automatically generating expressive synthetic performances. In this paper we present a singing synthesizer that using two rather small databases is able to generate expressive synthesis from an input consisting of notes and lyrics. The system is based on unit selection and uses the Wide-Band Harmonic Sinusoidal Model for transforming samples. The first database focuses on expression and consists of less than 2 minutes of free expressive singing using solely vowels. The second one is the timbre database which for the English case consists of roughly 35 minutes of monotonic singing of a set of sentences, one syllable per beat. The synthesis is divided in two steps. First, an expressive vowel singing performance of the target song is generated using the expression database. Next, this performance is used as input control of the synthesis using the timbre database and the target lyrics. A selection of synthetic performances have been submitted to the Interspeech Singing Synthesis Challenge 2016, in which they are compared to other competing systems.

A Neural Parametric Singing Synthesizer Modeling Timbre and Expression from Natural Songs

“…Additionally, phonetics can also cause inflections in F0, so-called microprosody [16]. Some approaches try to decompose these factors to various degrees, for instance by separating vibratos [4] or using source material without consonants [1,17]. In our approach, however, we model the F0 contour as-is, without any decomposition.…”

“…The more prominent of which are based on concatenative methods [1,2] and statistical parametric methods centered around Hidden Markov Models (HMMs) [3,4]. Similar to in this work, an important benefit of statistical models is that they allow joint modeling of timbre and musical expression from natural singing [18,35].…”

“…• IS16: A concatenative unit selection-based system [1], which was the highest rated system in the Interspeech 2016 Singing Synthesis Challenge.…”

“…Many of today's more successful singing synthesizers are based on concatenative methods [1,2]. That is, they transform and concatenate short waveform units selected from an inventory of recordings of a singer.…”

Singing Voice Synthesis Using Deep Autoregressive Neural Networks for Acoustic Modeling

“…into songs, which differs from traditional text-to-speech (TTS) synthesis. Some song synthesizers have been developed based on the unit selection speech synthesis approach [1,2]. Although this approach can achieve high sound quality, it relies on large corpora and its flexibility is limited.…”