Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning

Ping, Wei; Peng, Kainan; Gibiansky, Andrew; Arık, Sercan Ö.; Kannan, Ajay; Narang, Sharan; Raiman, Jonathan; Miller, J. J.

doi:10.48550/arxiv.1710.07654

Cited by 97 publications

(85 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…WaveNet can be used as a speech vocoder by conditioning auxiliary features such as Mel-spectrogram and acoustic features extracted by conventional signal processing-based vocoder [2]. This is also used in state-of-the-art speech synthesis systems, which greatly contributes to improving the quality of synthesized speech [3,4]. However, WaveNet suffers from slow inference speed because of the AR mechanism and huge network architectures.…”

Section: Introductionmentioning

confidence: 99%

Periodnet: A Non-Autoregressive Waveform Generation Model with a Structure Separating Periodic and Aperiodic Components

Hono

Takaki

Hashimoto

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

We propose PeriodNet, a non-autoregressive (non-AR) waveform generation model with a new model structure for modeling periodic and aperiodic components in speech waveforms. The non-AR waveform generation models can generate speech waveforms parallelly and can be used as a speech vocoder by conditioning an acoustic feature. Since a speech waveform contains periodic and aperiodic components, both components should be appropriately modeled to generate a high-quality speech waveform. However, it is difficult to decompose the components from a natural speech waveform in advance. To address this issue, we propose a parallel model and a series model structure separating periodic and aperiodic components. The features of our proposed models are that explicit periodic and aperiodic signals are taken as input, and external periodic/aperiodic decomposition is not needed in training. Experiments using a singing voice corpus show that our proposed structure improves the naturalness of the generated waveform. We also show that the speech waveforms with a pitch outside of the training data range can be generated with more naturalness.

show abstract

Section: Introductionmentioning

confidence: 99%

Periodnet: A Non-Autoregressive Waveform Generation Model with a Structure Separating Periodic and Aperiodic Components

Hono

Takaki

Hashimoto

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…With the increasing popularity of voice assistant, virtual reality and other artificial intelligence technologies, text-tospeech (TTS) is becoming an important component in a wide range of applications. While recent advancements in neural TTS technologies have brought significant improvement in audio quality, efficient synthesis remains challenging in many scenarios [1][2][3][4][5][6][7][8]. In practical applications, latency, computational complexity, synthesis speed and streamability are key metrics for a production TTS system, especially if it has limited computational resources such as on mobile devices.…”

Section: Introductionmentioning

confidence: 99%

“…Correspondence to Qing He: qinghe@fb.com Tacotron2 [2], FastSpeech [3], Deep voice 3 [8] etc, usually synthesize speech in two stages: 1: generate the speech spectrum from text; and 2: generate the speech waveform by conditioning on the predicted spectrum. We focus on the problem of two-stage TTS system design and propose a spectrum model that achieves low latency, supports streaming and produces high-quality TTS at the same time.…”

Section: Introductionmentioning

confidence: 99%

Multi-rate attention architecture for fast streamable Text-to-speech spectrum modeling

Xiu

Koehler

et al. 2021

Preprint

View full text Add to dashboard Cite

Typical high quality text-to-speech (TTS) systems today use a two-stage architecture, with a spectrum model stage that generates spectral frames and a vocoder stage that generates the actual audio. High-quality spectrum models usually incorporate the encoder-decoder architecture with selfattention or bi-directional long short-term (BLSTM) units. While these models can produce high quality speech, they often incur O(L) increase in both latency and real-time factor (RTF) with respect to input length L. In other words, longer inputs leads to longer delay and slower synthesis speed, limiting its use in real-time applications. In this paper, we propose a multi-rate attention architecture that breaks the latency and RTF bottlenecks by computing a compact representation during encoding and recurrently generating the attention vector in a streaming manner during decoding. The proposed architecture achieves high audio quality (MOS of 4.31 compared to groundtruth 4.48), low latency, and low RTF at the same time. Meanwhile, both latency and RTF of the proposed system stay constant regardless of input lengths, making it ideal for real-time applications.

show abstract

“…A development in TTS technology that is relevant to our work has been the introduction of attention-based architectures [12] such as Tacotron [13,14] and Deep Voice [15], attention-based sequence-to-sequence (AS2S) models which predict spectrograms that are subsequently used to synthesise a waveform with a vocoder. For the purposes of this paper, the most salient feature of AS2S architectures is that its only conditioning input is text (or a corresponding phoneme list), and not any additional model or piece of information.…”

Section: Introductionmentioning

confidence: 99%

Singing Synthesis: With a Little Help from my Attention

et al. 2020

View full text Add to dashboard Cite

We present a novel system for singing synthesis, based on attention. Starting from a musical score with notes and lyrics, we build a phoneme-level multi stream note embedding. The embedding contains the information encoded in the score regarding pitch, duration and the phonemes to be pronounced on each note. This note representation is used to condition an attention-based sequence-to-sequence architecture, in order to generate mel-spectrograms. Our model demonstrates attention can be successfully applied to the singing synthesis field. The system requires considerably less explicit modelling of voice features such as F0 patterns, vibratos, and note and phoneme durations, than most models in the literature. However, we observe that completely dispensing with any duration modelling introduces occasional instabilities in the generated spectrograms. We train an autoregressive WaveNet to be used as a neural vocoder to synthesise the mel-spectrograms produced by the sequence-to-sequence architecture, using a combination of speech and singing data.

show abstract

Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning

Cited by 97 publications

References 13 publications

Periodnet: A Non-Autoregressive Waveform Generation Model with a Structure Separating Periodic and Aperiodic Components

Periodnet: A Non-Autoregressive Waveform Generation Model with a Structure Separating Periodic and Aperiodic Components

Multi-rate attention architecture for fast streamable Text-to-speech spectrum modeling

Singing Synthesis: With a Little Help from my Attention

Contact Info

Product

Resources

About