Espnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit

Abstract

This paper introduces a new end-to-end text-to-speech (E2E-TTS) toolkit named ESPnet-TTS, which is an extension of the open-source speech processing toolkit ESPnet. The toolkit supports state-of-theart E2E-TTS models, including Tacotron 2, Transformer TTS, and FastSpeech, and also provides recipes inspired by the Kaldi automatic speech recognition (ASR) toolkit. The recipes are based on the design unified with the ESPnet ASR recipe, providing high reproducibility. The toolkit also provides pre-trained models and samples of all of the recipes so that users can use it as a baseline. Furthermore, the unified design enables the integration of ASR functions with TTS, e.g., ASR-based objective evaluation and semi-supervised learning with both ASR and TTS models. This paper describes the design of the toolkit and experimental evaluation in comparison with other toolkits. The experimental results show that our best model outperforms other toolkits, resulting in a mean opinion score (MOS) of 4.25 on the LJSpeech dataset. The toolkit is available on GitHub 1 .

Pretraining Techniques for Sequence-to-Sequence Voice Conversion

“…From Tables I and III, it was shown that without TTS-oriented pretraining, VTNs were less robust to training data reduction than RNNs in terms of objective measures but better in terms of subjective measures. This is possibly because that a more complex model like VTN is capable of generating better-sounding voices while being more prone to overfitting since it lacks attention regularizations such as the location-sensitive location, as suggested in [68]. As we applied TTS-oriented pretraining to both VTN and RNN, it could be clearly observed that VTNs outperformed RNNs in terms of all objective measures except F0RMSE and subjective scores.…”

“…In addition to the L1, L2, and weighted binary cross-entropy losses, the guided attention loss is also applied. As pointed out in [66], in Transformer-based speech synthesis, not all attention heads demonstrate diagonal alignments, so following [30], [68], the guided attention loss is applied to partial heads in partial decoder layers.…”

“…2) Implementation: The entire experiment was carried out on the open-source ESPnet toolkit [68], [72], including feature extraction, training and benchmarking. The official implementation has been made publicly available 1 , and since readers may access all the settings and configurations online, we omit the detailed hyperparameters here.…”

Enhancing Local Dependencies for Transformer-Based Text-to-Speech via Hybrid Lightweight Convolution

“…We build the experimental models by modifying the endto-end text-to-speech toolkit called ESPnet-TTS [37], which gives the unified and reproducible implementations of popular end-to-end TTS networks 3 . The models to be compared are briefly described as follows while more detailed hyperparameters can be found in the related references:…”

Enhancing Sequence-to-Sequence Text-to-Speech with Morphology

“…As this approach gains traction in the TTS field (c.f. the ESPnet-TTS project [9]), the continuing role of any separate front-end processing is naturally brought in question. Given its cost in time and money to develop, what value does it bring, and how can it be simplified and optimised for S2S TTS?…”