Modeling Prosodic Phrasing With Multi-Task Learning in Tacotron-Based TTS

Liu, Rui; Şişman, Berrak; Bao, Feilong; Gao, Guanglai; Li, Haizhou

doi:10.1109/lsp.2020.3016564

Cited by 18 publications

(4 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The test of their proposed approach on native speakers reveals that it outperforms the baseline system educated without prosodic annotation. To enhance the prosodic phrasing of the Tacotronbased TTSmodel [36] , suggested the novel two-task learning scheme. For Chinese, they used TH-CoSS (TsingHua Corpus of Speech Synthesis).…”

Section: Related Workmentioning

confidence: 99%

End-to-End Kurdish Speech Synthesis Based on Transfer Learning

muhamad

Veisi

2022

PJBAS

View full text Add to dashboard Cite

A text-to-speech (TTS) system converts the texts into speech in a specific language. Several TTS systems generate natural-like speech signals in numerous languages, such as English. On the other hand, the Kurdish language has just been examined. Existing preliminary research on Kurdish speech synthesis has utilized old methods and has generated low-quality speech. They also lack important aspects of speech, including intonation, emphasis, and rhythm. Some approaches were presented to address these challenges, including the use of concatenative systems. For example, the unit selection or statistical parametric methods. On the other hand, they need a great deal of time, effort, and domain knowledge. An additional factor for Kurdish speech synthesizers' low performance is the absence of publicly available speech corpora, unlike English, which has many freely-available corpora and audiobooks. The motivation of this paper is to create a Central Kurdish speech corpus and generate a human-like speech from the Kurdish text. This paper explains how to utilize Tacotron 2, an end-to-end neural network architecture and HiFi-GAN vocoder, to produce a high-quality, realistic, and human-like Kurdish voice. This work utilizes "text, audio" pairings, which contain 10 hours of recorded audio samples and texts collected from the Internet and textbooks. It shows how to use English character embedding as the pre-trained knowledge with Kurdish characters as input and how to preprocess these audio examples to get a great outcome. Our evaluations for various types of texts show a mean opinion score of 4.1, comparable with state-of-the-art synthesizers in other languages.

show abstract

Section: Related Workmentioning

confidence: 99%

End-to-End Kurdish Speech Synthesis Based on Transfer Learning

muhamad

Veisi

2022

PJBAS

View full text Add to dashboard Cite

show abstract

“…With the advent of deep learning, end-to-end generative TTS models simplify the synthesis pipeline with a single neural network. Tacotron-based neural TTS [6,7] and its variants [8][9][10] are such examples. In these techniques, the key idea is to integrate the conventional TTS pipeline into a unified encoder-decoder network and to learn the mapping directly from the <text, wav> pair.…”

Section: Introductionmentioning

confidence: 99%

Graphspeech: Syntax-Aware Graph Attention Network for Neural Speech Synthesis

Şişman

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Attention-based end-to-end text-to-speech synthesis (TTS) is superior to conventional statistical methods in many ways. Transformer-based TTS is one of such successful implementations. While Transformer TTS models the speech frame sequence well with a self-attention mechanism, it does not associate input text with output utterances from a syntactic point of view at sentence level. We propose a novel neural TTS model, denoted as GraphSpeech, that is formulated under graph neural network framework. GraphSpeech encodes explicitly the syntactic relation of input lexical tokens in a sentence, and incorporates such information to derive syntactically motivated character embeddings for TTS attention mechanism. Experiments show that GraphSpeech consistently outperforms the Transformer TTS baseline in terms of spectrum and prosody rendering of utterances.

show abstract

“…Recently, sequence-to-sequence learning based on encoderdecoder model has been successfully applied to a wide range of tasks, including neural machine translation [5][6][7], automatic speech recognition [8][9][10], text-to-speech [11][12][13], phrase break prediction [14,15], and grapheme-to-phoneme conversion [16][17][18]. Such models learn a direct mapping between the variable lengths of sequence pairs, and directly transcribes the graphemes to phonemes without requiring predefined alignment.…”

Section: Introductionmentioning

confidence: 99%

Joint Alignment Learning-Attention Based Model for Grapheme-to-Phoneme Conversion

Wang

Bao

Zhang

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Sequence-to-sequence attention-based models for grapheme-tophoneme (G2P) conversion have gained significant interests. The attention-based encoder-decoder framework learns the mapping of input to output tokens by selectively focusing on relevant information, and has been shown well performance. However, the attention mechanism can result in non-monotonic alignments, resulting in poor G2P conversion performance. In this paper, we present a novel approach to optimize the G2P conversion model directly alignment grapheme-phoneme sequence by using alignment learning (AL) as the loss function. Besides, we propose a multi-task learning method that uses a joint alignment learning model and attention model to predict the proper alignments and thus improve the accuracy of G2P conversion. Evaluations on Mongolian and CMUDict tasks show that alignment learning as the loss function can effectively train G2P conversion model. Further, our multi-task method can significantly outperform both the alignment learningbased model and attention-based model.

show abstract

Modeling Prosodic Phrasing With Multi-Task Learning in Tacotron-Based TTS

Cited by 18 publications

References 34 publications

End-to-End Kurdish Speech Synthesis Based on Transfer Learning

End-to-End Kurdish Speech Synthesis Based on Transfer Learning

Graphspeech: Syntax-Aware Graph Attention Network for Neural Speech Synthesis

Joint Alignment Learning-Attention Based Model for Grapheme-to-Phoneme Conversion

Contact Info

Product

Resources

About