Code-switching in Indic Speech Synthesisers

Thomas, Anju Leela; Prakash, Anusha; Baby, Arun; Murthy, Hema A.

doi:10.21437/interspeech.2018-1178

Cited by 13 publications

(11 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A code-mixable TTS is developed in the current work using a mixture of two monolingual datasets. This is similar to the approach in [4]. Code-mixing is the alternation between languages in a single sentence 3 .…”

Section: Introductionmentioning

confidence: 89%

“…Hence, the MLCM approach is not feasible for English text. Similar to the technique in [4], English words are parsed into CLS notations using a classification and regression tree (CART). The English text is then modified using the phone mapping approach.…”

Section: Phone Representation: Common Label Set (Cls)mentioning

confidence: 99%

“…ESPNet's implementation [18] of Tacotron2 is used in this work 4 . The basic architecture of Tacotron2 is an attention-based encoder-decoder network with a WaveNet vocoder [2].…”

Section: Training the End-to-end Tts Synthesisermentioning

confidence: 99%

“…TTS systems have been built in a cross-lingual manner in our earlier work [4]. In [4], code-switchable TTSes are built using a mixture of data from two languages. Initial experiments were conducted in the current work by similarly pooling data across two languages.…”

Section: Introductionmentioning

confidence: 99%

“…A Hindi+English TTS is trained using monolingual Hindi and English data. As English has a different set of characters from Hindi, English is also represented in terms of CLS labels using the technique in [4]. Additionally, a classification and regression tree (CART) is built so that out-of-vocabulary (OOV) words are also handled.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Building Multilingual End-to-End Speech Synthesisers for Indian Languages

Prakash¹,

Thomas²,

Umesh³

et al. 2019

10th ISCA Workshop on Speech Synthesis (SSW 10)

Self Cite

View full text Add to dashboard Cite

Building text-to-speech (TTS) synthesisers is a difficult task, especially for low resource languages. Language-specific modules need to be developed for system building. End-to-end speech synthesis has become a popular paradigm as a TTS can be trained using only pairs. However, end-to-end speech synthesis is not scalable in a multilanguage scenario, as the vocabulary increases with the number of different scripts. In this paper, TTSes are trained for Indian languages using two text representations-character-based and phone-based. For the character-based approach, a multi-language character map (MLCM) is proposed to easily train Indic speech synthesisers. The phone-based approach uses the common label set (CLS) representation for Indian languages. Both approaches leverage the similarities that exist among the languages. The advantage is a compact representation across multiple languages. Experiments are conducted by building TTSes using monolingual data and by pooling data across two languages. The ability to synthesise code-mixed text using the phone-based approach is also assessed. Subjective evaluations indicate that reasonably good quality Indic TTSes can be developed using both approaches. This emphasises the need to incorporate multilingual text processing in the end-to-end framework.

show abstract

Section: Introductionmentioning

confidence: 89%

Section: Phone Representation: Common Label Set (Cls)mentioning

confidence: 99%

“…ESPNet's implementation [18] of Tacotron2 is used in this work 4 . The basic architecture of Tacotron2 is an attention-based encoder-decoder network with a WaveNet vocoder [2].…”

Section: Training the End-to-end Tts Synthesisermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Building Multilingual End-to-End Speech Synthesisers for Indian Languages

Prakash¹,

Thomas²,

Umesh³

et al. 2019

10th ISCA Workshop on Speech Synthesis (SSW 10)

Self Cite

View full text Add to dashboard Cite

show abstract

Code-Mixed Text-to-Speech Synthesis Under Low-Resource Constraints

Joshi,

Garera

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Text-to-speech (TTS) systems are being built using end-to-end deep learning approaches. However, these systems require huge amounts of training data. We present our approach to built production quality TTS and perform speaker adaptation in extremely low resource settings. We propose a transfer learning approach using high-resource language data and synthetically generated data. We transfer the learnings from the out-domain high-resource English language. Further, we make use of out-of-the-box single-speaker TTS in the target language to generate in-domain synthetic data. We employ a three-step approach to train a high-quality single-speaker TTS system in a low-resource Indian language Hindi. We use a Tacotron2 like setup with a spectrogram prediction network and a waveglow vocoder. The Tacotron2 acoustic model is trained on English data, followed by synthetic Hindi data from the existing TTS system. Finally, the decoder of this model is fine-tuned on only 3 hours of target Hindi speaker data to enable rapid speaker adaptation. We show the importance of this dual pre-training and decoder-only fine-tuning using subjective MOS evaluation. Using transfer learning from high-resource language and synthetic corpus we present a low-cost solution to train a custom TTS model.

show abstract