Generic Indic Text-to-Speech Synthesisers with Rapid Adaptation in an End-to-End Framework

Prakash, Anusha; Murthy, Hema A.

doi:10.21437/interspeech.2020-2663

Cited by 12 publications

(19 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We obtained MLME values from the following studies: [6], [12], [13], [14], [16], [17], [18], [20], [22], [25], [26], and [27], and reported them in Table 3, both as a whole and in specific groups of evaluation metrics, in the form of median (M) and interquartile range (IQR). Also reported are the p-values of the corresponding one-sample Wilcoxon signed rank tests for the hypothesis that the median MLME values are larger than 0.…”

Section: Resultsmentioning

confidence: 99%

“…These resulting values (n = 880) were used for analysis. [6], [7], [8], [9], [10], [11], [12] Hidden Markov Model synthesis (HMM) 7 [12], [13], [14], [15], [16], [17], [18] Neural network (non-S2S) synthesis (DNN) 9 [19], [20], [21], [22], [23], [24], [25], [26], [27] Sequence-to-sequence synthesis (S2S)…”

Section: Characteristics Of the Included Studiesmentioning

confidence: 99%

“…From the chosen studies, notable studies include [18] (hereafter Study A), the latest included DNN-based study, which used 5 different evaluation metrics for TTS in Tibetan (with Mandarin as the source language). For S2S-based studies, among the latest are [27] (Study B), which explored TTS for Indic LRLs in the Indo-Aryan and Dravidian families, and [25] (Study C), which investigated strategies for using Dutch and other European languages to aid a limited amount of English data.…”

Section: Notable Studiesmentioning

confidence: 99%

“…As such a measure did not yet exist, we created one. This measure, hereafter referred to as the MultiLingual Model Effect (MLME), was derived from the reported results as follows: [18], [21], [27] F0 RMSE (Root-Mean-Square Error) Acoustics [12], [18] V/UV (Voiced/UnVoiced error) Acoustics [12], [18] BAP (Band APeriodicities distortion) Acoustics [18] DTWMCD (Dynamic Time Warping Mel-Cepstral Distortion) Acoustics [19] L2 NSE (L2 Norm-Squared on log-Mel spectrogram) Acoustics [24] LF0 RMSE (Log F0 Root-Mean-Square Error) Acoustics [11] LSD (normalized Log Cepstral Distance) Acoustics [12] MGC RMSE (Mel-Generalized Cepstrum coefficients Root-Mean-Square Error) Acoustics [11] MSE (Mean-Square Error) Acoustics [13] CER (Character Error Rate) Intelligibility [21], [26] Intelligibility % (percentage of intelligible sentences) Intelligibility [7] SUS-Wacc (Word accuracy in Semantically Unpredictable Sentences)…”

Section: Multilingual Model Effect (Mlme)mentioning

confidence: 99%

“…Hidden Reference & Anchor) Naturalness [25] DMOS (Degradation MOS) Similarity [18], [22], [27] where v multi and vmono are the reported values of output quality from the corresponding multilingual and monolingual models, respectively, and (*) is the scenario in which the metric m positively correlates with general output quality (the higher, the better, e.g., MOS, MUSHRA, etc. ), as opposed to the opposite correlation (the lower, the better, e.g., MCD, WER, etc.…”

Section: Mushra (Multiple Stimuli Withmentioning

confidence: 99%

See 4 more Smart Citations

A Systematic Review and Analysis of Multilingual Data Strategies in Text-to-Speech for Low-Resource Languages

Do¹,

Coler²,

Dijkstra³

et al. 2021

Interspeech 2021

View full text Add to dashboard Cite

We provide a systematic review of past studies that use multilingual data for text-to-speech (TTS) of low-resource languages (LRLs). We focus on the strategies used by these studies for incorporating multilingual data and how they affect output speech quality. To investigate the difference in output quality between corresponding monolingual and multilingual models, we propose a novel measure to compare this difference across the included studies and their various evaluation metrics. This measure, called the Multilingual Model Effect (MLME), is found to be affected by: acoustic model architecture, the difference ratio of target language data between corresponding multilingual and monolingual experiments, the balance ratio of target language data to total data, and the amount of target language data used. These findings can act as reference for data strategies in future experiments with multilingual TTS models for LRLs. Language family classification, despite being widely used, is not found to be an effective criterion for selecting source languages.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Characteristics Of the Included Studiesmentioning

confidence: 99%

Section: Notable Studiesmentioning

confidence: 99%

Section: Multilingual Model Effect (Mlme)mentioning

confidence: 99%

Section: Mushra (Multiple Stimuli Withmentioning

confidence: 99%

See 3 more Smart Citations

A Systematic Review and Analysis of Multilingual Data Strategies in Text-to-Speech for Low-Resource Languages

Do¹,

Coler²,

Dijkstra³

et al. 2021

Interspeech 2021

View full text Add to dashboard Cite

show abstract

Code-Mixed Text-to-Speech Synthesis Under Low-Resource Constraints

Joshi,

Garera

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Text-to-speech (TTS) systems are being built using end-to-end deep learning approaches. However, these systems require huge amounts of training data. We present our approach to built production quality TTS and perform speaker adaptation in extremely low resource settings. We propose a transfer learning approach using high-resource language data and synthetically generated data. We transfer the learnings from the out-domain high-resource English language. Further, we make use of out-of-the-box single-speaker TTS in the target language to generate in-domain synthetic data. We employ a three-step approach to train a high-quality single-speaker TTS system in a low-resource Indian language Hindi. We use a Tacotron2 like setup with a spectrogram prediction network and a waveglow vocoder. The Tacotron2 acoustic model is trained on English data, followed by synthetic Hindi data from the existing TTS system. Finally, the decoder of this model is fine-tuned on only 3 hours of target Hindi speaker data to enable rapid speaker adaptation. We show the importance of this dual pre-training and decoder-only fine-tuning using subjective MOS evaluation. Using transfer learning from high-resource language and synthetic corpus we present a low-cost solution to train a custom TTS model.

show abstract