Building Multilingual End-to-End Speech Synthesisers for Indian Languages

Abstract

Building text-to-speech (TTS) synthesisers is a difficult task, especially for low resource languages. Language-specific modules need to be developed for system building. End-to-end speech synthesis has become a popular paradigm as a TTS can be trained using only <text, audio> pairs. However, end-to-end speech synthesis is not scalable in a multilanguage scenario, as the vocabulary increases with the number of different scripts. In this paper, TTSes are trained for Indian languages using two text representations-character-based and phone-based. For the character-based approach, a multi-language character map (MLCM) is proposed to easily train Indic speech synthesisers. The phone-based approach uses the common label set (CLS) representation for Indian languages. Both approaches leverage the similarities that exist among the languages. The advantage is a compact representation across multiple languages. Experiments are conducted by building TTSes using monolingual data and by pooling data across two languages. The ability to synthesise code-mixed text using the phone-based approach is also assessed. Subjective evaluations indicate that reasonably good quality Indic TTSes can be developed using both approaches. This emphasises the need to incorporate multilingual text processing in the end-to-end framework.

A Systematic Review and Analysis of Multilingual Data Strategies in Text-to-Speech for Low-Resource Languages

“…We obtained MLME values from the following studies: [6], [12], [13], [14], [16], [17], [18], [20], [22], [25], [26], and [27], and reported them in Table 3, both as a whole and in specific groups of evaluation metrics, in the form of median (M) and interquartile range (IQR). Also reported are the p-values of the corresponding one-sample Wilcoxon signed rank tests for the hypothesis that the median MLME values are larger than 0.…”

“…These resulting values (n = 880) were used for analysis. [6], [7], [8], [9], [10], [11], [12] Hidden Markov Model synthesis (HMM) 7 [12], [13], [14], [15], [16], [17], [18] Neural network (non-S2S) synthesis (DNN) 9 [19], [20], [21], [22], [23], [24], [25], [26], [27] Sequence-to-sequence synthesis (S2S)…”

“…Hidden Reference & Anchor) Naturalness [25] DMOS (Degradation MOS) Similarity [18], [22], [27] where v multi and vmono are the reported values of output quality from the corresponding multilingual and monolingual models, respectively, and (*) is the scenario in which the metric m positively correlates with general output quality (the higher, the better, e.g., MOS, MUSHRA, etc. ), as opposed to the opposite correlation (the lower, the better, e.g., MCD, WER, etc.…”

“…Since all studies (except for [19] and [26]) mentioned this only in either time duration or number of utterances, an average estimation was needed to convert it to a common measurement. Following the descriptions provided in [10], [11], [12], [16], [17], [20], [21], [22], [23], [24], and [27], we obtained an average utterance length of the speech data sets used in these studies: 6.1 seconds. This was then used to convert all the training data quantities to the corresponding estimated number of utterances.…”

Exploring the use of Common Label Set to Improve Speech Recognition of Low Resource Indian Languages

“…The parser leveraged the phonetic similarity among the Indian languages to generate the lexicon. Further [4] explored ways of merging tokens, based either on characters or on phones. Based on subjective evaluations, both approaches gave reasonable quality speech synthesis.…”