The Thinkit System for Icassp2021 M2voc Challenge

Abstract

In this paper, we introduce the low resource text-to-speech system from the ThinkIT team submitted to Multi-Speaker Multi-Style Voice Cloning Challenge (M2VoC). The challenge has two tasks: few-shot track1 provides 100 samples for each person and one-shot track2 offers 5 samples only. Each track contains two sub-tracks A and B. Instead of subtrack A, sub-track B can use extra public data besides the released data. But we participate in the sub-track A only. We choose the finetune as our backbone strategy. Our submitted systems include BERT based prosody boundary prediction module, FastSpeech based acoustic model to generate acoustic features from text input, and HIFIGAN based vocoder to generate waveform from acoustic features. Among them, acoustic models are susceptible to low resource speakers. To prevent over-fitting, we modified the acoustic model and split out validation set to assist the manual model selection. Evaluation results provided by the challenges organizers demonstrate the effectiveness of our system.

The Multi-speaker Multi-style Voice Cloning Challenge 2021

“…T18 proposed to use a BERT [41] module to predict the break of each Chinese character in an input sentence. T15 [42] used a fine-grained encoder added at the decoder's tailor, which extracts variable-length detailed style information from multiple reference samples via an attention mechanism. T03 and T15 also used global style tokens (GST) for both speaker and style control, which consists of a reference encoder, style attention, and style embedding.…”