Transfer Learning for the Russian Language Speech Synthesis

Kuzmin, Aleksander D.; Ivanov, Sergey

doi:10.1109/itqmis53292.2021.9642715

Cited by 2 publications

(1 citation statement)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regarding speech synthesis tasks in low-resource scenarios, there are many approaches in recent years that have achieved good results in research related to speaker adaptation [17][18][19][20] and also incorporating cross-speaker style transfer for multi-language TTS [21]. Transfer learning can also help synthesize languages such as small languages where the amount of available training speech data is limited, such as Lombardy dialects [22], Indian Sanskrit [23], Russian [24], and even emotional Mongolian [25].…”

Section: Related Workmentioning

confidence: 99%

Semi-Supervised Learning for Robust Emotional Speech Synthesis with Limited Data

et al. 2023

View full text Add to dashboard Cite

Emotional speech synthesis is an important branch of human–computer interaction technology that aims to generate emotionally expressive and comprehensible speech based on the input text. With the rapid development of speech synthesis technology based on deep learning, the research of affective speech synthesis has gradually attracted the attention of scholars. However, due to the lack of quality emotional speech synthesis corpus, emotional speech synthesis research under low-resource conditions is prone to overfitting, exposure error, catastrophic forgetting and other problems leading to unsatisfactory generated speech results. In this paper, we proposed an emotional speech synthesis method that integrates migration learning, semi-supervised training and robust attention mechanism to achieve better adaptation to the emotional style of the speech data during fine-tuning. By adopting an appropriate fine-tuning strategy, trade-off parameter configuration and pseudo-labels in the form of loss functions, we efficiently guided the learning of the regularized synthesis of emotional speech. The proposed SMAL-ET2 method outperforms the baseline methods in both subjective and objective evaluations. It is demonstrated that our training strategy with stepwise monotonic attention and semi-supervised loss method can alleviate the overfitting phenomenon and improve the generalization ability of the text-to-speech model. Our method can also enable the model to successfully synthesize different categories of emotional speech with better naturalness and emotion similarity.

show abstract