A Mixed-Lingual Phonological Component Which Drives the Statistical Prosody Control of a Polyglot TTS Synthesis System

Romsdorfer, Harald; Pfister, Beat; Beutler, René

doi:10.1007/978-3-540-30568-2_23

“…e existing studies at home and abroad mostly concentrate on the methods, melodic forms, and tone synergy of computer music creation [22][23][24][25][26]. However, there are very few music melody synthesis models based on artificial neural networks (ANNs).…”

Section: Introductionmentioning

confidence: 99%

Automatic Synthesis Technology of Music Teaching Melodies Based on Recurrent Neural Network

Zhang

¹

,

Li

²

2021

Scientific Programming

View full text Add to dashboard Cite

Computer music creation boasts broad application prospects. It generally relies on artificial intelligence (AI) and machine learning (ML) to generate the music score that matches the original mono-symbol score model or memorize/recognize the rhythms and beats of the music. However, there are very few music melody synthesis models based on artificial neural networks (ANNs). Some ANN-based models cannot adapt to the transposition invariance of original rhythm training set. To overcome the defect, this paper tries to develop an automatic synthesis technology of music teaching melodies based on recurrent neural network (RNN). Firstly, a strategy was proposed to extract the acoustic features from music melody. Next, the sequence-sequence model was adopted to synthetize general music melodies. After that, an RNN was established to synthetize music melody with singing melody, such as to find the suitable singing segments for the music melody in teaching scenario. The RNN can synthetize music melody with a short delay solely based on static acoustic features, eliminating the need for dynamic features. The proposed model was proved valid through experiments.

show abstract

“…In [1], methods for language identification have been described. [2] focus on text analysis, while [3] concentrate on multilingual prosody modeling. Besides, some multilingual TTS systems have been developed.…”

Section: Introductionmentioning

confidence: 99%

Prosody Modification on Mixed-Language Speech Synthesis

Zhang

¹

,

Tao

²

2008

2008 6th International Symposium on Chinese Spoken Language Processing

View full text Add to dashboard Cite

This paper proposes a method to generate natural prosody parameters in Chinese and English mixed-language speech synthesis system which is based on separate Chinese, English, and a small bilingual corpus. Prosodic assimilation of English words to Chinese contexts can be found by observing the bilingual corpus. The most obvious assimilation characteristics are the wider pitch range and the longer duration. A prosody modification model based on this observation is proposed to modify mono-lingual prosody parameters to adapt for mixed-lingual environment. Experiments have proved that more natural mixed-lingual prosody can be generated with our model.

show abstract

“…In [1], methods for language identification have been described. [2] focus on text analysis, while [3] concentrate on multilingual prosody modeling. Besides, some multilingual TTS systems have been developed.…”

Section: Introductionmentioning

confidence: 99%

Prosody Modification on Mixed-language Speech Synthesis

Zhang,

Tao

2008

Int. Symp. On Chinese Spoken Language Processing

0

View full text Add to dashboard Cite

This paper proposes a method to generate natural prosody parameters in Chinese and English mixed-language speech synthesis system which is based on separate Chinese, English, and a small bilingual corpus. Prosodic assimilation of English words to Chinese contexts can be found by observing the bilingual corpus. The most obvious assimilation characteristics are the wider pitch range and the longer duration. A prosody modification model based on this observation is proposed to modify mono-lingual prosody parameters to adapt for mixed-lingual environment. Experiments have proved that more natural mixed-lingual prosody can be generated with our model.

show abstract

A Mixed-Lingual Phonological Component Which Drives the Statistical Prosody Control of a Polyglot TTS Synthesis System

Cited by 4 publications

References 7 publications

Automatic Synthesis Technology of Music Teaching Melodies Based on Recurrent Neural Network

Automatic Synthesis Technology of Music Teaching Melodies Based on Recurrent Neural Network

Prosody Modification on Mixed-Language Speech Synthesis

Prosody Modification on Mixed-language Speech Synthesis

Contact Info

Product

Resources

About