A Text-to-Audiovisual Synthesizer for Indonesian by Morphing Viseme

Arifin, Arifin; Sumpeno, Surya; Hariadi, Mochamad; Haryanto, Hanny

doi:10.15866/irecos.v10i11.7833

Cited by 5 publications

(8 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, the results of the syllable pronunciation visualization look smoother. Because of the pronunciation visualization of displacement phoneme does not occur drastically as in Indonesian Text-to-Audiovisual Synthesis System Based on Phoneme Speech Database [6] [20]. This method is an implementation of the dynamic visualization of phoneme pronunciation.…”

Section: Resultsmentioning

confidence: 99%

“…In this experiment, we also compare the testing results between this system and the system's Indonesian text-to-audiovisual which uses a speech database phoneme-based from our previous study [6]. Fig.…”

Section: Resultsmentioning

confidence: 99%

“…The experiment results that visual speech generated from the dynamic viseme is smoother and more realistic. An Indonesian text-to-audiovisual synthesis resulted from a study conducted by [6], is constructed based on Indonesian static viseme models. A model of static viseme classes is formed based on the results of a clustering process on a dataset of 2D images of mouth movements.…”

Section: Related Workmentioning

confidence: 99%

“…The data dimension that has characteristics that are not essential are removed and not used for the next process. The result of this process is the projection matrix PCA, which is calculated using (6).…”

Section: Formation Of Viseme Modelmentioning

confidence: 99%

See 3 more Smart Citations

Development of Indonesian Text-to-Audiovisual Synthesis System Using Syllable Concatenation Approach to Support Indonesian Learning

Arifin

Sumpeno

Hariadi

et al. 2017

Int. J. Emerg. Technol. Learn.

Self Cite

View full text Add to dashboard Cite

Abstract-This study aims to develop of Indonesian Text-to-Audiovisual synthesis system using a syllable concatenation approach to support Indonesian learning. This system can visualize the syllable pronunciation synchronized with the speech signal so that it can provide a realistic illustration of the articulator movement when each phoneme is pronounced. Syllable concatenation approach is used to realize a realistic visualization by assembling articulation and coarticulation in the form of syllables. In the development of the system, we have recorded speech database in the syllables form which refers to the patterns of syllables in Indonesian. The syllable concatenation approach is used to concatenate viseme of each phoneme, and to form the visualization of syllable pronunciations. It is synchronized with the corresponding speech from the speech database. Evaluation of this system is conducted based on a "lips-reading" of the 10 Indonesian sentences entered into the system. Ratings are based on the degree of correspondence between the syllable pronunciation and the speech produced. Assessment of all respondents is calculated using MOS (Mean Opinion Score). The calculation results show that the Indonesian text-to-audiovisual system has produced the pronunciation visualization more realistic and smoother.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Formation Of Viseme Modelmentioning

confidence: 99%

See 2 more Smart Citations

Development of Indonesian Text-to-Audiovisual Synthesis System Using Syllable Concatenation Approach to Support Indonesian Learning

Arifin

Sumpeno

Hariadi

et al. 2017

Int. J. Emerg. Technol. Learn.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The morphing viseme method is also used in the synthesis of visual speech for Indonesian [13]. The method is used to create the transition animation from one viseme to another.…”

Section: Related Workmentioning

confidence: 99%

A Realistic Visual Speech Synthesis for Indonesian Using A Combination of Morphing Viseme and Syllable Concatenation Approach to Support Pronunciation Learning

Aripin¹,

Haryanto²

2018

Int. J. Emerg. Technol. Learn.

Self Cite

View full text Add to dashboard Cite

This study aims to build a realistic visual speech synthesis for Indonesian so that it can be used to learn Indonesian pronunciation. In this study, We used the combination of morphing viseme and syllable concatenation method. The morphing viseme method is a process of deformation from one viseme to another so that the animation of the mouth shape looks smoother. This method is used to create the transition of animation between viseme. The Syllable Concatenation method is used to assemble viseme based on certain syllable patterns. We built a syllable-based voice database as a basis for synchronization between syllables, speech and viseme models. The method proposed in this study consists of several stages, namely the formation of Indonesian viseme models, designing facial animation character, development of speech database, a synchronization process and subjective testing of the resulting application. Subjective tests were conducted on 30 respondents who assessed the suitability and natural movement of the mouth when uttering the Indonesian texts. The MOS (Mean Opinion Score) method is used to calculate the average of respondents' scores. The MOS calculation results for the criteria of Synchronization and naturalness are 4,283 and 4,107 on the scale of 1 to 5. This result shows that the level of Synchronization and naturalness of the synthesis of visual speech is more realistic. Therefore, the system can display the visualization of phoneme pronunciation to support learning Indonesian pronunciation.

show abstract

A Finite State Machine Model to Determine Syllables of Indonesian Text

Haryanto

Aripin

2019

2019 1st International Conference on Cybernetics and Intelligent System (ICORIS)

View full text Add to dashboard Cite

A Text-to-Audiovisual Synthesizer for Indonesian by Morphing Viseme

Cited by 5 publications

References 0 publications

Development of Indonesian Text-to-Audiovisual Synthesis System Using Syllable Concatenation Approach to Support Indonesian Learning

Development of Indonesian Text-to-Audiovisual Synthesis System Using Syllable Concatenation Approach to Support Indonesian Learning

A Realistic Visual Speech Synthesis for Indonesian Using A Combination of Morphing Viseme and Syllable Concatenation Approach to Support Pronunciation Learning

A Finite State Machine Model to Determine Syllables of Indonesian Text

Contact Info

Product

Resources

About